[ENBD] can't write to nb device - nbd_get_req blocking req
Peter T. Breuer
enbd@lists.community.tummy.com
Thu, 24 Apr 2003 22:06:26 +0200 (MET DST)
"A month of sundays ago John Repass wrote:"
> I have been experimenting with enbd 2.4.31 on both debian stable and
> unstable distributions with 2.4.20 and 2.4.18 kernels, and every time I
> begin to access the nd device the same thing happens. The client locks
> up trying to write anything:
It looks strange - can you run the make test?
> enbd-test /dev/nda
> /dev/nda has 435907584 bytes in 425691 blocks of 1024 bytes each
437MB?
> flushing buffers..done
> writing....5%....10%....15%....20%....25%....30%....35%....40%....45%....50%....55%....60%....65%....70%....75%....80%....85%....90%....95%....done
> test 1 succeeded: 0 incorrect blocks
> flushing buffers..
Well, I guess there is a problem in the driver then. Can you prove to me
it is loaded and happy? Show me the kernel messages and /proc/nbdinfo.
> ------
> and it hangs forever until echo 1 > /proc/nbdinfo & killing the client
that's evil .. should be echo 0 !!
> --------
> cat /proc/nbdinfo
> Device a: Open
> [a] State: signed, rw, enabled, validated, plug, acct, last error
> 0, lives 0, bp 0
> [a] Queued: +0R/64W curr (check 0R/64W) +4R/64W max
> [a] Buffersize: 262144 (sectors=512, blocks=256)
> [a] Blocksize: 1024 (log=10)
> [a] Size: 425691KB
> [a] Blocks: 425691
> [a] Groups: 1 (0)
> [a] Sockets: 1 (*)
> [a] Requested: 74 (10) 8R/66W max 1
> [a] Despatched: 10 (10) 8R/2W md5 0W (0 eq, 0 ne, 0 dn)
It's received requests and sent some out.
> [a] Errored: 0 (0) 0+0
> [a] Pending: 2 (0) 0R/2W+0R/64W
And has two pending. SO it looks happy. Looks like a dead client to me.
> [a] B/s now: 0 (0R+0W)
> [a] B/s ave: 0 (0R+0W)
> [a] B/s max: 24.0K (8.00KR+21.0KW)
> [a] Spectrum: 100%1
> [a] Kthreads: 0 (0 waiting/0 running/1 max)
> [a] Cthreads: 0 (-)
> [a] Cpids: 0 (323)
No, dead server. Why are you running with only one channel, anyway?
Shouldn't it be 2 or 4?
> Device b-p: Closed
>
> -----------
> I've tried making a filesystem on the raw device and then mounting it,
There's no point in this.
> which works, and I can touch files on the mounted device but if I try to
> copy even a very small text file I get the same error. I've tried
I don't understand. Are you running a test? Please run make test.
> running the server with the -a option and writing very small amounts of
Please run make test. When that works, then worry about why your setup
does not work. I don't quite understand what your setup is yet!
> data: cpio if=/dev/null of=/dev/nda bs=1096 count=10 will work
> sometimes but anything larger that 500 causes the same condition.
>
> Here are relevant kernel log entries from the enbd-test above:
> ------------------
> Apr 24 12:43:09 porto kernel: ENBD enbd.c #8439[0]: nbd_init registered
> device at major 43
> Apr 24 12:44:15 porto kernel: ENBD enbd.c #1279[0]: nbd_open setup device 0
> Apr 24 12:44:20 porto kernel: ENBD enbd.c #4670[0]: nbd_set_sock
> increased socket count to 1
> Apr 24 12:44:20 porto kernel: ENBD enbd.c #4674[0]: nbd_set_sock
> increased group count to 1
> Apr 24 12:44:20 porto kernel: ENBD enbd.c #4001[0]:
> repoint_preferred_group reset active group to 0
> Apr 24 12:44:20 porto kernel: ENBD enbd.c #3962[0]: nbd_enable enabled
> device nda
> Apr 24 12:44:20 porto kernel: ENBD enbd.c #1177[0]:
> nbd_reread_partitions partition check on device nda
> Apr 24 12:44:20 porto kernel: nda:<1>ENBD enbd.c #3103[0]: nbd_get_req
> validation req for sector 0 seen OK!
> Apr 24 12:44:20 porto kernel: ENBD enbd.c #3111[0]: nbd_get_req blocking
> req on
> sector 2 while waiting for sector 0!
> Apr 24 12:44:21 porto kernel: ENBD enbd.c #3111[1]: nbd_get_req blocking
> req on
> sector 2 while waiting for sector 0!
Oh - this is an attempt to read the partition table.
> Apr 24 12:44:21 porto kernel: ENBD enbd.c #3111[2]: nbd_get_req blocking
> req on
> sector 2 while waiting for sector 0!
> Apr 24 12:44:22 porto kernel: ENBD enbd.c #3111[3]: nbd_get_req blocking
> req on
> sector 2 while waiting for sector 0!
> Apr 24 12:44:22 porto kernel: ENBD enbd.c #3111[4]: nbd_get_req blocking
> req on
> sector 2 while waiting for sector 0!
> Apr 24 12:44:23 porto kernel: ENBD enbd.c #3111[5]: nbd_get_req blocking
> req on
> sector 2 while waiting for sector 0!
> Apr 24 12:44:23 porto kernel: ENBD enbd.c #3111[6]: nbd_get_req blocking
> req on
> sector 2 while waiting for sector 0!
> Apr 24 12:44:24 porto kernel: ENBD enbd.c #3111[7]: nbd_get_req blocking
> req on
> sector 2 while waiting for sector 0!
> Apr 24 12:44:24 porto kernel: ENBD enbd.c #3111[8]: nbd_get_req blocking
> req on
> sector 2 while waiting for sector 0!
> Apr 24 12:44:25 porto kernel: ENBD enbd.c #3111[9]: nbd_get_req blocking
> req on
> sector 2 while waiting for sector 0!
> Apr 24 12:44:25 porto kernel: ENBD enbd.c #3111[10]: nbd_get_req
> blocking req on sector 2 while waiting for sector 0!
Yes - it has a problem reading the partiton table. Can you comment out
the grok_partitions call in the driver?
> Apr 24 12:44:26 porto kernel: ENBD enbd.c #3111[11]: nbd_get_req
> blocking req on sector 2 while waiting for sector 0!
> Apr 24 12:44:26 porto kernel: ENBD enbd.c #3111[12]: nbd_get_req
> blocking req on sector 2 while waiting for sector 0!
> Apr 24 12:44:27 porto kernel: ENBD enbd.c #3111[13]: nbd_get_req
> blocking req on sector 2 while waiting for sector 0!
> Apr 24 12:44:27 porto kernel: ENBD enbd.c #3111[14]: nbd_get_req
> blocking req on sector 2 while waiting for sector 0!
> Apr 24 12:44:28 porto kernel: ENBD enbd.c #3111[15]: nbd_get_req
> blocking req on sector 2 while waiting for sector 0!
> Apr 24 12:44:28 porto kernel: ENBD enbd.c #3111[16]: nbd_get_req
> blocking req on sector 2 while waiting for sector 0!
> Apr 24 12:44:29 porto kernel: ENBD enbd.c #3111[17]: nbd_get_req
> blocking req on sector 2 while waiting for sector 0!
> Apr 24 12:44:29 porto kernel: ENBD enbd.c #3111[18]: nbd_get_req
> blocking req on sector 2 while waiting for sector 0!
> Apr 24 12:44:30 porto kernel: ENBD enbd.c #3117[0]: nbd_get_req gave up
> waiting
> for validation req!
Well, it correctly decided to give up!
> Apr 24 12:44:30 porto kernel: ENBD enbd.c #3111[19]: nbd_get_req
> blocking req on sector 4 while waiting for sector 0!
> Apr 24 12:44:30 porto kernel: ENBD enbd.c #3111[20]: nbd_get_req
> blocking req on sector 4 while waiting for sector 0!
> Apr 24 12:44:31 porto kernel: ENBD enbd.c #3111[21]: nbd_get_req
> blocking req on sector 4 while waiting for sector 0!
> Apr 24 12:44:31 porto kernel: ENBD enbd.c #3111[22]: nbd_get_req
> blocking req on sector 4 while waiting for sector 0!
> Apr 24 12:44:32 porto kernel: ENBD enbd.c #3111[23]: nbd_get_req
> blocking req on sector 4 while waiting for sector 0!
> Apr 24 12:44:32 porto kernel: ENBD enbd.c #3111[24]: nbd_get_req
> blocking req on sector 4 while waiting for sector 0!
> Apr 24 12:44:33 porto kernel: ENBD enbd.c #3111[25]: nbd_get_req
> blocking req on sector 4 while waiting for sector 0!
> Apr 24 12:44:33 porto kernel: ENBD enbd.c #3111[26]: nbd_get_req
> blocking req on sector 4 while waiting for sector 0!
> Apr 24 12:44:34 porto kernel: ENBD enbd.c #3111[27]: nbd_get_req
> blocking req on sector 4 while waiting for sector 0!
> Apr 24 12:44:34 porto kernel: ENBD enbd.c #3111[28]: nbd_get_req
> blocking req on sector 4 while waiting for sector 0!
> Apr 24 12:44:35 porto kernel: ENBD enbd.c #3111[29]: nbd_get_req
> blocking req on sector 4 while waiting for sector 0!
> Apr 24 12:44:35 porto kernel: ENBD enbd.c #3111[30]: nbd_get_req
> blocking req on sector 4 while waiting for sector 0!
> Apr 24 12:44:36 porto kernel: ENBD enbd.c #3111[31]: nbd_get_req
> blocking req on sector 4 while waiting for sector 0!
> Apr 24 12:44:36 porto kernel: ENBD enbd.c #3111[32]: nbd_get_req
> blocking req on sector 4 while waiting for sector 0!
> Apr 24 12:44:37 porto kernel: ENBD enbd.c #3111[33]: nbd_get_req
> blocking req on sector 4 while waiting for sector 0!
> Apr 24 12:44:37 porto kernel: ENBD enbd.c #3111[34]: nbd_get_req
> blocking req on sector 4 while waiting for sector 0!
> Apr 24 12:44:38 porto kernel: ENBD enbd.c #3111[35]: nbd_get_req
> blocking req on sector 4 while waiting for sector 0!
> Apr 24 12:44:38 porto kernel: ENBD enbd.c #3111[36]: nbd_get_req
> blocking req on sector 4 while waiting for sector 0!
> Apr 24 12:44:39 porto kernel: ENBD enbd.c #3111[37]: nbd_get_req
> blocking req on sector 4 while waiting for sector 0!
> Apr 24 12:44:39 porto kernel: ENBD enbd.c #3117[1]: nbd_get_req gave up
> waiting
> for validation req!
But why twice?
> Apr 24 12:44:39 porto kernel: ENBD enbd.c #3111[38]: nbd_get_req
> blocking req on sector 6 while waiting for sector 0!
> Apr 24 12:44:40 porto kernel: ENBD enbd.c #3111[39]: nbd_get_req
> blocking req on sector 6 while waiting for sector 0!
> Apr 24 12:44:40 porto kernel: ENBD enbd.c #3111[40]: nbd_get_req
> blocking req on sector 6 while waiting for sector 0!
> Apr 24 12:44:41 porto kernel: ENBD enbd.c #3111[41]: nbd_get_req
> blocking req on sector 6 while waiting for sector 0!
> Apr 24 12:44:41 porto kernel: ENBD enbd.c #3111[42]: nbd_get_req
> blocking req on sector 6 while waiting for sector 0!
> Apr 24 12:44:45 porto kernel: ENBD enbd.c #3111[50]: nbd_get_req
> blocking req on sector 6 while waiting for sector 0!
> Apr 24 12:44:46 porto kernel: ENBD enbd.c #3111[51]: nbd_get_req
> blocking req on sector 6 while waiting for sector 0!
> Apr 24 12:44:46 porto kernel: ENBD enbd.c #3111[52]: nbd_get_req
> blocking req on sector 6 while waiting for sector 0!
> Apr 24 12:44:47 porto kernel: ENBD enbd.c #3111[53]: nbd_get_req
> blocking req on sector 6 while waiting for sector 0!
> Apr 24 12:44:47 porto kernel: ENBD enbd.c #3111[54]: nbd_get_req
> blocking req on sector 6 while waiting for sector 0!
> Apr 24 12:44:48 porto kernel: ENBD enbd.c #3111[55]: nbd_get_req
> blocking req on sector 6 while waiting for sector 0!
> Apr 24 12:44:48 porto kernel: ENBD enbd.c #3111[56]: nbd_get_req
> blocking req on sector 6 while waiting for sector 0!
> Apr 24 12:44:49 porto kernel: ENBD enbd.c #3117[2]: nbd_get_req gave up
> waiting
> for validation req!
I would guess that this is a consequence of running with one channel.
Please run with two or four.
> Apr 24 12:47:12 porto kernel: ENBD enbd.c #1689[0]: nbd_rollback
> rollback req f7e2d800, type 1, sector 4, 1 blks, 2 sectors from slot 0!
> Apr 24 12:48:12 porto kernel: ENBD enbd.c #1689[1]: nbd_rollback
> rollback req f7e2d600, type 1, sector 6, 1 blks, 2 sectors from slot 0!
> Apr 24 12:50:04 porto kernel: ENBD enbd.c #7251[0]: nbd_read_proc :
> altered queued count to 64 from 62
>
> --------------------------
>
> Thank you for any help getting this to work.
The driver is happy, more or less.
Can you run make test and let me know how that goes?
Peter