[ENBD] enbd: help needed
Peter Daum
gator_ml at yahoo.de
Wed Oct 17 06:03:32 MDT 2007
Hi,
after some attempts with nbd, I stumbled over enbd, which seemed
(at least in theory) just what I need. Unfortunately, I can
not get it to work even in a pretty minimalistic test scenario.
Here what I did:
- compile enbd-2.4.33
insmod ./enbd.ko
./enbd-server 666 /dev/raid6_12/nbd # lvm volume
./enbd-client localhost:666 -n 4 /dev/nda
The output looks not really like somethings going totally wrong
(attached below is the relevant part of the syslog) but any attempt
to read from /dev/nda fails.
The system is core2duo with linux 2.6.16.54.
I had tried enbd-2.4.34, too, which failed in a different way ;)
If anybody has an idea what is going wrong, any help is appreciated.
Regards,
Peter Daum
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
# cat /proc/nbdinfo
Device a: Open [a] State: verify, rw, enabled, remote invalid, last error 0, lives 0, bp 0
[a] Queued: +0R/0W curr (check 0R/0W) +0R/0W max
[a] Buffersize: 262144 (sectors=512, blocks=64)
[a] Blocksize: 4096 (log=12)
[a] Size: 838860800KB
[a] Blocks: 209715200
[a] Sockets: 4 (+) (+) (+) (*)
[a] Requested: 0 (0) (0) (0) (0) 0R/0W max 0
[a] Despatched: 0 (0) (0) (0) (0) 0R/0W md5 0W (0 eq, 0 ne, 0 dn)
[a] Errored: 306 (0) (0) (0) (0) 0+306
[a] Pending: 0 (0) (0) (0) (0) 0R/0W+0R/0W
[a] B/s now: 0 (0R+0W)
[a] B/s ave: 0 (0R+0W)
[a] B/s max: 0 (0R+0W)
[a] Spectrum:
[a] Kthreads: 0 (0 waiting/0 running/1 max)
[a] Cthreads: 4 (+) (+) (+) (+)
[a] Cpids: 4 (5906) (5907) (5908) (5909)
Device b-p: Closed
# sylog:
2007-10-16 19:59:05 root: insmod:
2007-10-16 19:59:10 ENBD #5525[0]: enbd_init Network Block Device originally by pavel at elf.mj.gts.cz
2007-10-16 19:59:10 ENBD #5526[0]: enbd_init Network Block Device port to 2.0 by ptb at it.uc3m.es
2007-10-16 19:59:10 ENBD #5528[0]: enbd_init Network Block Device move networking to user space by
amarin at it.uc3m.es
2007-10-16 19:59:10 ENBD #5530[0]: enbd_init Enhanced Network Block Device 2.4.30 $Date: 2002/09/17
16:33:22 $ by ptb at it.uc3m.es
2007-10-16 19:59:10 ENBD #5570[0]: enbd_init registered device at major 43
2007-10-16 19:59:10 ENBD #2487[1]: do_enbd_request device not enabled.
2007-10-16 19:59:10 printk: 6 messages suppressed.
2007-10-16 19:59:10 Buffer I/O error on device nda, logical block 17179869056
2007-10-16 19:59:10 Buffer I/O error on device nda, logical block 17179869057
2007-10-16 19:59:10 Buffer I/O error on device nda, logical block 17179869058
2007-10-16 19:59:10 Buffer I/O error on device nda, logical block 17179869059
2007-10-16 19:59:10 Buffer I/O error on device nda, logical block 17179869056
2007-10-16 19:59:10 Buffer I/O error on device nda, logical block 17179869057
2007-10-16 19:59:10 Buffer I/O error on device nda, logical block 17179869058
2007-10-16 19:59:10 Buffer I/O error on device nda, logical block 17179869059
2007-10-16 19:59:10 Buffer I/O error on device nda, logical block 17179869180
2007-10-16 19:59:10 Buffer I/O error on device nda, logical block 17179869181
2007-10-16 19:59:10 ENBD #3045[0]: enbd_set_remote_invalid INVALIDATE drive on ndb
2007-10-16 19:59:10 ENBD #3045[1]: enbd_set_remote_invalid INVALIDATE drive on nda
2007-10-16 19:59:10 ENBD #3045[2]: enbd_set_remote_invalid INVALIDATE drive on ndc
2007-10-16 19:59:10 ENBD #3045[3]: enbd_set_remote_invalid INVALIDATE drive on nde
2007-10-16 19:59:10 ENBD #3045[4]: enbd_set_remote_invalid INVALIDATE drive on ndg
2007-10-16 19:59:10 ENBD #3045[5]: enbd_set_remote_invalid INVALIDATE drive on ndh
2007-10-16 19:59:10 ENBD #3045[6]: enbd_set_remote_invalid INVALIDATE drive on ndd
2007-10-16 19:59:10 ENBD #3045[7]: enbd_set_remote_invalid INVALIDATE drive on ndj
2007-10-16 19:59:10 ENBD #3045[8]: enbd_set_remote_invalid INVALIDATE drive on ndi
2007-10-16 19:59:10 ENBD #3045[9]: enbd_set_remote_invalid INVALIDATE drive on ndf
2007-10-16 19:59:10 ENBD #3045[10]: enbd_set_remote_invalid INVALIDATE drive on ndk
2007-10-16 19:59:10 ENBD #3045[11]: enbd_set_remote_invalid INVALIDATE drive on ndl
2007-10-16 19:59:10 ENBD #3045[12]: enbd_set_remote_invalid INVALIDATE drive on ndn
2007-10-16 19:59:10 ENBD #3045[13]: enbd_set_remote_invalid INVALIDATE drive on ndm
2007-10-16 19:59:10 ENBD #3045[14]: enbd_set_remote_invalid INVALIDATE drive on ndo
2007-10-16 19:59:10 ENBD #3045[15]: enbd_set_remote_invalid INVALIDATE drive on ndp
2007-10-16 19:59:19 root: start server:
2007-10-16 19:59:29 enbd-server: enbd-server: server: will open resources in mode linear 2007-10-16
19:59:29 enbd-server: file: looking for blksize of /dev/raid6_12/nbd with fstat... 2007-10-16
19:59:29 enbd-server: file: blksize of /dev/raid6_12/nbd is 4096 2007-10-16 19:59:29 enbd-server:
file: set final blksize of whole resource to 4096 2007-10-16 19:59:29 enbd-server: file: looking for
size of fd 4 with seek SEEK_END... 2007-10-16 19:59:29 enbd-server: file: set size of fd 4 to
858993459200 2007-10-16 19:59:29 enbd-server: enbd-server: server: set blksize to 4096. 2007-10-16
19:59:29 enbd-server: enbd-server: size of exported file/device is 858993459200B (209715200 blocks)
2007-10-16 19:59:29 enbd-server: enbd-server: server (-2) set new signal handlers 2007-10-16
19:59:51 root: start client:
2007-10-16 20:00:00 enbd-client: enbd-client: client channels is 4 2007-10-16 20:00:00 enbd-client:
enbd-client: client says target 0 is localhost:666 2007-10-16 20:00:00 enbd-client: enbd-client:
client (-1) opened device /dev/nda 2007-10-16 20:00:00 enbd-client: enbd-client: client (-1) opened
NBD device /dev/nda (2b00) 2007-10-16 20:00:00 enbd-client: enbd-client: client (-1) left kernel
bdflush sync boundary at 80% 2007-10-16 20:00:00 enbd-client: enbd-client: client (-1) left kernel
bdflush async boundary at 10% 2007-10-16 20:00:00 enbd-client: enbd-client: client (-1) detaches
from shell 2007-10-16 20:00:00 enbd-client: enbd-client: client (-1) starts introduction sequence on
localhost:666 2007-10-16 20:00:00 enbd-server: enbd-server: server (-2) opened port 666 (socket 1)
for client 127.0.0.1 2007-10-16 20:00:00 enbd-server: nbd-shmem: shmem area total size 139264
2007-10-16 20:00:00 enbd-server: nbd-shmem: shmem hash area starts at offset 4096 2007-10-16
20:00:00 enbd-server: nbd-shmem: shmem hash area size 135168 2007-10-16 20:00:00 enbd-server:
nbd/hash: hash area total size 135168 2007-10-16 20:00:00 enbd-server: nbd/hash: hash buckets 256
2007-10-16 20:00:00 enbd-server: nbd/hash: reduce hash area effective size to 134992 2007-10-16
20:00:00 enbd-server: nbd/gm: pre-seeding gz heap with unit size 65536 2007-10-16 20:00:00
enbd-server: nbd/gm: pre-seeding gz heap with unit size 32768 2007-10-16 20:00:00 enbd-server:
nbd/gm: pre-seeding gz heap with unit size 16384 2007-10-16 20:00:00 enbd-server: nbd/gm:
pre-seeding gz heap with unit size 8192 2007-10-16 20:00:00 enbd-server: nbd/gm: pre-seeding gz heap
with unit size 4096 2007-10-16 20:00:00 enbd-server: nbd/gm: pre-seeding gz heap with unit size 2048
2007-10-16 20:00:00 enbd-server: nbd/gm: pre-seeding gz heap with unit size 1024 2007-10-16 20:00:00
enbd-server: nbd/gm: pre-seeding gz heap with unit size 512 2007-10-16 20:00:00 enbd-server: nbd/gm:
pre-seeding gz heap with unit size 256 2007-10-16 20:00:00 enbd-server: nbd/gm: pre-seeding gz heap
with unit size 64 2007-10-16 20:00:00 enbd-server: nbd/gm: pre-seeding gz heap with unit size 16
2007-10-16 20:00:00 enbd-server: nbd/hash: hash size 4096 header + 130896 data = 134992 2007-10-16
20:00:00 enbd-server: nbd/hash: hash entries initial lo/hi limits set at 2198/2443 entries
2007-10-16 20:00:00 enbd-server: enbd-server: server (-1) set default signal handlers 2007-10-16
20:00:00 enbd-server: enbd-server: server (-1) sent hello ok 2007-10-16 20:00:00 enbd-server:
enbd-server: server (-1) sent passwd ok 2007-10-16 20:00:00 enbd-server: enbd-server: server (-1)
got cliserv magic ok 2007-10-16 20:00:00 enbd-server: enbd-server: server (-1) received id device
2b00 ok 2007-10-16 20:00:00 enbd-server: enbd-server: server (-1) sent size 858993459200 ok
2007-10-16 20:00:00 enbd-server: enbd-server: server (-1) sent sig [gIrCLh] ok 2007-10-16 20:00:00
enbd-server: enbd-server: server (-1) suggested ro flags 0 ok 2007-10-16 20:00:00 enbd-client:
enbd-client: client (-1) got size 858993459200 2007-10-16 20:00:00 enbd-client: enbd-client: client
(-1) got signature [gIrCLh], had [] 2007-10-16 20:00:00 enbd-server: enbd-server: server (-1)
received blksize 1024 ok 2007-10-16 20:00:00 enbd-server: enbd-server: server (-1) sent/negotiated
blksize 4096 ok 2007-10-16 20:00:00 enbd-client: enbd-client: client (-1) negotiated blksize 4096
2007-10-16 20:00:00 enbd-server: enbd-server: server (-1) received pulse_intvl 10 ok 2007-10-16
20:00:00 enbd-server: enbd-server: server (-1) sent/negotiated pulse interval 10 ok 2007-10-16
20:00:00 enbd-client: enbd-client: client (-1) negotiated pulse_intvl 10 2007-10-16 20:00:00
enbd-server: enbd-server: server (-1) agreed 4 channels ok 2007-10-16 20:00:00 enbd-server:
enbd-server: server (-1) selected free port at 667 2007-10-16 20:00:00 enbd-server: enbd-server:
server (-1) posted port 667 ok 2007-10-16 20:00:00 enbd-server: enbd-server: server (-1) manager
started new process group 5901 2007-10-16 20:00:00 enbd-server: enbd-server: server (0) set default
signal handlers 2007-10-16 20:00:00 enbd-server: enbd-server: server (1) set default signal handlers
2007-10-16 20:00:00 enbd-server: enbd-server: server (2) set default signal handlers 2007-10-16
20:00:00 enbd-server: enbd-server: server (3) set default signal handlers 2007-10-16 20:00:00
enbd-server: enbd-server: server (-1) set new signal handlers 2007-10-16 20:00:05 enbd-client:
enbd-client: client (-1) got session port 667 ok 2007-10-16 20:00:05 enbd-client: enbd-client:
client (-1) introduction sequence ends ok 2007-10-16 20:00:05 enbd-client: enbd-client: client (-1)
set sig or passed sigchk OK 2007-10-16 20:00:05 enbd-client: enbd-client: client (-1) set device
size 858993459200 2007-10-16 20:00:05 enbd-client: enbd-client: client (-1) Warning! changing device
blksz from 1024 to 4096 2007-10-16 20:00:05 ENBD #3992[0]: fixup_slot failed to find slot for pid
5900 ioctl MY_NBD_SET_SIG arg (user 43724967) in user addr bfb95360
2007-10-16 20:00:05 enbd-client: enbd-client: client (-1) sets session slots to 0-3 2007-10-16
20:00:05 ENBD #4390[0]: enbd_ioctl cleared show_errs on nda
2007-10-16 20:00:05 enbd-client: enbd-client: client (0) opened device /dev/nda 2007-10-16 20:00:05
enbd-client: enbd-client: client (0) opened socket (6) to localhost:667 2007-10-16 20:00:05
enbd-client: enbd-client: client (-1) launched daemon 0 (5906) for localhost:667 2007-10-16 20:00:05
enbd-client: enbd-client: client (1) opened device /dev/nda 2007-10-16 20:00:05 enbd-client:
enbd-client: client (1) opened socket (6) to localhost:667 2007-10-16 20:00:05 enbd-client:
enbd-client: client (-1) launched daemon 1 (5907) for localhost:667 2007-10-16 20:00:05 enbd-server:
enbd-server: server (3) opened port 667 (socket 8) for client 127.0.0.1 2007-10-16 20:00:05
enbd-server: enbd-server: server (3) sent hello ok 2007-10-16 20:00:05 enbd-server: enbd-server:
server (3) sent passwd ok 2007-10-16 20:00:05 enbd-server: enbd-server: server (3) got cliserv magic
ok 2007-10-16 20:00:05 enbd-server: enbd-server: server (0) opened port 667 (socket 8) for client
127.0.0.1 2007-10-16 20:00:05 enbd-server: enbd-server: server (0) sent hello ok 2007-10-16 20:00:05
enbd-server: enbd-server: server (0) sent passwd ok 2007-10-16 20:00:05 enbd-server: enbd-server:
server (0) got cliserv magic ok 2007-10-16 20:00:05 enbd-server: enbd-server: server (3) sent sig
[gIrCLh] ok 2007-10-16 20:00:05 enbd-server: enbd-server: server (0) sent sig [gIrCLh] ok 2007-10-16
20:00:05 enbd-server: enbd-server: server (3) set new signal handlers 2007-10-16 20:00:05
enbd-client: enbd-client: client (0) read passwd ok from localhost:667 2007-10-16 20:00:05
enbd-client: enbd-client: client (0) got cliserv magic ok from localhost:667 2007-10-16 20:00:05
enbd-client: enbd-client: client (0) got a signature ok from localhost:667 2007-10-16 20:00:05
enbd-client: enbd-client: client (0) set sig or passed sigchk OK 2007-10-16 20:00:05 enbd-server:
enbd-server: server (0) set new signal handlers 2007-10-16 20:00:05 enbd-client: enbd-client: client
(0) begins main loop 2007-10-16 20:00:05 ENBD #2755[0]: enbd_get_device increased socket count on
nda to 1
2007-10-16 20:00:05 ENBD #2611[0]: enbd_enable_and_notify set ENABLED on nda
2007-10-16 20:00:05 enbd-client: enbd-client: client (1) read passwd ok from localhost:667
2007-10-16 20:00:05 enbd-client: enbd-client: client (1) got cliserv magic ok from localhost:667
2007-10-16 20:00:05 enbd-client: enbd-client: client (1) got a signature ok from localhost:667
2007-10-16 20:00:05 enbd-client: enbd-client: client (1) set sig or passed sigchk OK 2007-10-16
20:00:05 ENBD #2755[1]: enbd_get_device increased socket count on nda to 2
2007-10-16 20:00:05 enbd-client: enbd-client: client (1) begins main loop 2007-10-16 20:00:05 ENBD
#880[0]: enbd_open DISK CHECK wanted in open on device nda
2007-10-16 20:00:05 ENBD #4739[0]: enbd_media_changed MEDIA CHANGED called
2007-10-16 20:00:05 ENBD #4745[0]: enbd_media_changed REMOTE CHECK done locally, no remote ioctl!
2007-10-16 20:00:05 ENBD #4843[0]: enbd_revalidate REVALIDATE called on nda
2007-10-16 20:00:05 ENBD #4849[0]: enbd_revalidate asked to revalidate remotely dead drive nda
2007-10-16 20:00:05 ENBD #4843[1]: enbd_revalidate REVALIDATE called on nda
2007-10-16 20:00:05 ENBD #4849[1]: enbd_revalidate asked to revalidate remotely dead drive nda
2007-10-16 20:00:05 nda:<4>printk: 4790 messages suppressed.
2007-10-16 20:00:05 Buffer I/O error on device nda, logical block 0
2007-10-16 20:00:05 Buffer I/O error on device nda, logical block 0
2007-10-16 20:00:05 enbd-client: enbd-client: client (-1) launched daemon 2 (5908) for localhost:667
2007-10-16 20:00:05 unable to read partition table
2007-10-16 20:00:05 ENBD #880[1]: enbd_open DISK CHECK wanted in open on device nda
2007-10-16 20:00:05 ENBD #4739[1]: enbd_media_changed MEDIA CHANGED called
2007-10-16 20:00:05 ENBD #4745[1]: enbd_media_changed REMOTE CHECK done locally, no remote ioctl!
2007-10-16 20:00:05 ENBD #4843[2]: enbd_revalidate REVALIDATE called on nda
2007-10-16 20:00:05 ENBD #4849[2]: enbd_revalidate asked to revalidate remotely dead drive nda
2007-10-16 20:00:05 ENBD #4843[3]: enbd_revalidate REVALIDATE called on nda
2007-10-16 20:00:05 ENBD #4849[3]: enbd_revalidate asked to revalidate remotely dead drive nda
2007-10-16 20:00:05 enbd-client: enbd-client: client (-1) launched daemon 3 (5909) for localhost:667
2007-10-16 20:00:05 enbd-client: enbd-client: client (2) opened device /dev/nda 2007-10-16 20:00:05
enbd-server: enbd-server: server (2) opened port 667 (socket 8) for client 127.0.0.1 2007-10-16
20:00:05 enbd-server: enbd-server: server (2) sent hello ok 2007-10-16 20:00:05 enbd-server:
enbd-server: server (2) sent passwd ok 2007-10-16 20:00:05 enbd-server: enbd-server: server (2) got
cliserv magic ok 2007-10-16 20:00:05 enbd-server: enbd-server: server (2) sent sig [gIrCLh] ok
2007-10-16 20:00:05 enbd-server: enbd-server: server (2) set new signal handlers 2007-10-16 20:00:05
enbd-client: enbd-client: client (2) opened socket (6) to localhost:667 2007-10-16 20:00:05
enbd-client: enbd-client: client (2) read passwd ok from localhost:667 2007-10-16 20:00:05
enbd-client: enbd-client: client (2) got cliserv magic ok from localhost:667 2007-10-16 20:00:05
enbd-client: enbd-client: client (2) got a signature ok from localhost:667 2007-10-16 20:00:05
enbd-client: enbd-client: client (2) set sig or passed sigchk OK 2007-10-16 20:00:05 nda:<6>ENBD
#2755[2]: enbd_get_device increased socket count on nda to 3
2007-10-16 20:00:05 enbd-client: enbd-client: client (2) begins main loop 2007-10-16 20:00:05 Buffer
I/O error on device nda, logical block 0
2007-10-16 20:00:05 Buffer I/O error on device nda, logical block 0
2007-10-16 20:00:05 unable to read partition table
2007-10-16 20:00:05 enbd-client: enbd-client: client (3) opened device /dev/nda 2007-10-16 20:00:05
enbd-client: enbd-client: client (3) opened socket (6) to localhost:667 2007-10-16 20:00:05
enbd-server: enbd-server: server (1) opened port 667 (socket 8) for client 127.0.0.1 2007-10-16
20:00:05 enbd-server: enbd-server: server (1) sent hello ok 2007-10-16 20:00:05 enbd-server:
enbd-server: server (1) sent passwd ok 2007-10-16 20:00:05 enbd-client: enbd-client: client (3) read
passwd ok from localhost:667 2007-10-16 20:00:05 enbd-server: enbd-server: server (1) got cliserv
magic ok 2007-10-16 20:00:05 enbd-client: enbd-client: client (3) got cliserv magic ok from
localhost:667 2007-10-16 20:00:05 enbd-server: enbd-server: server (1) sent sig [gIrCLh] ok
2007-10-16 20:00:05 enbd-client: enbd-client: client (3) got a signature ok from localhost:667
2007-10-16 20:00:05 enbd-client: enbd-client: client (3) set sig or passed sigchk OK 2007-10-16
20:00:05 ENBD #2755[3]: enbd_get_device increased socket count on nda to 4
2007-10-16 20:00:05 enbd-client: enbd-client: client (3) begins main loop 2007-10-16 20:00:05
enbd-server: enbd-server: server (1) set new signal handlers 2007-10-16 20:00:10 ENBD #880[2]:
enbd_open DISK CHECK wanted in open on device nda
2007-10-16 20:00:10 ENBD #4739[2]: enbd_media_changed MEDIA CHANGED called
2007-10-16 20:00:10 ENBD #4745[2]: enbd_media_changed REMOTE CHECK done locally, no remote ioctl!
2007-10-16 20:00:10 ENBD #4843[4]: enbd_revalidate REVALIDATE called on nda
2007-10-16 20:00:10 ENBD #4849[4]: enbd_revalidate asked to revalidate remotely dead drive nda
2007-10-16 20:00:10 ENBD #4843[5]: enbd_revalidate REVALIDATE called on nda
2007-10-16 20:00:10 ENBD #4849[5]: enbd_revalidate asked to revalidate remotely dead drive nda
2007-10-16 20:00:10 nda:<3>Buffer I/O error on device nda, logical block 0
2007-10-16 20:00:10 Buffer I/O error on device nda, logical block 0
2007-10-16 20:00:10 unable to read partition table
2007-10-16 20:00:10 enbd-client: enbd-client: client (-1) opened device /dev/nda 2007-10-16 20:01:35
root: try read:
2007-10-16 20:01:39 ENBD #880[3]: enbd_open DISK CHECK wanted in open on device nda
2007-10-16 20:01:39 ENBD #4739[3]: enbd_media_changed MEDIA CHANGED called
2007-10-16 20:01:39 ENBD #4745[3]: enbd_media_changed REMOTE CHECK done locally, no remote ioctl!
2007-10-16 20:01:39 ENBD #4843[6]: enbd_revalidate REVALIDATE called on nda
2007-10-16 20:01:39 ENBD #4849[6]: enbd_revalidate asked to revalidate remotely dead drive nda
2007-10-16 20:01:39 ENBD #4843[7]: enbd_revalidate REVALIDATE called on nda
2007-10-16 20:01:39 ENBD #4849[7]: enbd_revalidate asked to revalidate remotely dead drive nda
2007-10-16 20:01:39 nda:<3>Buffer I/O error on device nda, logical block 0
2007-10-16 20:01:39 Buffer I/O error on device nda, logical block 0
2007-10-16 20:01:39 unable to read partition table
2007-10-16 20:01:39 Buffer I/O error on device nda, logical block 1
2007-10-16 20:01:39 Buffer I/O error on device nda, logical block 2
2007-10-16 20:01:39 Buffer I/O error on device nda, logical block 3
2007-10-16 20:01:39 Buffer I/O error on device nda, logical block 4
2007-10-16 20:01:39 Buffer I/O error on device nda, logical block 5
2007-10-16 20:01:39 Buffer I/O error on device nda, logical block 6
2007-10-16 20:01:39 Buffer I/O error on device nda, logical block 7
2007-10-16 20:01:39 Buffer I/O error on device nda, logical block 0
More information about the ENBD
mailing list