[ENBD] Re: ENBD on Debian 2.6.7/8 kernel
P.T. Breuer
ptb at it.uc3m.es
Wed Oct 13 18:11:50 MDT 2004
In article <41615BFB.2080601 at ociweb.com> you wrote:
> Ok. I will look at how the test environment is setup and attempt to
> duplicate it for the non-localhost case.
I gave up on getting a working, stable UML 2.6.8.1 environment and
caved in and rebooted by laptop to 2.6.8.1. It all seems to work fine.
> | Run make test and show the results.
>
>
> storageserver:/usr/src/nbd-2.4.32# make test
> grep: /lib/modules/2.4.26-oci-pii-1/build/include/linux/version.h: No
> such file or directory
> export OPTASYNC="" OPTBLK="-b 1024" OPTCLISSL="" OPTJRNL="" OPTLOCK=""
> OPTMD5SUM="" OPTORDER="-w 500" OPTRAID="-l" OPTRCACHE="-h 10000 "
> OPTRO="" OPTSASYNC="" OPTSIG="-i "NBDabcdefNBD"" OPTSOCK="-n 4"
> OPTSRVSSL="" OPTTIME="-t 120" OPT_REQ_TIMEO="-p 5" BUILD="/tmp"
> VPATH="/tmp" DEVICE="/dev/nda" SERVER="localhost" CLIENT="localhost"
> EXPORT="/tmp/core0 /tmp/core1" BLKSIZ="1024" SRVSIZ="4194304"
> RAHEAD="20" MERGE_REQS="0" CHK_PART="0" ;
> /usr/src/nbd-2.4.32/nbd/enbd-maketest --noreload
> /tmp/core0: No such file or directory
> /tmp/core1: No such file or directory
> 4096+0 records in
> 4096+0 records out
> 4194304 bytes transferred in 0.203615 seconds (20599183 bytes/sec)
> 4096+0 records in
> 4096+0 records out
> 4194304 bytes transferred in 0.203861 seconds (20574324 bytes/sec)
> enbd-server
> delay 5senbd-server 1737: server: will open resources in mode linear
> ~ ?
You got stuck there. Here it works fine (I loaded the module just
previously, with due caution, hence the --noreload). Excuse the
disorder from the cut-and-paste:
betty:/usr/oboe/ptb/lang/c/nbd/nbd-2.4.32% sudo nbd/enbd-maketest --notest --noreload
/tmp/core0: No such file or directory
4096+0 records in
4096+0 records out
4194304 bytes transferred in 0.034312 seconds (122240417 bytes/sec)
enbd-server
delay 5senbd-server 4802: server: will open resources in mode linear
file 4802: looking for size of fd 4 with seek SEEK_END...
file 4802: set size of fd 4 to 4194304
file 4802: looking for size of fd 5 with seek SEEK_END...
file 4802: set size of fd 5 to 4194304
file 4802: checking size of /tmp/core0...
file 4802: check says /tmp/core0 is already sized at 4194304
file 4802: checking size of /tmp/core1...
file 4802: check says /tmp/core1 is already sized at 4194304
enbd-server 4802: size of exported file/device is 8388608
|-enbd-server(4803)
enbd-server 4803: server (-2) locked
/var/state/nbd/server-NBDabcdefNBD.client_ips
enbd-server 4803: server (-2) pinged service enbd-cstatd at
127.0.0.1:5051
enbd-server 4803: with news "notice server-start 3033 127.0.0.1
163.117.139.222 163.117.139.222
quit
"
enbd-server 4803: server (-2) unlocked
/var/state/nbd/server-NBDabcdefNBD.client_ips
enbd-server 4803: server (-2) set new signal handlers
.....
enbd-client
enbd-client 4813: client says target 0 is localhost:3033
enbd-client 4813: client (-1) opened device /dev/nda
enbd-client 4813: client (-1) manager opened NBD device /dev/nda
(2b00)
enbd-client 4813: client (-1) left kernel bdflush sync boundary at
134617824%
enbd-client 4813: client (-1) set kernel bdflush async boundary to 25%
from 1074812608%
enbd-client 4813: client (-1) manager detaches from shell
enbd-client 4814: client (-1) manager made pidfile
/var/run/enbd-client-NBDabcdefNBD.pid with pid 4814
enbd-client 4814: client (-1) starts introduction sequence on localhost:3033
enbd-server 4803: server (-2) opened port 3033 (socket 4) for client 127.0.0.1
nbd-shmem 4815: shmem area total size 139264
nbd-shmem 4815: shmem hash area starts at offset 4096
nbd-shmem 4815: shmem hash area size 135168
nbd/hash 4815: hash area total size 135168
nbd/hash 4815: hash buckets 256
nbd/hash 4815: reduce hash area effective size to 134992
nbd/gm 4815: pre-seeding gz heap with unit size 65536
nbd/gm 4815: pre-seeding gz heap with unit size 32768
nbd/gm 4815: pre-seeding gz heap with unit size 16384
nbd/gm 4815: pre-seeding gz heap with unit size 8192
nbd/gm 4815: pre-seeding gz heap with unit size 4096
nbd/gm 4815: pre-seeding gz heap with unit size 2048
nbd/gm 4815: pre-seeding gz heap with unit size 1024
nbd/gm 4815: pre-seeding gz heap with unit size 512
nbd/gm 4815: pre-seeding gz heap with unit size 256
nbd/gm 4815: pre-seeding gz heap with unit size 64
nbd/gm 4815: pre-seeding gz heap with unit size 16
nbd/hash 4815: hash size 4096 header + 130896 data = 134992
nbd/hash 4815: hash entries initial lo/hi limits set at 2198/2443 entries
enbd-server 4815: server (-1) set default signal handlers
enbd-server 4815: server (-1) sent hello ok
enbd-server 4815: server (-1) sent passwd ok
enbd-server 4815: server (-1) got cliserv magic ok
enbd-server 4815: server (-1) received id device 2b00 ok
enbd-server 4815: server (-1) sent size 8388608 ok
enbd-server 4815: server (-1) sent sig [NBDabcdefNBD] ok
enbd-server 4815: server (-1) suggested ro flags 0 ok
|-enbd-client(4814)
enbd-client 4814: client (-1) got size 8388608
enbd-client 4814: client (-1) got signature [NBDabcdefNBD], had []
enbd-server 4815: server (-1) received blksize 1024 ok
enbd-server 4815: server (-1) sent/negotiated blksize 1024 ok
enbd-client 4814: client (-1) negotiated blksize 1024
enbd-server 4815: server (-1) received pulse_intvl 5 ok
enbd-server 4815: server (-1) sent/negotiated pulse interval 10 ok
enbd-client 4814: client (-1) negotiated pulse_intvl 10
enbd-server 4815: server (-1) agreed 4 channels ok
enbd-server 4815: server (-1) selected free port at 3034
enbd-server 4815: server (-1) posted port 3034 ok
checking 127.0.0.1
enbd-server 4815: server (-1) manager started new process group 4815
enbd-server 4819: server (0) set default signal handlers
enbd-server 4820: server (1) set default signal handlers
enbd-server 4821:enbd-server 4822: server (3) set default signal handlers
enbd-server 4815: server (-1) set new signal handlers
server (2) set default signal handlers
enbd-client 4814: client (-1) got session port 3034 ok
enbd-client 4814: client (-1) introduction sequence ends ok
enbd-client 4814: client (-1) set device size 8388608
enbd-client 4814: <#1684> setdeviceblksize client (-1) Warning! kernel says blksz is 4096 != 1024
enbd-client 4814: client (-1) sets session slots to 0-3
enbd-client 4814: client (-1) manager launched daemon 0 (4823) for localhost:3034
enbd-client 4814: client (-1) manager launched daemon 1 (4824) for localhost:3034
enbd-client 4814: client (-1) manager launched daemon 2 (4825) for localhost:3034
enbd-client 4823: client (0) opened device /dev/nda
enbd-server 4821: server (2) opened port 3034 (socket 9) for client 127.0.0.1
enbd-server 4821: server (2) sent hello ok
enbd-server 4821: server (2) sent passwd ok
enbd-server 4821: server (2) got cliserv magic ok
enbd-server 4821: server (2) sent sig [NBDabcdefNBD] ok
enbd-server 4821: server (2) set new signal handlers
enbd-client 4823: client (0) opened socket 5 to localhost:3034
enbd-client 4823: client (0) read passwd ok from localhost:3034
enbd-client 4823: client (0) got cliserv magic ok from localhost:3034
enbd-client 4823: client (0) got a signature ok from localhost:3034
enbd-client 4814: client (-1) manager launched daemon 3 (4826) for localhost:3034
enbd-client 4823: client (0) begins main loop
enbd-client 4825: client (2) opened device /dev/nda
enbd-server 4822: server (3) opened port 3034 (socket 9) for client 127.0.0.1
enbd-server 4822: server (3) sent hello ok
enbd-server 4822: server (3) sent passwd ok
enbd-server 4822: server (3) got cliserv magic ok
enbd-server 4822: server (3) sent sig [NBDabcdefNBD] ok
enbd-server 4822: server (3) set new signal handlers
enbd-client 4825: client (2) opened socket 5 to localhost:3034
enbd-client 4825: client (2) read passwd ok from localhost:3034
enbd-client 4825: client (2) got cliserv magic ok from localhost:3034
enbd-client 4825: client (2) got a signature ok from localhost:3034
enbd-client 4825: client (2) begins main loop
enbd-client 4824: client (1) opened device /dev/nda
enbd-server 4820: server (1) opened port 3034 (socket 9) for client 127.0.0.1
enbd-server 4820: server (1) sent hello ok
enbd-server 4820: server (1) sent passwd ok
enbd-client 4824: client (1) opened socket 5 to localhost:3034
enbd-client 4824: client (1) read passwd ok from localhost:3034
enbd-server 4820: server (1) got cliserv magic ok
enbd-server 4820: server (1) sent sig [NBDabcdefNBD] ok
enbd-server 4820: server (1) set new signal handlers
enbd-client 4824: client (1) got cliserv magic ok from localhost:3034
enbd-client 4824: client (1) got a signature ok from localhost:3034
enbd-client 4824: client (1) begins main loop
enbd-client 4826: client (3) opened device /dev/nda
enbd-server 4819: server (0) opened port 3034 (socket 9) for client 127.0.0.1
enbd-server 4819: server (0) sent hello ok
enbd-server 4819: server (0) sent passwd ok
enbd-server 4819: server (0) got cliserv magic ok
enbd-server 4819: server (0) sent sig [NBDabcdefNBD] ok
enbd-server 4819: server (0) set new signal handlers
enbd-client 4826: client (3) opened socket 5 to localhost:3034
enbd-client 4826: client (3) read passwd ok from localhost:3034
enbd-client 4826: client (3) got cliserv magic ok from localhost:3034
enbd-client 4826: client (3) got a signature ok from localhost:3034
enbd-client 4826: client (3) begins main loop
betty:/usr/oboe/ptb/lang/c/nbd/nbd-2.4.32% enbd-client 4814: client
(-1) opened device /dev/nda
I didn't actually run the test, since I wanted to see it setup first
(that was the --notest). I'll try it now ...
betty:/usr/oboe/ptb/lang/c/nbd/nbd-2.4.32% sudo /tmp/enbd-test /dev/nda
/dev/nda has 8388608 bytes in 8192 blocks of 1024 bytes each
flushing buffers..done
writing....5%....10%....15%....20%....25%....30%....35%....40%....45%....50%....55%....60%....65%....70%....75%....80%....85%....90%....95%....done
test 1 success: 0 incorrect blocks
flushing buffers..nbd/hash 4822: hash saturation reached at 2337 entries
nbd/hash 4822: hash hiwater set at 2337/2337
nbd/hash 4822: hash lowater set at 2104/2337
done
flushing buffers..done
reading....5%....10%....15%....20%....25%....30%....35%....40%....45%....50%....55%....60%....65%....70%....75%....80%....85%....90%....95%....done
test 2 success: 0 incorrect blocks
flushing buffers..done
flushing buffers..done
writing....5%....10%....15%....20%....25%....30%....35%....40%....45%....50%....55%....60%....65%....70%....75%....80%....85%....90%....95%....done
test 3 success: 0 incorrect blocks
flushing buffers..done
flushing buffers..done
reading....5%....10%....15%....20%....25%....30%....35%....40%....45%....50%....55%....60%....65%....70%....75%....80%....85%....90%....95%....done
test 4 success: 0 incorrect blocks
flushing buffers..done
flushing buffers..done
seeking and writing....5%....10%....15%....20%....25%....30%....35%....40%....45%....50%....55%....60%....65%....70%....75%....80%....85%....90%....95%....done
flushing buffers..done
reading and checking....5%....10%....15%....20%....25%....30%....35%....40%....45%....50%....55%....60%....65%....70%....75%....80%....85%....90%....95%....
(1000KB written in 0.005719s, 170.757563 MB/s, 349711.488022 seeks/s)
(1000KB read in 0.003521s, 277.353735 MB/s)done
test 5 success: 0 incorrect blocks
flushing buffers..done
betty:/usr/oboe/ptb/lang/c/nbd/nbd-2.4.32%
And that's that. It was horribly slow. I should have set
merge_requests. Or maybe it's a new kernel scheduler playing funny. But
it certainly works.
Since we MUST have the same kernel code, any difference is due to user
space development environments. I'll try a few more platforms, if I
can. But it's userspace if you have a problem.
I think!
Try reproducing the above results.
Peter
More information about the ENBD
mailing list