[ENBD] ENBD on 2.4.x

[Peter T. Breuer]

"A month of sundays ago Peter T. Breuer wrote:"
> "A month of sundays ago jona@orac.ensor.org wrote:"
> > Could somebody sanity check me here?  I'm still looking at issues

I'll just report the results of experiments I performed over the
weekend on the 2.4.0 kernel.

1) both kernel plugging and nbd's merge_requests=1 must be set in
   order for the kernel to build large requests. Neither alone is
   sufficient.

   In my code, the default for both is off, and enablable separately.
   In your code possibly the default for both is on, and only
   merge_requests is settable.

   With either of these constraints on, the kernel merged no requests.

   [b] Requested:  102K    (52521) (52476) 8R/104989W      max 1

   The "max 1" is the max number of blocks seen in a request.

2) The kernel can only merge those requests it has on the queue at one
   time, which appears to max at

   [b] Queued:     +0R/0W curr (check 0R/0W) +43R/128W max

   43 read requests and 128 write requests, in my tests (43 is 2/3 of
   64). That's a theoretical limit of 128 blocks in a merged request.

3) Even in favourable circumstances, I only saw the kernel merge at
   most 8 blocks.

   [b] Requested:  92187   (91679) 972R/91215W     max 8

   but this was with my laptop running at 366MHz. When I slowed the cpu
   to 175MHz, I only saw a maximum of 4 blocks in a request. So the
   result depends on cpu speed. A fast cpu and slow i/o may allow
   the kernel to stack more requests between i/0 ops and hence allow it
   to merge more requests.

   I suspect the elevator code in 2.4.1 has been tuned to allow you to
   get higher numbers here. That may be the problem.

4) There appears to be an absolute per device limit on request size
   imposed by the kernel max_sectors[] array. In my code this array is
   set for nbd, but in yours it is not. In your case, the kernel will
   be limited to a maximum of MAX_SECTORS (from ll_rw_blk.c). That's
   an upper limit if I read the code right. At least it is in
   __make_request, which is the default block device function.
   There is a generic_make_request in which they appear to have
   forgotten to do size limiting, but that is not the default function
   on the block devices. However it is called from submit_bh,
   so I am confused!

> > Mar 15 18:40:27 jarney kernel: NBD #1469[0]: nbd_get_req Warning!  Want to write something larger than my buffer 131072
> 
> That was REALLY REAAAAALLY REALLY mysterious. I am adding a max request
> display to the printout too.
> 
> I'll try kernel 2.4.2 this week.

I was running without the scattered PARANOIA size of requests detection
conditionals during my tests. But then I was confident that the impossible
was impossible. The accounting is accurate.

Peter