[ENBD] multiple clients connect to single server device

[Peter T. Breuer]

"Aris Sotiropoulos wrote:"
> In one of his emails to the list, Peter wrote:
> > You can read it from multiple locations, but don't expect to write to
> > it from multiple places and get a lot of sense as a result. Yes, you
> > can do that if you don't cache it on the clients (i.e. you access
> > the nbd on the clients via direct i/o or via a raw device binding),
> > but then you need the application to access the nbd device directly,
> > without going through a file system, because file systems will do their
> > own caching. That's OK for database applications.
> 
> It is said here that it's ok for multiple clients to
> connect to the same server device using NBD, as long as there is
> no client side caching.
> 
> Is this valid also for the standard kernel NBD driver?

I don't know if the standard kernel nbd server allows connects from
multiple clients. If it does, it is (valid).

> Are the only differences between NBD and ENBD the ones I see in
> http://www.it.uc3m.es/ptb/nbd/#Introduction ?

It's quite hard to say. The list isn't complete, but it is
approximately complete.

> How do NBD and ENBD differ in issues concerning multiple clients
> connecting to the same server device using NBD?

I don't know, since I don't know if the standard server even allows it.

> For clients to avoid caching in a situation where multiple
> clients are ENBDed connected to the same server device,
> they could use a special file system (shared disk file system?) like GFS
> (openGFS?).

If you could try it and let me know, I would be grateful! I suspect
that GFS would be confused.

> Are there any others that have the same functionality?

I think GFS is meant to work standalone and unify different resources
on different machines in a single FS.

> How can an application achieve direct I/O to a block device?

Depends on the kernel. It can use O_DIRECT in its open calls (if
supported by the kernel, if buffers are aligned properly, if there are
no bugs, etc.), or it can do i/o instead to a raw device bound to the
target.

> What about raw device binding?

Yes. As above.

> I would appreciate any answers and/or links that could help me
> understand these issues.

So would I!

Peter