[ENBD] diskless enbd-client/enbd on initrd

[Rudolph Bott]

Peter T. Breuer wrote:
> "Also sprach Rudolph Bott:"
> 
>>Peter T. Breuer wrote:
>>
>>>"Also sprach Rudolph Bott:"
>>>
>>>>does anyone have experience with enbd-client on an initrd?
>>>
>>>Never ... I suppose it would be possible, but difficult, since a bit of
>>>userspace needs to be set up ... how do you chroot while the client is
>>>still running on the old root?
>>>
>>
>>Well what do you mean by "a bit"?
> 
> 
> I'm not sure, never having done it.  But as you said, you need the
> client daemon running, and you need to mount the device, and you might
> want a bit of space in /var to handle the connection state repository
> (optional).  Can't think of anything else, but I'm sure there is more
> (chroot, sh, blah, blah).
> 
> 
>>btw, I forgot to mention that I'm using enbd-2.4.33pre.tar.gz
> 
> 
> OK.
> 
> 
>>As I mentioned before, the whole thing was working with nbd (v 2.6.0).
> 
> 
> Yes, but my question about how you chroot or umount the original root
> while the clients are running on it stands.
> 
> 
>>Do you have any hints what files/directorys/devices enbd-client might 
>>require to exist in the initrd-tree?
> 
> 
> None.  The error you showed didn't seem to me to be to do with it, but
> with the question I put a couple of times now!  How can you chroot while
> the client daemons are running on the original root?  At the very least
> they have the /dev/nda* entries open, no? That's what the error seemed
> to indicate.
> 
> I'll take a look at your script, but it wouldbe easier if you put a set
> -x in it and thus got info about where the "busy" error occurs. I
> assume it is at or near a chroot or umount!
> 
> Peter
> 
> 

Below you can see how the work is done in the nbd-environment (you can 
find the nbd_root package here: http://borderworlds.dk/projects/)

What it basically does is to read/parse the kernel boot param "nbdroot", 
which contains the host "host:port" information
then (after the fork() call) it runs nbd-client with the extracted 
parameters (assuming that nbd is built into the kernel so that no module 
loading is required).
Then it triggers the mount() call to mount the nb-device under /mnt/ of 
the initrd. There is no chroot or whatsoever; its a simple call to 
pivot_root(), which swaps the "roots" (initrd and rootfs on nbd). The 
nbd-client (and the ramdisk) still reside in memory and everything works 
fine.

Now to the enbd-version...i've already posted the source code in my last 
mail so i'll leave that out for now. The error messages you see are 
generated by the function perror(), which seems to display the last 
error returned by a given function (or probably class of functions, e.g. 
mount, exec etc.).
And since the mount fails (because there is nothing "behind" /dev/nda), 
pivot_root fails, too.

I Hope that clarified some things ;)



-- begin linuxrc.c (which the kernel runs as init process)

#include <sys/mount.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/wait.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <errno.h>
#include <linux/unistd.h>
#include <nbdroot_version.h>

int main () {
   struct stat statbuf;
   char *root, *host, *port;
   int pid, i, j;

   fprintf(stderr, "nbd_root version %s\n", NBDROOT_VERSION);
   pid = getpid();
   if (pid != 1) {
     fprintf(stderr, "My pid is %d but should be 1, please make sure 
that linuxrc gets executed as init.\n", pid);
     return 1;
   }

   if ((root = getenv("nbdroot")) != NULL) {
     fprintf(stderr, "root: %s\n", root);
     j = -1;
     for (i = 0; i < strlen(root); ++i) {
       if (root[i] == ':') {
         j = i;
       }
     }
     if (j == -1) {
       fprintf(stderr, "\"nbdroot\" must be on the form <host>:<port>.\n");
       return 1;
     }
     host = (char *)malloc(strlen(root) + 1);
     strncpy(host, root, j);
     host[j] = 0;
     port = root + j + 1;
   } else {
     fprintf(stderr, "No root specified via the \"nbdroot\" parameter.\n");
     return 1;
   }

   if ((pid = fork())) {
     waitpid(pid,NULL,0);
   } else {
     execl("bin/nbd-client", "nbd-client", host, port, "/dev/nb0", NULL);
     perror("exec");
   }

   if (mount("/dev/nb0", "/mnt", "ext3", MS_RDONLY, "")) {
     perror("mount");
   }
   if (stat("/mnt/initrd", &statbuf)) {
     fprintf(stderr, "The directory /initrd does not exist on the root 
filesystem.\nPlease create it and try again.\n");
     return 1;
   }
   if (pivot_root("/mnt","/mnt/initrd")) {
     perror("pivot_root");
   }
   chdir("/");
   execl("/sbin/init", "init", NULL);
   fprintf(stderr, "Failed to start init :(\n");

   return 1;
}

--end linuxrc.c

-- 
Mit freundlichen Gruessen / with kind regards
  Rudolph Bott

-------------------------------------------------------------------
Megabit Informationstechnik GmbH  Karstr.25  41068 Moenchengladbach
Tel: 02161/308980   mailto:info at megabit.net       ftp://megabit.net
Fax: 02161/3089818  mailto:support at megabit.net   http://megabit.net
-------------------------------------------------------------------