[SOLVED] How to NFS boot?

AlmostSurelyRob · 2012-10-08 14:18:18

Dear All,

I am trying to perform something similar to the setup presented here:
Diskless network boot

I have a virtual machine which I want to use in one of my projects but since the machine will be deployed on couple of computers and the disk space of these compters is limited I wanted to use NFS for root.

Server side:
According to the article I create on the server 138.250.86.221 an nfs export ofvm-root. I install system image there (base and base-devel). I believe I do not have to play with DNS as the computer has a fixed address in the local network. I verify that that the host machine can mount my exported folder.

Client side:
On my virtual machine I install mkinitcpio-nfs-utils and of course nfs-utils. I add net to HOOKS in /etc/mkinitcpio.conf and issue

mkinitcpio -p linux

to regenerate initramfs following what I have read in mkinitcpio article.

Finally, in my grub menu I hit e and edit the kernel line as suggested in the article on kernel parameters

I tried addings

nfsroot=138.250.86.221:/ofvm-root

and

root=/dev/nfs nfsroot=138.250.86.221:/ofvm-root

The first one booted using local root and the other one didn't boot at all giving an error:

ERROR: device '/dev/nfs' not found. Skipping fsck.
ERROR: Unable to find root device '/dev/nfs'.

Could you please advise what I am possibly missing or doing wrong? Please let me know. Are there any more links on Arch NFS boot setup?

Last edited by AlmostSurelyRob (2012-10-13 11:18:30)

nomorewindows · 2012-10-08 16:04:25

You don't need half the hooks in /etc/mkinitcpio.conf:
You only need : base, udev, net, autodetect. The rest of it (disks, etc.) will mount after root NFS mounts.
Your root fs comes in a directory under /srv/tftp with the current install of tftp-hpa (on the server).
You also need NFS in modules.
If you are using PXE you need to follow the whole guidelines with the whole appended line: root=/dev/nfs rootfstype=nfs nfsroot=x.x.x.x:/srv/tftp/wherever,v3,rsize=16384,wsize=16384 ip=::::::dhcp This whole line is passed onto the kernel to boot the diskless workstation.
You aso have to have your tftp in /etc/exports.
I'd say review the instructions and try again.

Last edited by nomorewindows (2012-10-08 16:06:23)

AlmostSurelyRob · 2012-10-09 10:26:07

Thanks for the reply. I have probabl forgot to put nfs into MODULES. I think I have made some progress and it seems to be a step away now, but the client is complaining that the the permission for the NFS aren't right.

mount call failed - server replied: Permission denied.
ERROR: Failed to mount the real root device.

The entry in server's /etc/exprts is:

/ofvm-root       *(rw,fsid=0,no_root_squash,no_subtree_check)

Also I don't quite understand why I am doing this dnsmasq setup. If I know the IP address of my nfs server is it really a necessary step? Another question is whether TFTP should be running on the server side or will it be activated. From the comments I understand that dnsmasq has an in-built TFTP service. Is that right?

Last edited by AlmostSurelyRob (2012-10-09 10:26:28)

nomorewindows · 2012-10-09 15:36:14

TFTP/dnsmasq has to be enabled along with NFS-server and DHCP and the rest of it.
I don't use dnsmasq myself. I use the separate services.
Your export is going to be under /srv/tftp/ofvm-root. You can create a directory under /srv/tftp called ofvm-root and then mount your partition here. Technically, the pxelinux/syslinux can go directly under the /srv/tftp directory and then leave the tftp directory for the storage of the actual root device under NFS (they are two separate services, but can be located in the same directory for ease of finding). When you reference the kernel file it will also be located under the tftp in relation to the tftp directory. It's just easier to have it all in one place. You can tell pxelinux.cfg/default to locate the root device elsewhere, it'll work. In fact, you can have the kernel/pxelinux files on the tftp server on one machine and point the root device to another machine on the network. That's what's nice about *NIX. Total flexibility.

AlmostSurelyRob · 2012-10-09 17:43:38

Thanks. Yes, I understand that these can be separate services, but I was convinced I don't need TFTP at all - just NFS. I also appreciate the flexibility of the system, I am probably just baffled by a number of options.

Interestingly, the article I quoted at the beginning got a complete rewrite over tha past two days. It looks quite promising and I will it give it a go.

Please let me post my /etc/dnsmasq.conf for I believe it is therein that my blunder lies. Only the uncommented entries are quoted.

port = 0
interface=eth0
bind-interfaces
dhcp-range=0.0.0.0,255.250.87.0,12h
dhcp-boot=/ofvm-root/boot/pxelinux.0
dhcp-option-force=208,f1:00:74:7e
dhcp-option-force=209,boot/pxelinux.cfg/default
dhcp-option-force=210,/ofvm-root/
enable-tftp
tftp-root=/srv/tftp/ofvm-root

I have now moved the image of the system to /srv/tftp/ofvm-root. I was playing with the range because of the permission problems.

nomorewindows · 2012-10-09 18:13:37

AlmostSurelyRob wrote:

Thanks. Yes, I understand that these can be separate services, but I was convinced I don't need TFTP at all - just NFS. I also appreciate the flexibility of the system, I am probably just baffled by a number of options.
Interestingly, the article I quoted at the beginning got a complete rewrite over tha past two days. It looks quite promising and I will it give it a go.
Please let me post my /etc/dnsmasq.conf for I believe it is therein that my blunder lies. Only the uncommented entries are quoted.
port = 0
interface=eth0
bind-interfaces
dhcp-range=0.0.0.0,255.250.87.0,12h
dhcp-boot=/ofvm-root/boot/pxelinux.0
dhcp-option-force=208,f1:00:74:7e
dhcp-option-force=209,boot/pxelinux.cfg/default
dhcp-option-force=210,/ofvm-root/
enable-tftp
tftp-root=/srv/tftp/ofvm-root
I have now moved the image of the system to /srv/tftp/ofvm-root. I was playing with the range because of the permission problems.

The dhcp options will point to the directory from the tftp-root which I think should just be /srv/tftp. From there the dhcp options will add ofvm-root to /srv/tftp and should work from there. Port=0 I'm not sure about, since I'm not using dnsmasq. Dhcp-range looks like it might be a little funny, not exactly sure about the syntax.

AlmostSurelyRob · 2012-10-10 16:42:50

Right, now I am following this:

Network Installation Guide

I have removed the old image and started from scratch and I am still using NFS. I am afraid that I may have a mistake in DHCP configuration, but again I would like to ask is it important? For now I switched off dhcpd on my computer.

Moroever I am using qemu-kvm to boot it as a "debug mode". NFS mounts okay. On the client computer I issue

qemu-kvm -kernel drives/ofvm-root/boot/vmlinuz-linux -initrd drives/ofvm-root/boot/initramfs-linux.img -append "root=/dev/nfs rootfstype=nfs nfs_root=138.250.86.221:/mnt/arch,v4,rsize=16384,wsize=16384 ip=:::::eth0:dhcp"

and I get a very disapointing error about wrong NFS parameters.

ERROR: Unable to mount root filesystem over NFS: wrong parameters.

The error is "disapointing" because when using VirtualBox I was getting it only when I mistyped something in grub boot menu, so I am afraid there might be also a typo in the parameters that I append. Please let me know if you can see anything that I am doing wrong.

Last edited by AlmostSurelyRob (2012-10-10 16:43:16)

AlmostSurelyRob · 2012-10-12 14:14:27

I've just realised that I can look up messages.log in /var/logs. There's a hint there, but I still don't know how to take advantage of it. When I try to boot from VirtualBox I get:

Oct 12 13:03:05 frankfurt rpc.mountd[4517]: refused mount request from 138.250.81.171 for /mnt/arch (/mnt/arch): illegal port 39531

Also I don't understand why qemu-kvm complains about parameters.

AlmostSurelyRob · 2012-10-12 15:38:11

Ok.. progress!

Two things:
1) It turns out that it is a network connection problem due to virtual machine software I am using. NFS cannot find my virtual machines. When a system is booted from a real machine it boots just fine. I guess this entitles the thread to [SOLVED] status, but please let me wait with this for a while...

2) There was of course an error in qemu-kvm nfsroot goes without an underscore which I was putting.

nomorewindows · 2012-10-12 16:04:33

The emulator likes to address your machine as the address it gives from its' dhcp server. Usually your host machine is at 10.0.2.2

AlmostSurelyRob · 2012-10-12 17:11:36

Right, I've just done a test on VMWare Player which is the target host application anyway and... it works fine (fanfare!).

I would just like to clarify three points:
1) What am I actually doing here? The guide is a little confusing in one place:

You will need a DHCP server to setup networking, as well as a TFTP server to transfer the boot image (a requirement of all PXE option roms). Unlike PXE which is designed for quick-and-dirty/temporary setups to boot installation media, here you're actually doing a network installation, which is more-permanent, and thus calls for doing things properly.

and provides a pxelinux configuration... I am not sure. I followed the guide exactly. Am I running on a network installation or on PXE?

2) How should I configure my client to make it as small as possible? What should be on it? Is it just boot partition, kernel and a bootloader?

3) The client I used froze on shutdown after closing network. Presumably this will vanish when I apply the advice from section Late boot networking

nomorewindows · 2012-10-12 20:20:51

AlmostSurelyRob wrote:

Right, I've just done a test on VMWare Player which is the target host application anyway and... it works fine (fanfare!).
I would just like to clarify three points:
1) What am I actually doing here? The guide is a little confusing in one place:
You will need a DHCP server to setup networking, as well as a TFTP server to transfer the boot image (a requirement of all PXE option roms). Unlike PXE which is designed for quick-and-dirty/temporary setups to boot installation media, here you're actually doing a network installation, which is more-permanent, and thus calls for doing things properly.
and provides a pxelinux configuration... I am not sure. I followed the guide exactly. Am I running on a network installation or on PXE?
2) How should I configure my client to make it as small as possible? What should be on it? Is it just boot partition, kernel and a bootloader?
3) The client I used froze on shutdown after closing network. Presumably this will vanish when I apply the advice from section Late boot networking

1) You aren't just installing an operating system, you are actually using a diskless workstation.
2) You can just install the base packages, that will be minimal.
3) The latest kernel seems to stop on reboot instead of rebooting.

AlmostSurelyRob · 2012-10-13 11:21:16

Thanks again for your attention and guidance. I marked the post as solved.

Last edited by AlmostSurelyRob (2012-10-13 11:21:30)

AlmostSurelyRob · 2012-10-17 10:15:23

Alas, I must report a regress. Due to problems with X configuration I had to redo initramfs and the system no longer boots. It replies with an error "mount: protocol not supported".

Two things:
1) I do not really understand which initramfs is being used on boot. On the server side there is /mnt/arch/boot and it contains initramfs based on 3.6.2 kernel. However, when I boot using my VM client I can see that it is 3.5.6 kernel that is being used (uname -a or the login prompt).
2) Because of the problems with X I decided I want to upgrade the client to 3.6.2 and redo my initramfs. I started the installation process anew. Using pacstrap I installed only base and mkinitcpio-nfs-utils grub-bios and initscripts (in another attempt I also tried adding nfs-utils). Then I rebooted to the vm-local root. This worked fine. The kernel was already at 3.6.2 so I've made changes to mkinitcpio.conf: MODULES="nfs" and added "net" to HOOKS. I run

mkinitcpio -p linux

All went fine. Then I went to my VMPlayer and booted. In grub I added

root=/dev/nfs 138.250.86.221:/mnt/arch ip=::::::dhcp

to kernel parameters. It found the compter, but mount announced that it does not support this protocol.

By the way, the client with initramfs based on 3.5.6 works with the above line. I must be doing something seriously wrong - I would appreciate sp,e advice. Also, is it all right if I mark the post back as "UNSOLVED"?

Last edited by AlmostSurelyRob (2012-10-17 10:15:46)

AlmostSurelyRob · 2012-10-18 08:56:45

Looks like I was simply unlucky. In this post the same error was encountered and solved. The upgrade to kernel 3.6.2 nfs module has changed to nfsv3 module. After adding nfsv4 to modules in mkinitcpio.conf the image boots fine. The topic stays solved.

Arch Linux

#1 2012-10-08 14:18:18

[SOLVED] How to NFS boot?

#2 2012-10-08 16:04:25

Re: [SOLVED] How to NFS boot?

#3 2012-10-09 10:26:07

Re: [SOLVED] How to NFS boot?

#4 2012-10-09 15:36:14

Re: [SOLVED] How to NFS boot?

#5 2012-10-09 17:43:38

Re: [SOLVED] How to NFS boot?

#6 2012-10-09 18:13:37

Re: [SOLVED] How to NFS boot?

#7 2012-10-10 16:42:50

Re: [SOLVED] How to NFS boot?

#8 2012-10-12 14:14:27

Re: [SOLVED] How to NFS boot?

#9 2012-10-12 15:38:11

Re: [SOLVED] How to NFS boot?

#10 2012-10-12 16:04:33

Re: [SOLVED] How to NFS boot?

#11 2012-10-12 17:11:36

Re: [SOLVED] How to NFS boot?

#12 2012-10-12 20:20:51

Re: [SOLVED] How to NFS boot?

#13 2012-10-13 11:21:16

Re: [SOLVED] How to NFS boot?

#14 2012-10-17 10:15:23

Re: [SOLVED] How to NFS boot?

#15 2012-10-18 08:56:45

Re: [SOLVED] How to NFS boot?

Board footer