read only rootfs

android · 2012-02-17 00:18:59

Hello Archers,

I'm trying to implement a read only rootfs.

The trac thread here:
https://bugs.archlinux.org/task/9384

Is closed in December 2011 and marked "Implemented".

Comments on the trac ticket and information here:
https://sites.google.com/site/linuxpendrive/rorootfs

Indicate that simply adding an entry in the fstab for the rootfs with an ro option can allow debug of which services etc need to be forced into read only compliance.

I add an entry to fstab:

# <file system> <dir>   <type>  <options>       <dump>  <pass>
/dev/mmcblk0p2  /       ext2    ro                0       1
tmpfs           /tmp    tmpfs   nodev,nosuid      0       0

However, after reboot, mount still shows root mounted read/write:

[root@beagle4 johnea]# mount
/dev/root on / type ext2 (rw,relatime,errors=continue,user_xattr,acl)
devtmpfs on /dev type devtmpfs (rw,relatime,size=244632k,nr_inodes=61158,mode=755)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
/sys on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
/run on /run type tmpfs (rw,nosuid,nodev,relatime,mode=755)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620)
shm on /dev/shm type tmpfs (rw,nosuid,nodev,relatime)
tmpfs on /tmp type tmpfs (rw,nosuid,nodev,relatime)

I believe rc.sysinit is responsible for the "Remounting Root" line during boot.

I'm not sure if rc.sysinit is responsible for the /dev/root device indicated in the mount output.

It seems this auto-generated device, or the rc.sysinit remount, are overriding the settings for / in the fstab.

There seemed to be a fair amount of interest expressed in read only root in comments of the above trac ticket.

Does anyone know what I should do to actually get the system to come up with a ro / partition?

Thanks for everything Arch!

johnea

lilsirecho · 2012-02-17 00:25:18

perhaps /etc/rc.local can be used for the purpose...........

falconindy · 2012-02-17 00:33:20

Works for me. What are you passing to the kernel in the bootloader?

android · 2012-02-17 00:43:22

falconindy wrote:

Works for me. What are you passing to the kernel in the bootloader?

From /proc:

[root@beagle4 johnea]# cat /proc/cmdline 
console=ttyO2,115200n8 mpurate=auto ... root=/dev/mmcblk0p2 ro rootfstype=ext2 rootwait

for u-boot I'm setting:

mmcroot=/dev/mmcblk0p2 ro

It's the exact default line, except the rw is changed to ro

Last edited by android (2012-02-17 00:51:30)

android · 2012-02-17 01:13:39

From dmesg.log:

[    3.422332] VFS: Mounted root (ext2 filesystem) readonly on device 179:2.

It seems the kernel is mounting the partition read only. It is sometime later that it is remounted read/write.

I've read through the wiki on initscripts, and the scripts themselves, I still don't understand how the rc.sysinit remount of root is controlled?

I'm running initscripts 2012.01.3-1

This is line 192 of rc.sysinit:

status "Remounting Root" \
        mount -o remount,rw /

It seems like this is always going to make / read/write.

falconindy, did you add an fstab entry? or only change the boot args?

Thanx!!

Last edited by android (2012-02-17 01:52:26)

android · 2012-02-17 02:14:03

OK, this is an archlinux-arm patch:

[root@traveler temp]# diff /etc/rc.sysinit rc.sysinit.beagle4 
10,11c10,11
< printhl "Arch Linux\n"
< printhl "${C_H2}http://www.archlinux.org"
---
> printhl "Arch Linux ARM\n"
> printhl "${C_H2}http://www.archlinuxarm.org"
193c193
<       mount -o remount /
---
>       mount -o remount,rw /

They use this to overcome not making a / entry in fstab.

This solves this initial question. Thank you for your reply falconindy!

However, this is still the beginning of the effort to actually boot into full functionality with ro rootfs. So I will not mark the thread SOLVED yet, and will instead continue to post here regarding further issues.

Once I've acheived read only rootfs, I'll use the thread info to create a wiki page.

Thanks Again! And any other info anyone can offer in getting read only / running is greatly appreciated!

johnea

kaos · 2012-02-28 14:07:19

Try this: https://bbs.archlinux.org/viewtopic.php … 6#p1062286
It requires aufs3 with aufs_friendly kernel. Set boot option rootmnt=aufs.
You will also need to set the rw option back on in rc.sysinit.

falconindy · 2012-02-28 16:52:37

Or try asking for help on the proper distro forum. OP is using ARM and didn't bother to tell us.

android · 2012-02-28 19:35:16

falconindy, Sorry to have given the impression that this is an ARM oriented effort. My objective is to deploy read only rootfs on archlinux, regardless of architecture.

kaos, that link is really the kind of info I'm missing. Can you offer something like sequential instructions for how to use the initcpio hook, and then what changes need to modify the boot sequence? Sorry to be dense, I'm still trying to understand how to put the whole thing together.

It seems like the dropping of unionfs from the mainline kernel has left a lot of the existing instructions outdated. aufs3 seems to be where it's at now in terms of ISO archlinux, or any other archlinux system that desires read only root.

tomegun · 2012-02-28 21:00:38

android wrote:

kaos, that link is really the kind of info I'm missing. Can you offer something like sequential instructions for how to use the initcpio hook, and then what changes need to modify the boot sequence? Sorry to be dense, I'm still trying to understand how to put the whole thing together.

If I understand your intentions correctly, you are interested in finding and fixing problems in order to make read-only root work "out of the box" (as opposed to finding a workaround just to "make it work")? If so, that's great :-)

I think the kind of workaround linked to by kaos should be avoided. In principle you should be able to just add your "ro" line to /etc/fstab and everything should just work. In practice, things probably will break as some daemons/libs probably still write to /etc at runtime. It would be great if you would work on finding the culprits, filing bugs upstream, or finding ways of configuring the software to avoid the problems.

An example is /etc/resolv.conf. NetworkManager will write the address of the dns server to this file whenever you connect to the network. However, I believe you can configure NetworkManager to use a local dnsmasq instance, and then /etc/resolv.conf can be a static file pointing to 127.0.0.1. Alternatively, this problem might be solved by using resolvconf.

Maybe the above example is not entirely correct (as it is from memory, and I never actually tried), but it was just meant to be for illustration purposes :-)

falconindy · 2012-02-28 21:14:53

android wrote:

falconindy, Sorry to have given the impression that this is an ARM oriented effort. My objective is to deploy read only rootfs on archlinux, regardless of architecture.

Fine, but Arch Linux ARM does packaging separate from us. The point is that the "bug" you've found is already solved here in x86 land.

kaos · 2012-02-29 21:09:06

I'm not on the arch machine right now, so forgive me potential errors and typos.

android wrote:

something like sequential instructions

are as follows:

If you have tmpfs mounted on /tmp umount it, because it will run out of space in the next step.

# umount /tmp

Install aufs. This will also compile and install 3.2.6 linux-aufs_friendly kernel.

# yaourt -S aufs3

Copy script from https://bbs.archlinux.org/viewtopic.php … 6#p1062286 to /lib/initcpio/hooks/unionmount
Copy hook from https://bbs.archlinux.org/viewtopic.php … 6#p1062286 to /lib/initcpio/install/unionmount and modify it: add the type of your root partition to the MODULES array. Mine is etx3, that's why there are only ext3 and aufs.
Add "unionmount" to the HOOKS array in /etc/mkinitcpio.conf.
Regenerate initramfs:

# mkinitcpio -p linux-aufs_friendly

In /boot/grub/menu.lst change linux to linux-aufs_friendly. You can keep your previous entries in case of problems. Add option rootmnt=aufs to the entry with linux-aufs_friendly.

In my menu.lst there are 3 entries with linux-aufs_friendly...

# (0) Arch Linux RO
title  Arch Linux
root   (hd0,1)
kernel /vmlinuz-linux-aufs_friendly root=/dev/sda2 rootmnt=aufs
initrd /initramfs-linux-aufs_friendly.img
# (1) Arch Linux RW
title  Arch Linux
root   (hd0,1)
kernel /vmlinuz-linux-aufs_friendly root=/dev/sda2
initrd /initramfs-linux-aufs_friendly.img

... and a fallback entry, so each boot I can choose between normal and ro system.

I'm not sure of "/initramfs-linux-aufs_friendly.img" check your /boot contents what's the file name.

Set the rw option in your /etc/rc.sysinit back on.

 status "Remounting Root" \
	mount -o remount,rw /

Reboot and test. There will be 2 errors while running the hook. One about root not being empty, and the second one from switch_root complaining about sth else. Ignore them.
The /rw and /ro dirs are bindings of originally mounted $root and tmpfs, so you can always access your $root to write it. You can easily disable the bindings by removing lines:

mount --bind $ro $rob
mount --bind $rw $rwb

from the script and regenerating your initramfs. Check /rw contents - it should contain all the files created or modified since boot. There should be some run/sth.pids and other stuff.

In case of problems check https://wiki.archlinux.org/index.php/Mkinitcpio

tomegun wrote:

In principle you should be able to just add your "ro" line to /etc/fstab and everything should just work. In practice, things probably will break as some daemons/libs probably still write to /etc at runtime.

So the things do.

Last edited by kaos (2012-02-29 21:14:37)

android · 2012-03-01 03:43:39

tomegun wrote:

In principle you should be able to just add your "ro" line to /etc/fstab and everything should just work.

This would be an awesome feature for arch!

resolv.conf is certainly one of the things that would need resolving 8-)

In addition to that, do you have an idea of how many more things would need a solution? How big of a job is this?

Does this still require a RW /var be available?

kaos wrote:

android wrote:
something like sequential instructions
are as follows:

Thanks kaos! Awesome instructions. I'll try it out and report.

The aufs solution is something that can work now, so I'll use it to solve the present problem. If the native RO rootfs can be configured, this seems a much better final solution.

A list of issues that need resolving would make a good start to a wiki page.

kaos · 2012-03-01 09:19:10

android wrote:

A list of issues that need resolving

can be found in /rw when booted with union mounted /.

tomegun · 2012-03-01 12:18:25

kaos wrote:

android wrote:
A list of issues that need resolving
can be found in /rw when booted with union mounted /.

Yeah, this is actually a really neat way of finding the problems (didn't realise that at first :-) ).

tomegun · 2012-03-01 12:27:58

android wrote:

In addition to that, do you have an idea of how many more things would need a solution? How big of a job is this?

I don't know, that was the only issue I could think of, but I have never actually tried this, so there are almost certainly more (things should have improved a lot recently with the last version of util-linux, which was a big offender).

android wrote:

Does this still require a RW /var be available?

Yes, a rw /var is always needed. In a "perfect world" you'll have something like this:

/etc: only writable by the admin for configuration purposes (manual remount is acceptable).
/usr: only writable by the package manager for updating purposes (manual remount is acceptable).
/var: only writable by programs at run-time (needs to be mounted rw at all times).
/run: similarly (only difference that contents are not preserved on reboot).
/home and /srv: only writable by users/admin (needs to be mounted rw at all times).
/dev, /sys, /proc: kernel stuff, let the init system deal with it :-)

Things that should eventually go away (or maybe they won't but at least they are ugly edge-cases):

/lib, /bin, /sbin, /opt: treated like /usr
/tmp, /media: treated like /run
/root, /mnt: treated like /home and /srv

android · 2012-03-01 19:27:00

kaos wrote:

android wrote:
A list of issues that need resolving
can be found in /rw when booted with union mounted /.

This is a HUGE insight into what needs to be changed to support native ro rootfs!

I'm almost there 8-/

Ran the mkinitcpio, setup the boot param: rootmnt=aufs,

the thing that's still not quite right is the /dev/sda3 mount on /ro, it's still showing as rw:

[root@beagle johnea]# mount
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
/sys on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
/dev on /dev type devtmpfs (rw,nosuid,relatime,size=244736k,nr_inodes=61184,mode=755)
/run on /run type tmpfs (rw,nosuid,nodev,relatime,mode=755)
root.union on / type aufs (rw,relatime,si=da2a884f)
/dev/sda3 on /ro type ext3 (rw,relatime,errors=continue,user_xattr,acl,barrier=1,data=ordered)
root.rw on /rw type tmpfs (rw,relatime,size=98304k)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620)
shm on /dev/shm type tmpfs (rw,nosuid,nodev,relatime)
tmpfs on /tmp type tmpfs (rw,nosuid,nodev,relatime)
[root@beagle johnea]# 
[root@beagle johnea]# cat /proc/cmdline 
console=ttyO2,115200n8 rootmnt=aufs mpurate=auto root=/dev/sda3 ro rootfstype=ext3 rootwait
(edited out extraneous params for clarity)

I tried setting the boot param: root=/dev/sda3 ro , as well as: root=/dev/sda3 rw

It didn't seem to make any difference.

Do I need a / entry in the fstab?

falconindy: This testing is on ARM. It's just the only machine I have right here that I can continuously reboot for testing. Otherwise I'd be rebooting the laptop I'm working on 8-( I think I've gotten past the bootloader setup, based on the /proc/cmdline.

Thanks to everyone for the help on this!

android · 2012-03-01 23:27:08

android wrote:

the thing that's still not quite right is the /dev/sda3 mount on /ro, it's still showing as rw:

OK, after playing with this a bit, and reading the aufs_mount_handler() function in the hook,I think this is all working as it should be.

The /ro branch is actually writable, but is not written to.

I'm able to:

# mount -o remount,ro /ro

And the /ro mount will show as read only:

[root@beagle5 fat-boot]# mount
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
/sys on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
/dev on /dev type devtmpfs (rw,nosuid,relatime,size=244736k,nr_inodes=61184,mode=755)
/run on /run type tmpfs (rw,nosuid,nodev,relatime,mode=755)
root.union on / type aufs (rw,relatime,si=d9d9484f)
/dev/sda3 on /ro type ext3 (ro,relatime,errors=continue,user_xattr,acl)
root.rw on /rw type tmpfs (rw,relatime,size=98304k)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620)
shm on /dev/shm type tmpfs (rw,nosuid,nodev,relatime)
tmpfs on /tmp type tmpfs (rw,nosuid,nodev,relatime)

If the /ro is left mounted as rw,
then If I write to a file in /ro/some/dir/some/file
I'm able to see the changes in /some/dir/some/file
and they are non-volitile and persist through a reboot.

The union features really are what a lot of usage models would want. It seems like it might be a good way to deal with /var even when the / partition is native media mounted ro. Then maybe it wouldn't require the initrd, just remount /var to aufs in rc.local.sysinit or something like that?

I think the aufs setup is working at this point. I'll put kaos' instructions on a wiki page, with notes about actual read only / and maybe some listings from the /rw as an illustration of what is being written to where.

Thanks Again for the help!

Last edited by android (2012-03-01 23:29:20)

kaos · 2012-03-02 01:35:16

android wrote:

android wrote:
the thing that's still not quite right is the /dev/sda3 mount on /ro, it's still showing as rw:
OK, after playing with this a bit, and reading the aufs_mount_handler() function in the hook,I think this is all working as it should be.
The /ro branch is actually writable, but is not written to.

That's right, it is working as desired to. The /ro dir name is a bit misleading though. The /ro and /rw names reflect options used in aufs mount (dirs=/root/rw=rw:/root/ro=ro). /root/ro is actually mounted rw and bound to /new_root/ro so it can be modified even with aufs running. If you want it completely ro add -o ro option in the script:

mount -t "${fstype}" "${root}" -o ro $ro

or run

# mount -o remount,ro /ro

as you already pointed it out.

android wrote:

If the /ro is left mounted as rw,
then If I write to a file in /ro/some/dir/some/file
I'm able to see the changes in /some/dir/some/file
and they are non-volitile and persist through a reboot.

Correct.

android wrote:

The union features really are what a lot of usage models would want. It seems like it might be a good way to deal with /var even when the / partition is native media mounted ro. Then maybe it wouldn't require the initrd, just remount /var to aufs in rc.local.sysinit or something like that?

I've read about this solution after I used aufs, but having seen /rw contents I'm not going to try it out.

BTW: Thanks go to Forest as I based my script on his http://www.logicsupply.com/blog/2009/01 … ux-system/

Last edited by kaos (2012-03-02 01:57:54)

Arch Linux

#1 2012-02-17 00:18:59

read only rootfs

#2 2012-02-17 00:25:18

Re: read only rootfs

#3 2012-02-17 00:33:20

Re: read only rootfs

#4 2012-02-17 00:43:22

Re: read only rootfs

#5 2012-02-17 01:13:39

Re: read only rootfs

#6 2012-02-17 02:14:03

Re: read only rootfs

#7 2012-02-28 14:07:19

Re: read only rootfs

#8 2012-02-28 16:52:37

Re: read only rootfs

#9 2012-02-28 19:35:16

Re: read only rootfs

#10 2012-02-28 21:00:38

Re: read only rootfs

#11 2012-02-28 21:14:53

Re: read only rootfs

#12 2012-02-29 21:09:06

Re: read only rootfs

#13 2012-03-01 03:43:39

Re: read only rootfs

#14 2012-03-01 09:19:10

Re: read only rootfs

#15 2012-03-01 12:18:25

Re: read only rootfs

#16 2012-03-01 12:27:58

Re: read only rootfs

#17 2012-03-01 19:27:00

Re: read only rootfs

#18 2012-03-01 23:27:08

Re: read only rootfs

#19 2012-03-02 01:35:16

Re: read only rootfs

Board footer