You are not logged in.

#1 2023-04-04 21:54:15

xenobro
Member
Registered: 2019-05-25
Posts: 100

[SOLVED] root=tmpfs fails

Hi, I've been playing with the idea of running my system from RAM, and decided to try using the "root=tmpfs" kernel parameter mentioned here and here, but when I put it on GRUB, boot failed with those messages:

 ERROR device 'tmpfs' not found. Skipping fsck.
:: mounting 'tmpfs' on real root
mount: /new_root: no filesystem type specified.
 You are now being dropped into an emergency shell.
sh: can't access tty: job control turned off.
[rootfs ]#

These are the commands I'm using to boot and what's shown on the "edit commands" section of GRUB:

set gfxpayload=text
        insmod gzio
        insmod part_gpt
        insmod fat
        set root='hd0,gpt1'
        if [ x$feature_platform_search_hint = xy ]; then
          search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt1 --hint-efi=hd0,gpt1 --hint-baremetal=ahci0,gpt1  4354-283A
        else
          search --no-floppy --fs-uuid --set=root 4354-283A
        fi
        echo    'Carregando o Linux linux...'
        linux   /vmlinuz-linux root=tmpfs rw  
        echo    'Carregando o ramdisk inicial...'
        initrd  /intel-ucode.img /amd-ucode.img /initramfs-linux.img

I get the same error even putting it on /etc/fstab like this:

tmpfs / tmpfs defaults,noatime 0 1

Also, F1 doesn't work on GRUB, I have to use Ctrl+X or F10 after editing the command line to continue booting.

Can somebody help me?
I'm using kernel 6.2.9-arch1-1, systemd 253.2-1-arch and GRUB 2:2.06.r456.g65bc45963-1.

Thank you very much,
xenobro.

Last edited by xenobro (2023-04-16 09:55:08)

Offline

#2 2023-04-05 09:42:17

seth
Member
Registered: 2012-09-03
Posts: 49,982

Re: [SOLVED] root=tmpfs fails

I've been playing with the idea of running my system from RAM

https://wiki.archlinux.org/title/Multib … nd_memdisk
https://wiki.archlinux.org/title/Improv … s_to_tmpfs

and decided to try using the "root=tmpfs" kernel parameter

What do you think that does? Where does the system come from with that configuration?

Offline

#3 2023-04-05 10:52:58

xenobro
Member
Registered: 2019-05-25
Posts: 100

Re: [SOLVED] root=tmpfs fails

Hi, Seth, and thank you very much for your reply.

I'll give the syslinux+memdisk a try as soon as I can, and I already do most things in your second link, including having my home folder on RAM, which is what gave that idea.
As for what "root=tmpfs" should do, from the systemd-fstab-generator OFFICIAL page,

...
Kernel Command Line
systemd-fstab-generator understands the following kernel command line parameters:
...
root=
Configures the operating system's root filesystem to mount when running in the initrd. This accepts a device node path (usually /dev/disk/by-uuid/… or /dev/disk/by-label/… or similar), or the special values "gpt-auto" and "tmpfs".
...
Use "tmpfs" in order to mount a tmpfs(5) file system as root file system of the OS. This is useful in combination with mount.usr= (see below) in order to combine a volatile root file system with a separate, immutable /usr/ file system. Also see systemd.volatile= below.

So, from what I could gather, it would mount a tmpfs as root.
What would happen next, if systemd-fstab-generator would do something about the system or if I woul have to create a systemd service to copy the files in their place as I already do with my home folder on RAM, that's what I was trying to finding out.
As I've mentioned, this is just an experiment, a curiosity: tinkering to see how things work.

And I should mention that many other systems allow you to remove the pendrive/sdcard after loading the system into memory, and I've had to use "break=premount" to change a pendrive from a USB 2.0 which was the only booting port to a USB 3.0 during boot.
So those things gave me the idea to try and reproduce something similar in Arch, even though it might not be very practical.
Also, there's "systemd.volatile=" and the "mount.usr=" mentioned in the systemd-fstab-generator page, so this isn't really that far-fetched.
But I couldn't get there and find these things out because it fails before that: "root=tmpfs" SHOULD at least mount and then complain after, if it was the case, or maybe systemd-fstab-generator should do something with it, but the failure I'm getting is way before that, as in kernel doesn't seem to understand it, even though systemd-fstab-generator's official says it should. Am I using it wrong? Is there another place I should put "root=tmpfs" or anything else I should put on the  cmdline to make it work?

Thank you very much once again,
xenobro.

Last edited by xenobro (2023-04-05 10:56:26)

Offline

#4 2023-04-05 11:00:41

V1del
Forum Moderator
Registered: 2012-10-16
Posts: 21,426

Re: [SOLVED] root=tmpfs fails

If that would work it would only work on a systemd based initramfs are you using that? If you aren't using systemd as your initramfs it can only become relevant once the root partition is mounted. You'd havev to adjust your mkinitcpio.conf to switch to a systemd based initrd: https://wiki.archlinux.org/title/Mkinit … mmon_hooks (note the table and which kinds of hooks you'd replace with the systemd hook)

Last edited by V1del (2023-04-05 11:03:18)

Offline

#5 2023-04-07 05:02:35

xenobro
Member
Registered: 2019-05-25
Posts: 100

Re: [SOLVED] root=tmpfs fails

Hi, V1del , and thank you very much for your help.

Adjusting mkinitcpio.conf like this

...
#HOOKS=(base udev autodetect modconf block filesystems keyboard fsck consolefont keymap)
HOOKS=(systemd autodetect modconf block filesystems keyboard fsck sd-vconsole)
...

As told in the wiki link you've posted did the trick, "root=tmpfs" was recognized by systemd-fstab-generator and boot could proceed, only to halt a bit further at this point:

...
[ FAILED ] Failed to start Switch Root.
See 'systemctl status initrd-switch-root.service' for details.
You are in emergency mode. After logging in, type "journalctl -xb" to view system logs, "systemctl reboot" to reboot, "systemctl default" or "exit" to boot into default mode.

Cannot open access to console, the root account is locked.
See sulogin(8) man page for details.

Press Enter to continue.

And when I press Enter...

Reloading system manager configuration
Starting default.target
You are in emergency mode. After logging in...

And the same thing repeats over and over, until I reboot without the "root=tmpfs" kernel parameters and use a normal "root=" destination instead.
I'm not really sure what's the issue here, maybe systemd just mount the tmpfs on root and didn't copy anything to it, so I'd have to create a service to run right after "sysroot.mount" and way before "initrd-switch-root.service" and copy what's necessary into place, but I'm having problems trying to make mkinitcpio pack my service in initramfs and making systemd execute it.
For testing purposes, I've created this file:

[Unit]
Description=Populate /
After=sysroot.mount
Requires=sysroot.mount
Before=initrd-root-fs.target

[Service]
#ExecStart=/usr/bin/echo "HERE RUNS ROOTFILLER!"
ExecStart=/usr/bin/bash -c 'read -n1'

[Install]
RequiredBy=initrd-root-fs.target

I've put it into /usr/lib/systemd/system and and I have to use "RequiredBy=" in it and "Requires=" in the files mentioned here, because otherwise the file doesn't get packed into the initramfs - seems like "WantedBy=", "Wants=", "After=", "Before=", etc. are not enough to be considered a dependency and packed it in by mkinitcpio.
But then, after putting these "Before", "After", etc. referencing "rootlfiller.service" in the other files (i.e. initrd-root-fs.target), I started to get randomly locked out of my system, getting a message with something like "too early for unauthorized user login, try again later" on a loop, only to autologin without any issues after a reboot a two,
Can you help making a service that can execute something from systemd initramfs so I can try to find out how to proceed to get it to boot from RAM?

Once again, thank you very much,
xenobro.

Last edited by xenobro (2023-04-07 05:04:39)

Offline

#6 2023-04-07 05:33:59

seth
Member
Registered: 2012-09-03
Posts: 49,982

Re: [SOLVED] root=tmpfs fails

Do you plan to move your entire system into the initramfs?
Otherwise, did you see

The manpage wrote:

This is useful in combination with mount.usr= (see below) in order to combine a volatile root file system with a separate, immutable /usr/ file system.

?

You'll still lack any files in /etc and /var

Offline

#7 2023-04-07 07:12:36

xenobro
Member
Registered: 2019-05-25
Posts: 100

Re: [SOLVED] root=tmpfs fails

Hi, seth, and thank you for your reply.

I do intend to move everthing into RAM, but I have tried using "mount.usb=" and have not found a single example on exactly how to use it on the entire internet, so I've tried everything:

"mount.usr=UUID=123abc123bab..." and "mount.usr=/dev/sdxN" ("N" being the partition number) get messages like these

...
      Starting File System Check on /dev/sdxN...
[ OK ] Finished File System Check on /dev/sdxN.
      Mounting /sysusr/usr...
[ OK ] Mounted /sysusr/usr.
      Mounting /sysroot/usr...
[ OK ] Mounted /sysroot/usr.
[ OK ] Reached target Initrd File Systems.
...

And then fails at Switch Root with this message

      Starting Switch Root...

Welcome to Arch Linux!

[      17.123123] systemd[1]: Failed to fork off sandboxing environment for executing generators: Protocol error
[!!!!!!] Failed to start up manager.
[      17.321321] systemd[1]: Freezing execution.

"mount.usr=/usr" gives me the same errors as before and "mount.usr=usr" gives even more error messages, so I take it the correct ones to be "mount.usr=UUID=123132..." or "mount.usr=/dev/sdxN", right?

Anyway, as long as there is "root=tmpfs" in the cmdline, it seems like I'm gonna need to copy a lot of stuff either to new_root or to sysroot. I'm still in doubt over which, since it seems like the initramfs process is working, it's only when switchting to the "real, final root" that the problems seem to appear. Can you help me making a service that is at least able to execute something right after sysroot.mount?

Also, if I set something in the kernel "root=", it doesn't matter what's in /etc/fstab:
For instance, if I have "root=UUID=123123abcabc..." in the kernel line in GRUB but put "tmpfs / tmpfs defaults,noatime 0 1" in /etc/fstab, it always gets mounted as the kernel root parameter said, it overrides fstab. Is this what it is supposed to happen?

Thank you very much,
xenobro.

Last edited by xenobro (2023-04-07 07:14:40)

Offline

#8 2023-04-07 07:15:32

3beb6e7c46a615a
Member
Registered: 2021-03-27
Posts: 165

Re: [SOLVED] root=tmpfs fails

seth wrote:

Do you plan to move your entire system into the initramfs?
Otherwise, did you see

The manpage wrote:

This is useful in combination with mount.usr= (see below) in order to combine a volatile root file system with a separate, immutable /usr/ file system.

?

You'll still lack any files in /etc and /var

I think the idea of systemd-volatile-root.service is that "/etc/" is mostly empty because all relevant static configuration is in "/usr" as part of the system image or gets automatically created on first boot (e.g. with systemd-sysusers for /etc/passwd and related files) and everything under "/var" gets created by systemd-tmpfiles immediately after switching root. 

On Arch that's probably not as easy as e.g. under Fedora, because everything in Arch assumes that "filesystem" exists as part of "base", and many (most?) packages create their "/var/" hierarchy in the package itself and thus likely don't ship an appropriate tmpfiles configuration.

xenobro wrote:

Can you help making a service that can execute something from systemd initramfs so I can try to find out how to proceed to get it to boot from RAM?

I'd try to get away without a custom service first.   I think systemd-tmpfiles already provides you with everything you need to already; in particular you can use a tmpfiles entry of type "C" to copy files to "rootfs", and then place everything you'd need to have copied to "root" into "/usr/share/factory".  You'd not even have to add anything to your initramfs for this; you'd just need to put an appropriate tmpfiles configuration into "/usr/lib/tmpfiles.d" on your "usr" partition. Assuming you're having an "/usr" partition, that is.

Offline

#9 2023-04-07 22:45:39

xenobro
Member
Registered: 2019-05-25
Posts: 100

Re: [SOLVED] root=tmpfs fails

Hi, lunaryorn, and thank you very much for your help.

systemd-tmpfiles seems like a great idea, especially since, at least in my normal system, I've checked and it runs in a service seemingly right before root switching occurs and right after local-fs.target:

[x@archlinux ~]$ systemd-analyze critical-chain initrd-switch-root.target
The time when unit became active or started is printed after the "@" character.
The time the unit took to start is printed after the "+" character.

initrd-switch-root.target
└─systemd-tmpfiles-setup.service @4.705s +86ms
  └─local-fs.target @4.700s
    └─run-credentials-systemd\x2dresolved.service.mount @4.831s
      └─local-fs-pre.target @2.533s
        └─systemd-tmpfiles-setup-dev.service @2.352s +179ms
          └─kmod-static-nodes.service @1.926s +145ms
            └─systemd-journald.socket
              └─system.slice
                └─-.slice

There are some issues however that I don't really know how to solve even with systemd-tmpfiles but without an extra service file, the main one being that, until now, there was no (at least explicit) mention whatsoever of a /dev/sdxN or a UUID=... in the system: root was mounted on a tmpfs in initramfs and now /etc/fstab also tells it to mount it as a tmfps. And I have no idea what has been mounted or copied to root so far, so how should I address source and destination in tmpfiles.d? /usr/share for instance is going to point to initramfs. And is /sysroot the destination here? Because remember, this happens BEFORE switching root. And what about the source? The sdd/pendrive/sd card hasn't even been mounted yet, so how do I point to it? And how to umount it after, since it isn't necessary anymore?
Also, I need to make sure this runs ONLY ONE TIME at boot, since the systemd-tmpfiles may be called more times during the boot process.
One thing still confusing me is that, on my current "normal" system, I've unpacked initramfs-linux.img and noticed there is no "systemd-tmpfiles-setup.service" in it, only "systemd-tmpfiles-setup-dev.service", and both most definitely run BEFORE switching root, so I must have missed where was the sdd mounted in initramfs and if it is then mounted twice?
Anyway, can you help me with these issues?

Once again, thank you very much,
xenobro.

Last edited by xenobro (2023-04-07 22:50:13)

Offline

#10 2023-04-08 07:11:08

seth
Member
Registered: 2012-09-03
Posts: 49,982

Re: [SOLVED] root=tmpfs fails

Copied what from where?

lunaryorn wrote:

Assuming you're having an "/usr" partition, that is.

https://man.archlinux.org/man/core/syst … -root.8.en
https://man.archlinux.org/man/systemd-f … rator.8.en (check for systemd.volatile)

Maybe we should take a HUGE step back here:
*Why* do you want to be "running my system from RAM"?
What's your idea behind that?

Offline

#11 2023-04-08 09:05:03

3beb6e7c46a615a
Member
Registered: 2021-03-27
Posts: 165

Re: [SOLVED] root=tmpfs fails

Anyway, can you help me with these issues?

I don't think I can because I don't know what exactly you're trying to do.  As seth said it'd be helpful to illustrate your idea and perhaps also give a precise description of what you'd like to do, and how your target configuration should look like.

But assuming you're bootstrapping from an "/usr" partition/image, I think you'd add your tmpfiles configuration for populating root to "/usr/lib/tmpfiles.d" and put all files/directories you'd like to copy to "/" to "/usr/share/factory".  Then you'd configure your initrd generator to mount that "/usr" partition.  In the initrd systemd will then mount a tmpfs as "/", and then mount the "/usr" partition.  Then system will switch root to the real userspace (which as far as systemd is concerned runs from "/usr" alone), mount remaining file systems, i.e "local-fs.target" (in your setup probably none) and run tmpfiles which then populates "/".    Up to this point the system runs from "/usr" alone, but systemd is fine with that, and all other services, etc. only start now.

Note that in this scenario "/usr" can also be e.g. a squashfs image or any other kind of image (you could probably even get systemd to boot of an OCI image); it doesn't have to be a real disk partition.  But it must be mountable and it must contain a fully populated "/usr" fileystem.

But I must say that I don't really understand the point of this exercise when it comes to Arch.  This is something you'd do for closed down appliances where you require read-only images, A/B updates and absolutely reliable factory reset, because you have no direct access to the system.   But in a desktop system running on a rolling release distributions you always have full access and can recover e.g. by booting of an USB stick into an emergency system, and A/B updates in Arch are probably going to be a massive pain.

If you're doing this for intellectual curiosity (e.g. to understand how systemd does  all this) I think you're better off experimenting with "mkosi" and Fedora.  mkosi can populate an "/usr" image for you, and boot directly into a VM with "mkosi qemu", and Fedora already comes with a much more thorough tmpfiles configuration for its packages. 

In Arch most packages still create files outside "/usr" as part of their installation instead of including a "tmpfiles" configuration to have them created at boot, so it's likely a lot of work to get an Arch system boot cleanly of "/usr" alone.

Last edited by 3beb6e7c46a615a (2023-04-08 09:06:36)

Offline

#12 2023-04-09 07:43:42

xenobro
Member
Registered: 2019-05-25
Posts: 100

Re: [SOLVED] root=tmpfs fails

Hi seth and lunaryorn and thank you once again for replying.

So, maybe I wasn't clear enough about what I'm trying to do, so I'm sorry if that is the case. This is what I'm getting at:
Insert pendrive/sd card in my computer; start boot; mount the pendrive/sd card at some temporary folder; mount tmpfs at new_root, for instance; copy everything or what I want from the pendrive/sd card to new_root; unmount pendrive/sd card; switch root to new_root; finish booting running from RAM; REMOVE PENDRIVE/SD CARD and continue running the system.
And I'm trying to do that with systemd initramfs, because I have managed to do this with busybox initramfs adapting what I found in this link, by changing /usr/lib/initcpio/init_functions like this:

...
default_mount_handler() {
#    msg ":: mounting '$root' on real root"
#    if ! mount -t "${rootfstype:-auto}" -o "${rwopt:-ro}${rootflags:+,$rootflags}" "$root" "$1"; then
#        echo "You are now being dropped into an emergency shell."
#        # shellcheck disable=SC2119
#        launch_interactive_shell
#        msg "Trying to continue (this will most likely fail) ..."
#    fi

     msg "::creating /roottmp"
     mkdir /roottmp

     msg ":: mounting '$root' on /roottmp"
     mount -t ${rootfstype:-auto} -o ${rwopt:-ro}${rootflags:+,$rootflags} $root /roottmp

     msg ":: mounting tmpfs on '$1'"
     mount -t tmpfs -o size=100% tmpfs $1

     msg ":: entering '$1'"
     cd $1

     msg ":: copying the contents of /roottmp to '$1'"
     cp -rfa /roottmp/* $1

     msg ":: umounting /roottmp"
     umount /roottmp

     msg "::deleting /roottmp"
     rmdir /roottmp
}

Remaking initramfs and rebooting, though this isn't ideal since it lacks the test and the contingency for failure. There's also an issue where, when I change the copy command to

cp -rfa /roottmp/{bin,boot,dev,etc,home,lib,lib64,mnt,opt,proc,root,run,sbin,srv,sys,tmp,usr,var,x} $1

Or anything like that to avoid copying my backup & personal folder with it, it fails with

::creating /roottmp
:: mounting 'UUID=1234abcd-12ab-34cd-56ef-1234abcd5678' on /roottmp
mount: /roottmp: can't find .UUID=1234abcd-12ab-34cd-56ef-1234abcd5678.
:: mounting tmpfs on '/new_root'
:: entering 'new_root'
:: copying the contents of /roottmp to 'new_root'
cp: can't stat 'cp -rfa /roottmp/{bin,boot,dev,etc,home,lib,lib64,mnt,opt,proc,root,run,sbin,srv,sys,tmp,usr,var,x}': No such file or directory.
:: unmounting /roottmp
umount: can't unmount /rootmp: Invalid argument
:: deleting /rootmp
:: running cleanup hook [udev]
ERROR: Root device mounted successfully, but /sbin/init does not exist.
Bailing out, you are on your own. Good luck.

sh: can't access tty: job control turned off
[rootfs new_root]# _

And then, when I get to /usr/lib/initcpio/init_functions in this pendrive, my changes have mysteriously been reverted and the file is back to its last version before the fail. So much so that justing mounting this unit as /mnt, its boot partition as /mnt/boot, arch-chrooting into /mnt and regenerating initramfs goes back to the working version again. But I digress, I'm guessing I should just make another topic for this later since it's not related to the issue at hand, which is booting from RAM using systemd.

Edit: I've just realized a dot (".") appeared before "UUID" in the second case for whatever reason. I wonder where that came from.

Once again, there is really no why I'm doing this other than trying to learn more about the system and curiosity.
It's not really such an alien and weird concept, many distros do it, some of the more famous being Alpine, TAILS and LPS/TENS - these last two focused on security and not quite that small, but there also inumerous people who experimented with that and made it work in their systems, like for instance this one, this one, this other one and so many others, besides what I've already mentioned here.

I have already read those pages, seth, and they're still not doing it for me, unfortunately. Maybe I'm not getting something, but my final objective here is pretty straight forward:
Copy the contents (or at least the necessary parts) of my pendrive to a tmpfs on boot, change root into it, finish booting, REMOVING THE PENDRIVE and continue running the system on RAM -  and do this with systemd initramfs. That is all.
It doesn't matter if it's not efficient or silly, I just really want to learn how to do it.
And also to learn how to make a simple service run a simple command during the systemd initramfs to understand how it works. It could be as silly as "echo 'Hello World!' && sleep 10", as long as it's a service and it's included in initramfs.

Also, seth, my question about copy destination and source is because I have to state those things in the /usr/lib/tmpfiles.d: at that point of boot, how is the pendrive/sdd referred to? Is it /sysroot? Because THAT pendrive is the source of files and it most certainly isn't "/", since it wasn't mounted as such. Maybe I need a non-"root=" start of boot to be able to copy everything to RAM and have a final root as tmpfs. I'm stil confused as to which root is tmpfs when you use "root=tmpfs" or systemd-volatile: is it just the first initramfs root and then it switches to sdd/pendrive root? Because what I'm trying to do is make the LAST root tmpfs.

Anyway, sorry for the long message: please help me do with systemd initramfs what

...
mkdir /roottmp
mount -t ${rootfstype:-auto} -o ${rwopt:-ro}${rootflags:+,$rootflags} $root /roottmp
mount -t tmpfs -o size=100% tmpfs $1
cd $1
cp -rfa /roottmp/* $1
umount /roottmp
...

In default_mount_handler() inside /usr/lib/initcpio/init_functions did with busybox initramfs, which is to boot completely from RAM.

Once again, thank you very much,
xenobro.

Offline

#13 2023-04-09 11:11:23

3beb6e7c46a615a
Member
Registered: 2021-03-27
Posts: 165

Re: [SOLVED] root=tmpfs fails

So you don't have an existing /usr partition.  Sorry, this wasn't clear to me, and I misunderstood much of what you were trying to do.

I'm not sure whether you can rely on tmpfiles in this case.  My answer was assuming you were trying to fill a rootfs from a mounted /usr partition after switching root, which tmpfiles can do easily.  But in your setup you'd need to fill the rootfs from an arbitrary mount point before switching root, and I'm not sure whether the appropriate tmpfiles services run in initramfs, and even if, whether you can get ordering right wrt to a mount unit for your pendrive.

A custom service might indeed be your best bet.  I think you need a mount unit for your pendrive to have it mounted to e.g. "/mnt/portable_root". This mount unit needs an after dependency for "local-fs-pre.target" (no filesystems may be mounted before that target in an initramfs).  Then you need a custom service which runs after this mount unit and "initrd-root-fs.target", and before "initrd-parse-etc.service", and is required by "initrd-fs.target".  In this service you can fill rootfs, and by means of these dependencies you ensure that the rootfs is filled, and filled before systemd scans "/etc/fstab" in the initramfs (so you can still mount further filesystems through /etc/fstab).  This service can then copy relevant files from "/mnt/portable_root" to "/sysroot" where the new rootfs is mounted (i.e. your tmpfs if you use "root=tmpfs").  The service would just execute a simple script to copy the contents. See bootup(7) for the relevant targets.

I'd probably not go all in with a tmpfs rootfs right away.  Instead try to get the whole setup working first with a real root filesystem, and in your service just copy any file (e.g. "it-worked.txt") from your pendrive to the new rootfs.  This way you always have a bootable system, and don't drop to a rescue shell if you make e.g. a typo.  And you retain all logd, so if your service fails or the mount unit doesn't become available or … you have `journalctl` available to debug the issue and see what's wrong.  And only if all this works, change your service to copy the rootfs contents and pass "root=tmpfs".

I'd perhaps even test the entire workflow with mkosi first, because then you can automate your experiments, and directly test it in a VM with "mkosi qemu", which I think drastically shortens the feedback loop.

But at this point you're mostly on your own as far as I'm concerned.  I've never done this, and to be honest, I think it's a pointless exercise wink so I can't really help you any further.

Offline

#14 2023-04-09 13:34:45

Head_on_a_Stick
Member
From: London
Registered: 2014-02-20
Posts: 7,680
Website

Re: [SOLVED] root=tmpfs fails

xenobro wrote:

It's not really such an alien and weird concept, many distros do it, some of the more famous being Alpine, TAILS and LPS/TENS

Alpine uses overlayfs from the initramfs init script:

https://gitlab.alpinelinux.org/alpine/m … #L537-L551

Something similar could probably be achieved with a hook for mkinitcpio(8). See also https://github.com/Antynea/grub-btrfs/b … ap_ro-hook.

Offline

#15 2023-04-09 19:20:41

seth
Member
Registered: 2012-09-03
Posts: 49,982

Re: [SOLVED] root=tmpfs fails

Anyway, sorry for the long message: please help me do with systemd initramfs what … In default_mount_handler() inside /usr/lib/initcpio/init_functions did with busybox initramfs

This thread sarted out w/ you using busybox, you only switched to systemd after V1del's comment in #4 ?

At best you'd be fixated of somehow using root=tmpfs for that, but afaict that's exclusively relevant to systemd's volatile root featues (overlay or not) which is not what you described to be wanting.

Offline

#16 2023-04-16 09:52:49

xenobro
Member
Registered: 2019-05-25
Posts: 100

Re: [SOLVED] root=tmpfs fails

Hi, everyone, thank you once more for your help and patience and sorry for taking so long to reply this time.

So I've finally managed to boot entirely from RAM using systemd initramfs following lunaryorn's instructions, though it was a bit more complicated than that, but I'll get to it. Step by step, here's what I did.

First, as we've mentioned, I've had to change /etc/mkinitcpio.conf to use systemd initramfs:

...
# HOOKS=(base udev autodetect modconf block filesystems keyboard fsck consolefont keymap)
HOOKS=(systemd autodetect modconf block filesystems keyboard fsck sd-vconsole)
...

Then, I've created the 2 files from lunaryorn's post in /usr/lib/systemd/system, rootsource.mount

[Unit]
After=local-fs-pre.target
Before=rootfiller.service
StopWhenUnneeded=true

[Mount]
What=/dev/disk/by-uuid/1234abcd-1234-abcd-1234-1234abcd1234
Where=/rootsource
Type=ext4
Options=defaults,noatime

And rootfiller.service

[Unit]
Description=Populate /
After=rootsource.mount initrd-root-fs.target
Requires=rootsource.mount sysroot.mount
Before=initrd-parse-etc.service

[Service]
Type=oneshot
# ExecStart=/usr/bin/bash /usr/bin/rootfiller.sh
ExecStart=/usr/bin/rsync -av /rootsource/ /sysroot --exclude MYBACKUPFOLDER --exclude lost+found

[Install]
RequiredBy=initrd-fs.target

Some notes here:
The *.mount file needs "StopWhenUnneeded=true", otherwise the unit continues to be "busy" and can't be mounted after switching root.
The files HAVE to be in /usr/lib/systemd/system; if they are in /etc/systemd/system, they are not included in initramfs-linux.img, and initrd-fs.target needs to have

...
Requires=rootfiller.service

Otherwise the files are also left out of initramfs: "RequiredBy=initrd-fs.target" in rootfiller.service is not enough for some reason.
Finally, I choose to use the "ExecStart=" line with rsync, but there's also the option of using an external script, like the commented line above, that looks somewhat like this:

#!/bin/bash

for i in $(ls /rootsource/ | grep -v 'MYBACKUPFOLDER\|lost+found')
do
        cp -a /rootsource/$i /sysroot
done

I wound up preferring rsync because it was simpler and also because it had to pull less things into initramfs, which takes us to the next step: the commands used in "ExecStart=" need to be "manually" included in the image, they're not pulled in automatically, so I've had to create a mkinitcpio hook to do this in /etc/initcpio/install, rootfiller.hook:

# Hook to put the rootfiller.service's neeeded files in initramfs-linux.img

build() {
        add_binary "rsync"
        # add_binary "cp"
        # add_binary "grep"
        # add_binary "ls"
        # add_file "/usr/bin/rootfiller.sh"
}

help() {
    cat <<HELPEOF
This hook includes the files and binaries needed for rootfiller.service in the initramfs-linux.img file.
HELPEOF
}

As before, the commented lines are in case one decides to go the external script route.

Then I changed /etc/fstab like so:

# /dev/sda2
# UUID=1234abcd-1234-abcd-1234-1234abcd1234              /               ext4    defaults,noatime                                0 1
tmpfs                                                   /               tmpfs   defaults,noatime,size=100%                      0 1

"size=100%" should be needed, since otherwise the systemd would only give tmpfs half of RAM and that could be a problem, but systemd-fstab-generator seems to completely ignore /etc/fstab and ONLY use the information passed in "root=" and "rootflags=", while the man page says the exact opposite would happen, that fstab should take precedence. Anyone have any idea why is this happening?

Anyway, I then recreated initramfs with:

mkinitcpio -A rootfiller.hook -g /boot/initramfs-linux.img

Rebooted and pressed "e" on GRUB to edit the cmdline like this:

/vmlinuz-linux root=tmpfs rootflags=defaults,noatime,size=100% rw

It took some 3 minutes to put about 5GB in 8GB of RAM, and then it proceeded to boot normally until the login part, where this message appeared:
"System is booting up. Unprivileged users are not permitted to log in yet. Please come back later. For technical details, see pam_login(8)."
If I was using autologin, I'd have to Ctrl+Alt+Del once or twice and then it would boot without problems. If I was logging in normally, I'd have to do it as root, erase /run/nologin/ and exit, and I would be able to do it normally.
After a few tests, I've noticed that "systemd-user-sessions.service" ran normally but hasn't executed "/usr/lib/systemd/systemd-user-sessions start". To test it, I've ran "systemctl start systemd-user-sessions.service" which didn't delete /run/nologin, while running the command itself, "/usr/lib/systemd/systemd-user-sessions start" does delete it. Any idea what could be causing the service to run without errors but not its "ExecStart="?

Also, no matter what I did, I couldn't get the "/vmlinuz-linux root=tmpfs..." line to "stick": it would always be back to the pendrive's UUID even after mounting it on /mnt and its boot partition in /mnt/boot, arch-chroot'ing there and grub-mkconfig -o /boot/grub/grub.cfg. Anyone knows if that's possible?

This little experiment wound up being a tad more complicated than I originally expected, the busybox version being way easier to set up:
That would be just making sure /etc/mkinitcpio.conf is using busybox, changing default_mount_handler() function in /usr/lib/initcpio/init_functions like this:

default_mount_handler() {
#    msg ":: mounting '$root' on real root"
#    if ! mount -t "${rootfstype:-auto}" -o "${rwopt:-ro}${rootflags:+,$rootflags}" "$root" "$1"; then
#        echo "You are now being dropped into an emergency shell."
#        # shellcheck disable=SC2119
#        launch_interactive_shell
#        msg "Trying to continue (this will most likely fail) ..."
#    fi

     msg "::creating /rootsource"
     mkdir /rootsource

     msg ":: mounting '$root' on /rootsource"
     mount -t "${rootfstype:-auto}" -o "${rwopt:-ro}${rootflags:+,$rootflags}" "$root" /rootsource

     msg ":: mounting tmpfs on '$1'"
     mount -t tmpfs -o size=100% tmpfs "$1"

     msg ":: copying the contents of /rootsource to '$1'"
     for i in $(ls /rootsource/ | grep -v 'MYBACKUPFOLDER\|lost+found'); do cp -a /roottmp/$i "$1"; done

     msg ":: umounting /rootsource"
     umount /rootsource

     msg "::deleting /rootsource"
     rmdir /rootsource
}

Running "mkinitcpio -p linux" to recreate initramfs, changing fstab appropriately (though this doesn't seem necessary, since this file is again ignored) and then rebooting. That's all.

And seth, I've started the post trying to use "root=tmpfs", and then I found out that "root=tmpfs" needs systemd initramfs, therefore I needed to work with systemd initramfs to boot from RAM using "root=tmpfs", got it?

Finally, thank you for your suggestion, Head_on_a_Stick, I'll check overlayfs too. But doesn't Alpine use that just for the "changeable" layer of the system? I think it uses squashfs for the "unchangeable" part, isn't it?

Anyway, despite these questions I still have, I managed to copy everything to RAM during boot using "root=tmpfs", ran the system from there and was able to remove the pendrive as was the intention of this post, so I'm marking it as solved, though I'd love if anyone could clear up the doubts I've expressed here.

Sorry for the huge wall of text post and once again thank you very much,
xenobro.

Last edited by xenobro (2023-04-16 10:01:31)

Offline

#17 2023-04-16 10:16:51

3beb6e7c46a615a
Member
Registered: 2021-03-27
Posts: 165

Re: [SOLVED] root=tmpfs fails

xenobro wrote:

So I've finally managed to boot entirely from RAM using systemd initramfs following lunaryorn's instructions, though it was a bit more complicated than that

I'm sorry; I've never done this, so I had no idea of the details.

xenobro wrote:

The files HAVE to be in /usr/lib/systemd/system; if they are in /etc/systemd/system, they are not included in initramfs-linux.img

See https://bugs.archlinux.org/task/65978  Opened years ago, but was never fixed, and at some point I lost interest.

But if you're using a custom mkinitcpio hook anyway you could use it to add whatever files to whatever place.

As for your service issues, it's impossible to diagnose without journal logs, but I'm not sure how you'd retain those if your entire system runs in RAM.  Same goes for apparent mismatches between what's in the manpage and actual behaviour; we'd also need to see logs.  But as said, I think is not a useful setup: You've effectively created a fragile system which takes minutes to boot (that's an order of magnitude more than my slowest system takes to boot), all to be able to remove a pendrive.  I'm not sure what you're trying to achieve here which kinda kills my motivation to dig deeper smile

Finally, thank you for your suggestion, Head_on_a_Stick, I'll check overlayfs too. But doesn't Alpine use that just for the "changeable" layer of the system? I think it uses squashfs for the "unchangeable" part, isn't it?

You're right.  Overlayfs won't help you to move the entire system to RAM at once, because you still need some lower filesystem, even if the top-most one is tmpfs.

Offline

#18 2023-04-16 11:02:03

xenobro
Member
Registered: 2019-05-25
Posts: 100

Re: [SOLVED] root=tmpfs fails

Hi, lunaryorn, and thank you once again.

lunaryorn wrote:

I'm sorry; I've never done this, so I had no idea of the details.

No need to be sorry, your file suggestions were key to solve the issue here, after all! smile

Oh, that explains it, thanks!

I'd prefer not to have to use a hook, that systemd would "automagically" figure out what is needed from the "ExecStart=" service line and pull it in, but we can't always get what we want, right? smile

As for the journal logs, I can boot with the error as root, exit, log back in as my user, mount the pendrive and copy the logs there, then upload it here. Is it the /run/log/journal/*/system.journal one?
Because that one is not text, can I upload it somewhere so you can take a look?

As for the run from RAM thing, it could have its uses: for the privacy minded it's pretty good, while there are certainly other distros more appropriate for that. And there was also an advantage: my startx usually takes 4~5 secs to open - even though I'm using evilwm and a very minimalistic setup (still trying to figure that one out...) without even a wallpaper and firefox for instance takes a few secs to open.
From RAM everything is instantaneous. So you lose a few minutes at boot to have everything open immediately during system use. It is a VERY big gain in speed.
But of course, with 8GB of RAM and a 5GB system, that's a recipe for disaster for daily use, so just testing and playing, really... Now on a 32GB RAM system... smile

Anyway, tell me which log files you'd need and I'll post it here, I can save to a mounted pendrive if that's the case.
Again, thank you very much.

Last edited by xenobro (2023-04-16 11:05:30)

Offline

#19 2023-04-16 11:15:53

3beb6e7c46a615a
Member
Registered: 2021-03-27
Posts: 165

Re: [SOLVED] root=tmpfs fails

You can either copy the journal files and later read them with "journalctl --file" on another system, or dump the journal to a plaintext file with "journalctl -b > /path/to/file".

But as far as I'm concerned I don't think I'll take a look. I'm sorry but at this point I don't think I have time and interest to dig deeper smile

But since you've figured out his whole intricate process all on your own, with just some marginal name dropping from our side, I have faith that you'll absolutely be able to figure the rest out yourself, with the help of the journalctl logs smile

Nice work, I'm quite impressed by all this!

Offline

#20 2023-04-16 11:20:32

3beb6e7c46a615a
Member
Registered: 2021-03-27
Posts: 165

Re: [SOLVED] root=tmpfs fails

PS: Aside from the logs I'd also recommend to check whether the machine ID remains stable (ie is the same in your booted live system as on the pendrive).  Systemd uses this ID a lot in many places, and could reat badly to it being changed half through the boot process.

And since you fill root after it was mounted it could happen that systemd tries to access the machine ID file before it was copied, prompting it to generate a new one and initialize with that new ID, only to have the ID overwritten again by your root filler.

Not sure if that's actually the case, but it's a potential race condition I see with this process.

Offline

#21 2023-04-17 10:54:35

xenobro
Member
Registered: 2019-05-25
Posts: 100

Re: [SOLVED] root=tmpfs fails

Hi, lunaryorn, and thank you once again for you reply.

As usual, you were on point: after deleting /etc/machine-id, adding "--exclude etc/machine-id", recreating initramfs and rebooting, the issue was solved, thank you again!
Strangely enough, there are still 2 folders under /run/log/journal, one logging exactly until switch root and the other exactly after - and this also happens on my normal system. Even after deleting /etc/machine-id and the contents of /run/log/journal I still get 2 folders, one for logs before rootswitching and one after. The only way of making it just one log folder and file is appending "systemd.machine-id=somethingsomething..." to the kernel cmdline, but that's not ideal.

Anyway, I understand you not being really interested in this subject anymore, thank you very much for your help anyway.

I'm also having another problem - more frequent with the RAM experiment, but also sometimes with my actual system, so I think I'm gonna have to make another post here: my touchpad is sometimes not recognized even in /proc/bus/input/devices, and all it takes is a reboot (or 2, in the case of the system on RAM) for it to reappear and work perfectly.

Anyway, I'll make its own post.
Thank you all very much,
xenobro.

Offline

Board footer

Powered by FluxBB