[SOLVED] Emergency mode at boot after rsync (systemd issue?)

pasturegliding · 2022-10-29 20:16:28

My kernel is the latest linux-zen, and I use dkms-nvidia.

What I did:
I needed to extend my /boot partition, but the rest of the drive ("Disk 1") was / in a LUKS partition without logical volumes. So, I used dd to copy the LUKS partition to another drive ("Disk 2"), then used parted to delete the Disk 1 LUKS partition, extended the Disk 1 /boot partition, and ran luksFormat to create a new LUKS partition in the remainder of Disk 1. Because I had some trouble extending /boot, I just deleted the partition and recreated it at the proper size then re-installed the kernel.

Then, from the arch install iso, I opened the encryption and mounted the Disk 2 partition to /mnt, and mounted the Disk 1 partition to a directory I made at /mnt/del

Then, intending for rsync to simply move everything I needed to the newly resized Disk 1 partition, I ran this code:

rsync -aAXHv /mnt --exclude={"/mnt/dev/*","/mnt/proc/*","/mnt/sys/*","/mnt/tmp/*","/mnt/run/*","/mnt/mnt*","/mnt/media/*","/mnt/lost+found"} /mnt/del

At first glance, this seemed to work.

The problem:
I booted once from Disk 1, and it didn't load my wifi device (and probably lots of other devices) and threw a bunch of systemd errors. It still booted X, and I was able to login and use the desktop, but some modules weren't loaded. I checked to see if I could access the files on the other drive, but it's not visible (but it is visible from the install iso) I rebooted so I could record the errors as Arch booted, and this time X didn't start because it put me in emergency mode.

journalctl -xb

throws:

1. Some systemd "failed to find module" errors;
2. Some cryptsetup errors (which might be from me messing with crypttab or fstab, so I don't think they're pressing because I can fix that unless someone better versed than I knows better lol);
3. Some Nvidia errors
4. Some systemd var-lib-snap errors saying, "failed with result 'exit-code'" (I hardly even use snapd, but it might be relevant)
5. Some systemd "failed to mount Mount unit" errors
6. A "/boot: unknown filesystem type vfat" error (the installation guide told me to make /boot fat32 using parted. Should I do it again with a different filesystem?)
7. A systemd "boot.mount: Mount process exited, code=exited, status=32/n/a" error and another "boot.mount: failed with result 'exit-code'" systemd error.
8. A systemd "dependency failed for Local File Systems" error

Here is the whole journalctl: https://0x0.st/oYbn.txt

Is there some other command I need to run after copying the whole / with rsync?

Thanks in advance

Last edited by pasturegliding (2022-10-30 01:18:21)

cfr · 2022-10-29 20:52:55

You didn't exclude /mnt/del when copying to /mnt/del? Shouldn't you have a trailing slash on the source (/mnt/ rather than /mnt)?

Please post a complete journal rather than bits. See https://wiki.archlinux.org/title/List_o … n_services for tidy ways to do this from the command line.

pasturegliding · 2022-10-29 21:11:39

cfr wrote:

You didn't exclude /mnt/del when copying to /mnt/del? Shouldn't you have a trailing slash on the source (/mnt/ rather than /mnt)?
Please post a complete journal rather than bits. See https://wiki.archlinux.org/title/List_o … n_services for tidy ways to do this from the command line.

I did not do the trailing slash. The /mnt/mnt/* kept it out because I think I was chrooted in /mnt already. I just typed it from memory here. I excluded it properly, either way or it would have gone in a loop (happened once lol)

Here is the journalctl:
https://0x0.st/oYbn.txt

I'm running it again with the trailing slash now to see if that fixes it.

Last edited by pasturegliding (2022-10-29 21:18:42)

cfr · 2022-10-29 22:50:02

What is in the LUKS container? Does this include root etc.? Because your command line doesn't seem to involve any means of decrypting it, if so. What are you using to boot?

How did you reinstall the kernel to the boot partition exactly? Is it possible that kernel doesn't match the modules installed?

pasturegliding · 2022-10-29 23:30:01

cfr wrote:

What is in the LUKS container? Does this include root etc.?

Yes, it includes all of root.

cfr wrote:

Because your command line doesn't seem to involve any means of decrypting it, if so. What are you using to boot?

I use refind. Something I did wrong when switching to refind made it so it never finds the right root, so it drops to a cli and I just do

cryptsetup open /dev/sdb2 a

mount /dev/mapper/a new_root

exit

It isn't much of an inconvenience, so I never bothered fixing it lol

cfr wrote:

How did you reinstall the kernel to the boot partition exactly? Is it possible that kernel doesn't match the modules installed?

arch-chroot /dev/mapper/c

pacman -S linux-zen linux-zen-headers

mkinitcpio --preset linux-zen

I don't think it's possible that the kernel doesn't match the modules.

pasturegliding wrote:

I'm running it again with the trailing slash now to see if that fixes it.

Alas, it did not fix it lol

pasturegliding · 2022-10-29 23:40:25

pasturegliding wrote:

I don't think it's possible that the kernel doesn't match the modules.

Actually it seems that I am wrong because I'm back in the desktop and virtualbox is telling me the VirtualBox Linux kernel driver is either not loaded or not set up correctly...

I did

# mkinitcpio -P

and rebooted, but it didn't fix it.

I ran

$ uname -srm

And it says I'm running Linux 6.0.2-zen1-1-zen x86_64, but linux-zen should be at 6.0.5 right now, that's what's in my /boot

So I downgraded my kernel to 6.0.2 (https://wiki.archlinux.org/title/Downgrading_packages) and it worked. How do I upgrade my kernel now without re-breaking things?

Last edited by pasturegliding (2022-10-30 00:04:04)

cfr · 2022-10-30 00:04:36

pasturegliding wrote:

arch-chroot /dev/mapper/c

You need to mount the partition first. I assume you did something like

cryptsetup open /dev/sdb2 c

? So c is the LUKS container? But you want to mount the root partition and arch-chroot into that. You also need your boot partition mounted before you reinstall the kernel.

cfr · 2022-10-30 00:07:05

pasturegliding wrote:

So I downgraded my kernel to 6.0.2 (https://wiki.archlinux.org/title/Downgrading_packages) and it worked. How do I upgrade my kernel now without re-breaking things?

Make sure your boot partition is mounted when you upgrade the kernel. Otherwise /boot is not your boot partition but inside your root partition. When you reboot, you get the old kernel (which is still on /boot) but have updated modules (in /usr/lib on the root partition).

pasturegliding · 2022-10-30 01:17:45

cfr wrote:

pasturegliding wrote:
arch-chroot /dev/mapper/c
You need to mount the partition first. I assume you did something like
cryptsetup open /dev/sdb2 c
? So c is the LUKS container? But you want to mount the root partition and arch-chroot into that. You also need your boot partition mounted before you reinstall the kernel.

Yes, c was the name of the LUKS container.
I figured it out.

When I had deleted the /boot partition from Disk 1, I had forgotten that parted just sets the filesystem type as Fat32, and does not actually put a Fat32 system on there, so the first thing I had to do was

# mkfs.vfat -F 32 /dev/sdb1

In order to mount it. I must have been booting off of another disk without knowing it. I did that, mounted it, deleted everything in /boot, and, knowing the only packages would be the kernel, ran

# pacman -Syu

Then

$ refind-install

Then

# mkinitcpio -P

Then I rebooted and everything is good in the world again lol

Thanks for your help

Arch Linux

#1 2022-10-29 20:16:28

[SOLVED] Emergency mode at boot after rsync (systemd issue?)

#2 2022-10-29 20:52:55

Re: [SOLVED] Emergency mode at boot after rsync (systemd issue?)

#3 2022-10-29 21:11:39

Re: [SOLVED] Emergency mode at boot after rsync (systemd issue?)

#4 2022-10-29 22:50:02

Re: [SOLVED] Emergency mode at boot after rsync (systemd issue?)

#5 2022-10-29 23:30:01

Re: [SOLVED] Emergency mode at boot after rsync (systemd issue?)

#6 2022-10-29 23:40:25

Re: [SOLVED] Emergency mode at boot after rsync (systemd issue?)

#7 2022-10-30 00:04:36

Re: [SOLVED] Emergency mode at boot after rsync (systemd issue?)

#8 2022-10-30 00:07:05

Re: [SOLVED] Emergency mode at boot after rsync (systemd issue?)

#9 2022-10-30 01:17:45

Re: [SOLVED] Emergency mode at boot after rsync (systemd issue?)

Board footer