Arch Linux

apaz

Member

Registered: 2018-07-23

Posts: 116

Linux 5.18 and boot time

Switching from kernel 5.17 to 5.18 increased my boot time by 3 seconds.

$ systemd-analyze
Startup finished in 12.433s (firmware) + 952ms (loader) + 4.531s (kernel) + 1.512s (userspace) = 19.429s
graphical.target reached after 1.511s in userspace.

This increase is only due to the boot "kernel" step, which went from max=1.5s to 4.5s.
Have you also noticed this increase or does it only affect my system? Do you have any advice on how to investigate the cause?
I attach the dmesg:

https://www.dropbox.com/s/ypbebg0yxil4i … g.txt?dl=0

PS: out of personal curiosity I do a systemd-analyze and geekbench at every kernel, graphics driver and Bios update! I can say that the timing of the "firmware" step is quite variable; but the "kernel" step has always been the same for years and years, until now...

venomega

Member

Registered: 2022-05-30

Posts: 2

Re: Linux 5.18 and boot time

i noted few minutes ago that booting lts is much faster that booting 5.18

apaz

Member

Registered: 2018-07-23

Posts: 116

Re: Linux 5.18 and boot time

Thanks; it would seem that 5.18 has a slower boot (while as performance I notice an improvement over 5.17).
Did you do a "systemd-analyze" to see where the 2 boot parts differ? The following command might also be useful:

systemd-analyze plot > boot.svg

That produces an svg image.

V1del

Forum Moderator

Registered: 2012-10-16

Posts: 21,627

Re: Linux 5.18 and boot time

Journal wise it looks like the XFS root is at play here? Does the FS maybe have a role? FWIW it looks like a lot of changes happened in the random subsystem and changes there can definitely affect a lot of stuff during early boot. Quite a bit was adjusted for the 5.18.1 release as well so there might be improvements there, from reading a few of the changes there where a load of functions that simply returned a constant rather than a random value in situations where a random value would be preferred so maybe that's a fallout from that?

stanczew

Member

Registered: 2021-03-02

Posts: 63

Re: Linux 5.18 and boot time

After updating to 5.18, my boot time for kernel went from ~1.9 s to ~5.5 s

$ journalctl | grep '(Linux version|Startup finished.*kernel)'
May 27 13:41:01 nyx kernel: Linux version 5.17.9-arch1-1 (linux@archlinux) (gcc (GCC) 12.1.0, GNU ld (GNU Binutils) 2.38) #1 SMP PREEMPT Wed, 18 May 2022 17:30:11 +0000
May 27 13:41:01 nyx systemd[1]: Startup finished in 16.319s (firmware) + 221ms (loader) + 1.973s (kernel) + 680ms (userspace) = 19.195s.
May 28 11:01:58 nyx kernel: Linux version 5.17.9-arch1-1 (linux@archlinux) (gcc (GCC) 12.1.0, GNU ld (GNU Binutils) 2.38) #1 SMP PREEMPT Wed, 18 May 2022 17:30:11 +0000
May 28 11:01:59 nyx systemd[1]: Startup finished in 16.176s (firmware) + 221ms (loader) + 1.971s (kernel) + 1.258s (userspace) = 19.628s.
May 28 18:53:01 nyx kernel: Linux version 5.17.9-arch1-1 (linux@archlinux) (gcc (GCC) 12.1.0, GNU ld (GNU Binutils) 2.38) #1 SMP PREEMPT Wed, 18 May 2022 17:30:11 +0000
May 28 18:53:01 nyx systemd[1]: Startup finished in 16.162s (firmware) + 224ms (loader) + 1.959s (kernel) + 648ms (userspace) = 18.995s.
# Updated to 5.18
May 28 18:54:26 nyx kernel: Linux version 5.18.0-arch1-1 (linux@archlinux) (gcc (GCC) 12.1.0, GNU ld (GNU Binutils) 2.38) #1 SMP PREEMPT_DYNAMIC Tue, 24 May 2022 22:00:36 +0000
May 28 18:54:27 nyx systemd[1]: Startup finished in 16.181s (firmware) + 220ms (loader) + 5.537s (kernel) + 1.272s (userspace) = 23.211s.
May 29 10:23:15 nyx kernel: Linux version 5.18.0-arch1-1 (linux@archlinux) (gcc (GCC) 12.1.0, GNU ld (GNU Binutils) 2.38) #1 SMP PREEMPT_DYNAMIC Tue, 24 May 2022 22:00:36 +0000
May 29 10:23:16 nyx systemd[1]: Startup finished in 16.329s (firmware) + 220ms (loader) + 5.708s (kernel) + 507ms (userspace) = 22.766s.
May 29 11:45:11 nyx kernel: Linux version 5.18.0-arch1-1 (linux@archlinux) (gcc (GCC) 12.1.0, GNU ld (GNU Binutils) 2.38) #1 SMP PREEMPT_DYNAMIC Tue, 24 May 2022 22:00:36 +0000
May 29 11:45:12 nyx systemd[1]: Startup finished in 16.180s (firmware) + 220ms (loader) + 5.502s (kernel) + 413ms (userspace) = 22.317s.

I noticed that with 5.18 the "triggering uevents" step takes a considerably longer time than before. The first 4 lines of logs are printed, and then it just waits for a couple of seconds with a blinking cursor, before proceeding with the rest of the boot.

Journal wise it looks like the XFS root is at play here? Does the FS maybe have a role?

I'm using ext4, and I don't have any XFS partitions.
Also I think journal/dmesg will not be useful here, as there is no "triggering uevents" logs saved in them. (Does this mean the log, and by extension the problem, is in initramfs?)

Quite a bit was adjusted for the 5.18.1 release as well so there might be improvements there

5.18.1 didn't change the behavior:

May 31 09:12:22 nyx kernel: Linux version 5.18.1-arch1-1 (linux@archlinux) (gcc (GCC) 12.1.0, GNU ld (GNU Binutils) 2.38) #1 SMP PREEMPT_DYNAMIC Mon, 30 May 2022 17:53:11 +0000
May 31 09:12:23 nyx systemd[1]: Startup finished in 17.105s (firmware) + 263ms (loader) + 5.478s (kernel) + 1.284s (userspace) = 24.131s.

Edit:
My other system doesn't experience this issue – kernel boot time is basically the same with 5.17 and 5.18:

May 29 13:16:52 abaddon kernel: Linux version 5.17.9-arch1-1 (linux@archlinux) (gcc (GCC) 12.1.0, GNU ld (GNU Binutils) 2.38) #1 SMP PREEMPT Wed, 18 May 2022 17:30:11 +0000
May 29 13:16:53 abaddon systemd[1]: Startup finished in 9.761s (firmware) + 187ms (loader) + 1.153s (kernel) + 1.464s (userspace) = 12.567s.
May 31 09:29:20 abaddon kernel: Linux version 5.18.0-arch1-1 (linux@archlinux) (gcc (GCC) 12.1.0, GNU ld (GNU Binutils) 2.38) #1 SMP PREEMPT_DYNAMIC Tue, 24 May 2022 22:00:36 +0000
May 31 09:29:21 abaddon systemd[1]: Startup finished in 9.774s (firmware) + 188ms (loader) + 1.157s (kernel) + 1.493s (userspace) = 12.614s.
May 31 09:32:57 abaddon kernel: Linux version 5.18.1-arch1-1 (linux@archlinux) (gcc (GCC) 12.1.0, GNU ld (GNU Binutils) 2.38) #1 SMP PREEMPT_DYNAMIC Mon, 30 May 2022 17:53:11 +0000
May 31 09:32:58 abaddon systemd[1]: Startup finished in 9.805s (firmware) + 181ms (loader) + 1.118s (kernel) + 1.460s (userspace) = 12.565s.

Last edited by stanczew (2022-05-31 07:33:43)

apaz

Member

Registered: 2018-07-23

Posts: 116

Re: Linux 5.18 and boot time

@V1del
Yes, I have been using XFS for 4 years; I don't think it has anything to do with booting, since the delay appeared only with the 5.18 kernel.
I used rng-tools with the usual delay due to the 5.17 > 5.18 upgrade. Now that I have removed it (and also upgraded to 5.18.1) the delay has remained the same.

@stanczew
Interesting that the other system does not introduce the delay. My PC is with AMD hardware; Could this be the problem?
Where can I see the "triggering uevents"?

stanczew

Member

Registered: 2021-03-02

Posts: 63

Re: Linux 5.18 and boot time

My PC is with AMD hardware; Could this be the problem?

It could be; my PC which sees the issue has an AMD CPU (3700X), and the one without the issue has an Intel CPU (4460).

Where can I see the "triggering uevents"?

At the very beginning of OS boot, just after BIOS/bootloader. For me it says:

:: running early hook [udev]
Starting version 251.1-1-arch
:: running hook [udev]
:: Triggering uevents...
_

And then there is a noticeable pause, with just the cursor blinking.
(I don't know if this log is saved anywhere, I took a photo and copied the text by hand.)

Sant0s

Member

Registered: 2020-04-05

Posts: 11

Re: Linux 5.18 and boot time

Hi, I have the same issue. Did you manage to solve it?

stanczew

Member

Registered: 2021-03-02

Posts: 63

Re: Linux 5.18 and boot time

Nope, it's still happening with 5.18.7.

apaz

Member

Registered: 2018-07-23

Posts: 116

Re: Linux 5.18 and boot time

No, for me as well. Do you have AMD hardware?

Last edited by apaz (2022-06-30 06:36:51)