[SOLVED] Headless system fails to boot without monitor attached

tokenize · 2023-01-19 10:51:52

Hello,

After the latest grub and mesa drivers update, my system fails to boot if I don't have a monitor attached. This is very odd. I use this machine headless, so this is a major hassle.
Anyone with the same issue?

Best

EDIT - At first I thought it was a grub change but now I think it's related with the mesa driver update instead.

Last edited by tokenize (2023-01-19 16:25:25)

tokenize · 2023-01-19 11:05:57

Meanwhile I followed the solution from this post https://bbs.archlinux.org/viewtopic.php?id=233260 to add "nomodeset text" to cmdline and uncomment #GRUB_TERMINAL_OUTPUT=console and it solved it.

However, it's very strange, because, until the latest update, I didn't have to do this.

What am I missing?

tokenize · 2023-01-19 11:23:13

My system uses intel integrated graphics and, upon further investigation, I see that the mesa drivers were updated on 18/01/2023, so this could also be the culprit.

So, meanwhile, if you have the same situation as myself, adding the nomodeset kernel option should fix the issue.

seth · 2023-01-19 13:38:28

Don't bump and please post a complete system journal (from a failing boot)

tokenize · 2023-01-19 13:58:33

I was just posting more information as I found it.

Is there a way to fetch the failing boot journal if the system is unreachable? If I plug the monitor it works, if I unplug it it does not work and I have no way to access the system.

seth · 2023-01-19 14:26:14

I was just posting more information as I found it.

That's fine, but unless somebody responded, simply edit you previouss post.

You can access journals of previous boots:

sudo journalctl -b -1 | curl -F 'file=@-' 0x0.st # to feed the previous (-1) boot into a pastebin service

https://wiki.archlinux.org/title/Journalctl

If will be beneficial if you did NOT using the power button, but eg. ctrl+alt+del ("beneficial" means there'll actually be a useful journal)

tokenize · 2023-01-19 14:40:33

When the boot failed I shutdown the system with the power button actually. Will ctrl + alt + del "properly" shutdown if the boot failed?

V1del · 2023-01-19 14:45:38

If the system is responsive and not actually crashed/failed in any way yes it will.

tokenize · 2023-01-19 15:19:10

Unfortunately I can't shutdown the system with ctrl + alt + del when the boot fails. Only with the shutdown button.

Check this journalctl log please: http://0x0.st/ohp5.txt

Last edited by tokenize (2023-01-19 15:20:45)

seth · 2023-01-19 15:22:10

Press the shortcut more frenetically (systemd will only at some point interpret that as desire to shut down)
If that doesn't work, try https://wiki.archlinux.org/title/Keyboa … el_(SysRq) and in any event post the journal of an LTS boot to maybe highlight pot. issues.

Edit: that's just the tail of some journal and endeavour OS isn't supported here.

Last edited by seth (2023-01-19 15:23:51)

V1del · 2023-01-19 15:24:11

Booting into a working setup and doing

sudo journalctl -b-1

does not contain ANY trace of the issue? This sounds more like either not using persistent namespaces or something tripping the initramfs up. FWIW what are your current

/etc/fstab
/boot/grub/grub.cfg

random guess, remove the kms hook from mkinitcpio.conf and regenerate the initramfs but that's a stretch.

F5...

But yes as stated in the rules you agreed to when registering, we don't know what Endeavour OS is configuring/doing that might have an effect here https://bbs.archlinux.org/misc.php?action=rules

Closing.

Last edited by V1del (2023-01-19 16:07:40)

seth · 2023-01-19 15:25:26

OP can boot nomodeset, though.

tokenize · 2023-01-19 15:53:59

My topic was closed but just wanted to announce, in case you get the same problem as I did.

If you are using an Intel integrated GPU with the mesa driver, after the last update you need to enable early KMS as stated in the wiki here: https://wiki.archlinux.org/title/Kernel … _KMS_start

Basically you need to add i915 to the MODULES array in the mkinitcpio.conf file and then regenerate it with mkinitcpio -P.

Hope someone finds this helpful.

PS - To the mods that closed my previous topic: I'm not using endeavourOS, the "endeavour" string you saw in my pastebin is the hostname of my machine. My system is running Arch Linux, not some distro.

V1del · 2023-01-19 16:07:17

Reinstating and merging the follow up on the report that this might be a coincidence.

Early KMS should not be entirely relevant on a headless system normally. Are you booting a graphical.target regardless?

Also note for the future that you can make a "report" even on a moderator action if you have an argument against the closure - and this would be the preferred way if you think it was closed in error.

Last edited by V1del (2023-01-19 16:12:05)

tokenize · 2023-01-19 16:21:33

V1del wrote:

Early KMS should not be entirely relevant on a headless system normally. Are you booting a graphical.target regardless?

My installation does not contain any graphical packages. If I attach a monitor to the machine I will get a command line prompt. I only installed the base package and the mesa drivers so that I have hardware acceleration on video processing tasks.
I don't know why, but my system will only boot without nomodeset when using early KMS. This has to be related with the latest mesa driver update, because before today's update I didn't need it. I update my system every week and never had this problem before.

V1del wrote:

Also note for the future that you can make a "report" even on a moderator action if you have an argument against the closure - and this would be the preferred way if you think it was closed in error.

Thanks for the info, will do.

Last edited by tokenize (2023-01-19 16:24:49)

seth · 2023-01-19 16:24:45

Please post a complete system journal now.

sudo journalctl -b | curl -F 'file=@-' 0x0.st

I can currently not imagine how early kms would get you from "doesn't respond to ctrl+alt+del" to "boots totally fine"
You did try the shortcut from a physially attached keyboard?

tokenize · 2023-01-19 16:28:16

Here is the journal with the command you posted: http://0x0.st/ohfL.txt

I spammed ctrl+alt+del from an attached keyboard that I made sure was working before I got the system into the unbootable configuration.

V1del · 2023-01-19 16:32:54

Are you starting any of these video processing tasks on bootup directly? I could see a relation if something in the systemd startup chain as part of multi-user.target (maybe even earlier?) or so would need access to the graphics card and then stalls getting insufficient access. This sounds quite strange...

tokenize · 2023-01-19 16:34:36

Actually, I'm firing up some ffmpeg commands with systemd multi-user.target. But this was never a problem before.
Also some docker containers that might use the GPU.

Last edited by tokenize (2023-01-19 16:35:19)

V1del · 2023-01-19 16:39:26

Well then this starts to make more sense and I can see why you'd need the device available potentially earlier. I somewhat doubt the mesa update is directly related, but the inherent raciness of service startup on systemd can occasionally hit the case that your driver will not be ready by the time something wants to use it.

tokenize · 2023-01-19 16:43:02

Well, then it was a coincidence maybe? It's really strange though, that after more than a year with this exact hardware and software setup, I only get this problem now.

In any case, this wouldn't explain why the system would boot if I plugged in a monitor. This is the strange part for me. If the monitor was plugged the system booted. If I unplugged the monitor it wouldn't boot anymore. Doesn't make any sense.

V1del · 2023-01-19 16:46:21

The "necessity" of early KMS is inherently due to things being unpredictable without it because stuff is initialized in parallel, maybe adding the monitor induced "just" enough of additional verification/checking that the device would come available for the services earlier.... or so. None of that explains that nothing landed in the journal though

tokenize · 2023-01-19 16:49:56

Well, that make senses. Do you know if there are any drawbacks of having early KMS? If not, I will just leave it like this and forget about it.

seth · 2023-01-19 17:15:47

Do you know if there are any drawbacks of having early KMS?

None.

The video service is

jan 19 16:04:56 endeavour systemd[1]: Started My service to record

?
That'd be well after

jan 19 15:34:47 endeavour systemd[1]: Reached target Graphical Interface.

Docker could be a problem?
But I still don't see anything in that journal that could prevent a regular boot and be fixed by early kms.

Sanity check: if you remove i915 from the initramfs again, does the problem re-appear?
Maybe it was simply the initramfs rebuild?

tokenize · 2023-01-19 17:34:02

Well, you were right. I removed early KMS, regenerated initramfs and it's working... I don't know what happened.

I can't say 100% now, but I saw that the pacman system update successfully regenerated initramfs in the post operations so I thought everything was fine.

So I wasted everyone's time for nothing, sorry. Maybe you can close the topic since there's no issue after all.

Arch Linux

#1 2023-01-19 10:51:52

[SOLVED] Headless system fails to boot without monitor attached

#2 2023-01-19 11:05:57

Re: [SOLVED] Headless system fails to boot without monitor attached

#3 2023-01-19 11:23:13

Re: [SOLVED] Headless system fails to boot without monitor attached

#4 2023-01-19 13:38:28

Re: [SOLVED] Headless system fails to boot without monitor attached

#5 2023-01-19 13:58:33

Re: [SOLVED] Headless system fails to boot without monitor attached

#6 2023-01-19 14:26:14

Re: [SOLVED] Headless system fails to boot without monitor attached

#7 2023-01-19 14:40:33

Re: [SOLVED] Headless system fails to boot without monitor attached

#8 2023-01-19 14:45:38

Re: [SOLVED] Headless system fails to boot without monitor attached

#9 2023-01-19 15:19:10

Re: [SOLVED] Headless system fails to boot without monitor attached

#10 2023-01-19 15:22:10

Re: [SOLVED] Headless system fails to boot without monitor attached

#11 2023-01-19 15:24:11

Re: [SOLVED] Headless system fails to boot without monitor attached

#12 2023-01-19 15:25:26

Re: [SOLVED] Headless system fails to boot without monitor attached

#13 2023-01-19 15:53:59

Re: [SOLVED] Headless system fails to boot without monitor attached

#14 2023-01-19 16:07:17

Re: [SOLVED] Headless system fails to boot without monitor attached

#15 2023-01-19 16:21:33

Re: [SOLVED] Headless system fails to boot without monitor attached

#16 2023-01-19 16:24:45

Re: [SOLVED] Headless system fails to boot without monitor attached

#17 2023-01-19 16:28:16

Re: [SOLVED] Headless system fails to boot without monitor attached

#18 2023-01-19 16:32:54

Re: [SOLVED] Headless system fails to boot without monitor attached

#19 2023-01-19 16:34:36

Re: [SOLVED] Headless system fails to boot without monitor attached

#20 2023-01-19 16:39:26

Re: [SOLVED] Headless system fails to boot without monitor attached

#21 2023-01-19 16:43:02

Re: [SOLVED] Headless system fails to boot without monitor attached

#22 2023-01-19 16:46:21

Re: [SOLVED] Headless system fails to boot without monitor attached

#23 2023-01-19 16:49:56

Re: [SOLVED] Headless system fails to boot without monitor attached

#24 2023-01-19 17:15:47

Re: [SOLVED] Headless system fails to boot without monitor attached

#25 2023-01-19 17:34:02

Re: [SOLVED] Headless system fails to boot without monitor attached

Board footer