Arch Linux

eomanis

Member

Registered: 2013-04-17

Posts: 67

[Solved] What can I expect to see on a netconsole receiver?

Edit 2
I found out that you need to set the kernel options "debug" and "ignore_loglevel" for netconsole to print anything worthwhile.
There is no need to keep systemd off the kernel log, either.

------------------

Edit
A reboot did end up sending some kernel messages to the receiver:

[141846.368611] watchdog: watchdog0: watchdog did not stop!
[141847.768997] watchdog: watchdog0: watchdog did not stop!

Which, on the sender, are not present in the systemd log, because it stops earlier with:

[141844.640231] thesender systemd-journald[1036435]: Received SIGTERM from PID 1 (systemd-shutdow).

This looks as if systemd-journald, as long as it is running, is consuming all kernel messages, causing them not to be propagated to netconsole.

How do I change that? Or will that not matter in a kernel panic situation?

------------------

Is it expected to not see anything until the kernel on the sender panics, when something below systemd kicks in and shunts the panic message to netconsole, or should I also see the "regular" kernel messages, such as those as can be seen on the sender in e. g. journalctl -b -0 -k --follow?

Because so far, after loading the module on the sender machine, I have not seen any such kernel messages arrive at the receiver.

Background:
I have a new headless system, the sender, that sometimes spontaneously reboots without logging anything in the journal, and I want to find out which hardware part I need to replace to make that stop.
Rasdaemon is installed and running, but it has not found anything, therefore I have turned to netconsole to go and catch myself a kernel panic to look at.

The sender has a local IPv4 address of 192.168.0.89.
The receiver has a local IPv4 address of 192.168.0.90.

I have verified that the receiver works; it runs socat in a systemd service that both prints what comes in into the journal and also appends it to a file:

socat -u udp4-recv:10234,range=192.168.0.90/24 - | tee -a nc-recv-thesender.log

On the sender, I can run e. g.

echo test-from-thesender | socat -u - udp4-sendto:192.168.0.90:10234

and it shows up on the receiver all right.

The sender currently has this in /etc/modprobe.d/netconsole.conf:

options netconsole netconsole=@192.168.0.89/br0,10234@192.168.0.90/02:53:32:1e:bb:4e

The sender's kernel log indicates that netconsole has been set up successfully when manually loading it with modprobe after startup:

Aug 19 22:38:06 thesender kernel: netpoll: netconsole: local port 6665
Aug 19 22:38:06 thesender kernel: netpoll: netconsole: local IPv4 address 192.168.0.89
Aug 19 22:38:06 thesender kernel: netpoll: netconsole: interface 'br0'
Aug 19 22:38:06 thesender kernel: netpoll: netconsole: remote port 10234
Aug 19 22:38:06 thesender kernel: netpoll: netconsole: remote IPv4 address 192.168.0.90
Aug 19 22:38:06 thesender kernel: netpoll: netconsole: remote ethernet address 02:53:32:1e:bb:4e
Aug 19 22:38:06 thesender kernel: printk: legacy console [netcon0] enabled
Aug 19 22:38:06 thesender kernel: netconsole: network logging started

I have also tried it without the receiver MAC address after the last slash, without success.

Last edited by eomanis (2026-03-09 17:39:52)