[SOLVED] Systemd regression. Broken sayodevice keyboard.

Volfit · 2026-04-10 09:24:47

A few days before I've updated my system with pacman -Syu and yesterday when I tried to use my sayodevice O3C keyboard it didn't worked. More exact, it worked in tty, sddm, xfce, but didn't when I logged in any wayland session like hyprland, kde plasma and niri. After nuking my main system and partially upgrading my second archlinux system I manage to replicate the issue when updating next packages:

systemd
systemd-sysvcompat
systemd-libs

Downgrading those packages fixed the issue:

downgrading package systemd (260.1-1 => 258.2-2)
warning: downgrading package systemd-sysvcompat (260.1-1 => 258.2-2)
warning: downgrading package systemd-libs (260.1-1 => 258.2-2)

It's my first post, and I know it's can be wrong place to make a report about regression, so I'm looking to guidance.

Last edited by Volfit (2026-04-14 08:14:16)

seth · 2026-04-10 15:38:18

Probably some udev rule for access control, recently keyboards w/ a bogus joystick device have flared up.
Ideally post a journal for the good and bad config.
Please post your complete system journal for the boot:

sudo journalctl -b -δ | curl -s -H "Accept: application/json, */*" --upload-file - 'https://paste.c-net.org/'

will upload the journal of a previous boot, the bigger δ the older the journal, "0" is the current boot.

Volfit · 2026-04-10 17:33:27

Here's logs with old systemd version(258.2-2)
And with new version (260.1-1)

Last edited by Volfit (2026-04-11 06:22:22)

seth · 2026-04-10 23:08:07

No such problem w/ the logitech keyboard?

Apr 10 20:27:55 archlinux kernel: usb 6-1: USB disconnect, device number 2

You physically reconnected the device?

Apr 10 20:25:02 archlinux otd-daemon[1117]: [HidSharpDeviceRootHub:Debug]        Changes: 2, Add: 2, Remove: 0

The thing doesn't show up as keyboard - is it handled by otd-daemon?
This line is missing, one reason might be that otd-daemon starts 10s before the SayoDevice shows up in the 260.1 boot.

Volfit · 2026-04-11 07:07:17

No, it's not a otd-daemon related. I'm sorry for that, I had a otd-daemon started around when I plugged the keyboard, and it's decided to do it's things right after the plug.
Also, yes. In second journal I did plug the keyboard, I've tried to type, and then I replugged it. It didn't worked.
And I also never had any problem with any device on both my systems, except the SayoDevice.

Also something interesting just happened. Yesterday, I did two clean installs with Niri on my main system and later another clean install with kde plasma. Both times keyboard didn't worked untill I downgraded the systemd, but I was still doing postinstall. But when I tried to update the systemd today the keyboard decided to work, but only on the main system. The second system that I've given you original logs from is still infected.

Should I try update it?
Is it actually systemd related anymore?

Here's journal from my main (good) system.

systemd 258.2-2
systemd 260.1-1

You will see PenTablet here, but no otd-daemon, as I didn't installed it yet.

Also, here's another logs I did by testing both my systems with both versions of systemd: (260.1-1 now works on main system)

I've redone the logs below. Now you don't need to download them, and they're much cleaner. Still, the results are the same.
second (bad) system old systemd
second (bad) system new systemd
main (good) system old systemd
main (good) system new systemd

In bad system with new systemd the keyboard does not work, but the evtest shows the opposite.
What's also interesting is now on my main (good) clean installed system systemd and kwin_wayland grab my Sayodevice, and with new systemd they doesn't, when on my second (bad) system they does not seem to do so.

Another interesting detail is that on both systems with systemd(260.1-1) when using webapp for Sayodevice when connecting it identifies itself as Unknown device (8089:0009).
When using systemd version(258.2-2) on both system it identifies in the same webapp as Sayodevice.

I will also point out that I have one udev rule from Here.
This is the only udev rule I have for Sayodevice, it's needed to fix the connectivity to webapp and removing it does not affects the issue.

Last edited by Volfit (2026-04-12 17:51:22)

seth · 2026-04-11 13:12:26

Apr 11 09:06:00 archlinux kernel: usb 1-2: new full-speed USB device number 3 using xhci_hcd
Apr 11 09:06:00 archlinux kernel: usb 1-2: New USB device found, idVendor=8089, idProduct=0009, bcdDevice= 9.0c
Apr 11 09:06:00 archlinux kernel: usb 1-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
Apr 11 09:06:00 archlinux kernel: [19B blob data]
Apr 11 09:06:00 archlinux kernel: usb 1-2: Manufacturer: SayoDevice
Apr 11 09:06:00 archlinux kernel: usb 1-2: SerialNumber: 004DC71088197076BF39E339E30000
Apr 11 09:06:00 archlinux kernel: [119B blob data]
Apr 11 09:06:00 archlinux flatpak[1204]: Info: grabbed device: /dev/input/event22
Apr 11 09:06:00 archlinux kernel: [114B blob data]
Apr 11 09:06:00 archlinux kernel: [119B blob data]
Apr 11 09:06:00 archlinux kernel: [128B blob data]
Apr 11 09:06:00 archlinux flatpak[1204]: Info: grabbed device: /dev/input/event24
Apr 11 09:06:00 archlinux kernel: [123B blob data]
Apr 11 09:06:00 archlinux mtp-probe[1998]: checking bus 1, device 3: "/sys/devices/pci0000:00/0000:00:01.3/0000:01:00.0/usb1/1-2"
Apr 11 09:06:00 archlinux mtp-probe[1998]: bus: 1, device: 3 was not an MTP device
Apr 11 09:06:00 archlinux systemd-logind[615]: [60B blob data]
Apr 11 09:06:00 archlinux kwin_wayland[899]: Failed to open /dev/input/event22 device (No such device)
Apr 11 09:06:00 archlinux systemd-logind[615]: [69B blob data]
Apr 11 09:06:00 archlinux kwin_wayland[899]: Failed to open /dev/input/event23 device (No such device)
Apr 11 09:06:00 archlinux kwin_wayland[899]: Failed to open /dev/input/event24 device (No such device)

but non of that grabbery is in the journals in #3 - nor does any of the

Apr 11 08:55:16 archlinux kernel: usb 3-1: Product: USB KEYBOARD
Apr 11 08:55:16 archlinux kernel: input: SINO WEALTH USB KEYBOARD as /devices/pci0000:00/0000:00:07.1/0000:09:00.3/usb3/3-1/3-1:1.0/0003:258A:0001.0001/input/input2
Apr 11 08:55:16 archlinux kernel: hid-generic 0003:258A:0001.0001: input,hidraw0: USB HID v1.10 Keyboard [SINO WEALTH USB KEYBOARD] on usb-0000:09:00.3-1/input0
Apr 11 08:55:16 archlinux kernel: input: SINO WEALTH USB KEYBOARD System Control as /devices/pci0000:00/0000:00:07.1/0000:09:00.3/usb3/3-1/3-1:1.1/0003:258A:0001.0002/input/input3
Apr 11 08:55:16 archlinux kernel: input: SINO WEALTH USB KEYBOARD Consumer Control as /devices/pci0000:00/0000:00:07.1/0000:09:00.3/usb3/3-1/3-1:1.1/0003:258A:0001.0002/input/input4

show up in the older journals.

Let's see what the device explicitly actually does.
Boot w/o it, run

sudo journalctl -f

attach it and post the tail that generates on a good and a bad system.

Volfit · 2026-04-11 16:46:56

Here's output from journalctl -f

good system old systemd
good system new systemd
bad system old systemd
bad system new systemd

In third log you'll see otd-daemon. I've already mentioned that's has nothing to do with the Sayodevice, however everytime I reconnected the keyboard it appeared right after. After updating systemd it never reacted to the keyboard. That's why it's missing in fourth log.

seth · 2026-04-11 19:45:56

everytime I reconnected the keyboard it appeared right after. After updating systemd it never reacted to the keyboard

There's gonna be a udev rule blocking it (I assume it itself is triggered by udev rules) and that rule will likely also block more crucial ones.

https://wiki.archlinux.org/title/Udev#Debug_output will blow up the journal a bit but hopefully highlight the relevant bit
Possibly

sudo udevadm control --log-priority=debug

is sufficient to enable debugging at runtime and then plugging the SayoDevice

Volfit · 2026-04-12 09:14:13

udevadm control --log-priority=debug

bad system old systemd
bad system new systemd
good system old systemd
good system new systemd

There's gonna be a udev rule blocking it

Can be. If I exclude otd-daemon from logs, in both systems after updating systemd the amount of logs goes down by about 200 lines (excluding the otd-daemon).

Last edited by Volfit (2026-04-12 09:22:41)

seth · 2026-04-12 14:39:08

This is new (on both systems)

Apr 11 23:11:17 archlinux systemd-udevd[462]: sd-device-monitor(manager): Failed to create device from received message: Invalid argument

Comes from https://github.com/systemd/systemd/blob … tor.c#L667 and previously was triggered by a race condition… https://github.com/systemd/systemd/issues/11314

Patched by https://github.com/triton/systemd/commi … 8375eb7fc4 "Fixes systemd#11314 (hopefully )." - might have re-emerged?

Volfit · 2026-04-12 15:23:18

That's possible. I should've reported the issue on systemd github first.

Any way to test this?

I'll do a bit more testing. I want to test different systemd versions to find when regression started for me. I'll also create a clean system install with some extra space I have on my drive, to see how it behaves on a very basic, clean system.
After that I'll report issue to https://github.com/systemd/systemd/

Last edited by Volfit (2026-04-12 15:26:22)

schard · 2026-04-13 08:27:34

Mod note: Moving to Kernel & Hardware Issues.

Volfit · 2026-04-13 11:06:31

The regression started from systemd_259.4-1
The last version where the issue doesn't exist is systemd_259.3-1

Interesting detail is that you don't need to reboot after systemd was installed, and your Initramfs was rebuid. Your keyboard will work if it was an upgrade, however replugging it will result the lost of it's functionality. The opposite will happen if it was a downgrade.

I've also made multiple testings on a test system, to confirm that the main issue exist purely on wayland sessions, and to eliminate the black magic that happened on my good system.
The test system was just a base system made with archinstall, with firefox as additional package, and multiple DEs. It also was made on the same machine, as the good system.

So far, I can confirm that the SayoDevice keyboard worked in every X11 session I've tested + sddm and tty, despite the systemd version.
It never worked in any Wayland session I've tested, untill I downgraded the systemd to systemd_259.3-1.
Also, every test with systemd_259.4-1 and systemd_260.1-1 contained:

sd-device-monitor(manager): Failed to create device from received message: Invalid argument

Despite what session it was.

If you want to take a look at logs, I've collected all of them in Here

I also opened an issue on systemd github: https://github.com/systemd/systemd/issues/41614

If there will be any news, I'll let know.

Last edited by Volfit (2026-04-13 11:12:56)

seth · 2026-04-13 12:33:21

systemd will get re-executed and udev rules reload
https://github.com/systemd/systemd/comp … ...v259.4/ - probably one of the .hwdb patches?
Didn't spot your vendor/product ID though

Volfit · 2026-04-13 12:55:38

seth wrote:
Didn't spot your vendor/product ID though

If I understand correctly, you get it with lsusb command?
If yes, then I've made evtest previously, where I've also run that command, and you can also find vendor/product ID in pretty much every journal I've posted, or added to the google drive.

Apr 13 12:08:40 archlinux kernel: usb 1-2: new full-speed USB device number 33 using xhci_hcd
Apr 13 12:08:41 archlinux kernel: usb 1-2: New USB device found, idVendor=8089, idProduct=0009, bcdDevice= 9.0c
Apr 13 12:08:41 archlinux kernel: usb 1-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
Apr 13 12:08:41 archlinux kernel: [19B blob data]
Apr 13 12:08:41 archlinux kernel: usb 1-2: Manufacturer: SayoDevice

seth · 2026-04-13 13:34:09

no, I meant in the patches.

Volfit · 2026-04-13 13:37:14

Well... My bad

To be honest, this is expected, considering how rare my device is.

Volfit · 2026-04-13 15:12:12

Hi. I think I fixed the SayoDevice.
As I mentioned before I had missing naming for the SayoDevice.

I accidentally found out that there's a desktop app for it, and when trying it out I found out that it can be renamed. I remember renaming it back when I was setting it up on windows long before. After that I've tried the keyboard on my good system, bad system and test system. On all of them the keyboard worked with the newest systemd vesrion, and it was named correctly in both web app and desktop app.

Also, I no longer see:

Failed to create device from received message: Invalid argument

good system systemd 258.2-2 -all -f -debug
good system systemd 260.1-1 -all -f -debug

I think, the problem was in that naming, I've always seen a character that couldn't be displayed. The newer version of systemd likely didn't liked it.

https://www.pasteboard.co/kWhE3YPMND6D.png

Even the otd-daemon now reacts to it on my bad, second system.

Now when I look more in the logs, I see messages like this:

Apr 13 16:29:16 archlinux (udev-worker)[37641]: 0003:8089:0009.0010: sd-device: Failed to handle uevent entry 'HID_NAME=SayoDevice ', ignoring: Invalid argument

bad system systemd-259.3-1 journal --all -f debug
bad system systemd-259.4-1 journal --all -f debug
bad system systemd-260.1-1 journal --all -f debug

It appears in systemd_259.4-1 and above, but not in 259.3-1 and below, only when running:

udevadm control --log-priority=debug
journactl --all -f

Last edited by Volfit (2026-04-13 15:51:21)

Arch Linux

#1 2026-04-10 09:24:47

[SOLVED] Systemd regression. Broken sayodevice keyboard.

#2 2026-04-10 15:38:18

Re: [SOLVED] Systemd regression. Broken sayodevice keyboard.

#3 2026-04-10 17:33:27

Re: [SOLVED] Systemd regression. Broken sayodevice keyboard.

#4 2026-04-10 23:08:07

Re: [SOLVED] Systemd regression. Broken sayodevice keyboard.

#5 2026-04-11 07:07:17

Re: [SOLVED] Systemd regression. Broken sayodevice keyboard.

#6 2026-04-11 13:12:26

Re: [SOLVED] Systemd regression. Broken sayodevice keyboard.

#7 2026-04-11 16:46:56

Re: [SOLVED] Systemd regression. Broken sayodevice keyboard.

#8 2026-04-11 19:45:56

Re: [SOLVED] Systemd regression. Broken sayodevice keyboard.

#9 2026-04-12 09:14:13

Re: [SOLVED] Systemd regression. Broken sayodevice keyboard.

#10 2026-04-12 14:39:08

Re: [SOLVED] Systemd regression. Broken sayodevice keyboard.

#11 2026-04-12 15:23:18

Re: [SOLVED] Systemd regression. Broken sayodevice keyboard.

#12 2026-04-13 08:27:34

Re: [SOLVED] Systemd regression. Broken sayodevice keyboard.

#13 2026-04-13 11:06:31

Re: [SOLVED] Systemd regression. Broken sayodevice keyboard.

#14 2026-04-13 12:33:21

Re: [SOLVED] Systemd regression. Broken sayodevice keyboard.

#15 2026-04-13 12:55:38

Re: [SOLVED] Systemd regression. Broken sayodevice keyboard.

#16 2026-04-13 13:34:09

Re: [SOLVED] Systemd regression. Broken sayodevice keyboard.

#17 2026-04-13 13:37:14

Re: [SOLVED] Systemd regression. Broken sayodevice keyboard.

#18 2026-04-13 15:12:12

Re: [SOLVED] Systemd regression. Broken sayodevice keyboard.

Board footer