/dev/kvm keeps changing group to vboxusers after reboot

abutanaev · 2024-09-02 11:09:07

Hi,

I uninstalled virtualbox, but nevertheless after reboot I have the following:

$ ls -l /dev/kvm 
crw-rw---- 1 root vboxusers 10, 232 сен  2 18:00 /dev/kvm

And that group virtualbox also didn't remove. Maybe I need to remove somewhat else?

pacman -Qs |grep  virtualbox

Gives nothing

Anton

Last edited by abutanaev (2024-09-07 07:05:09)

astralc · 2024-09-02 12:09:46

groups and users are not removed when you remove packages, by design.

what is the group number of vboxusers (I think it should be 108) and kvm group?
the group is assigned by udev rule (on my machine it is the default from /usr/lib/udev/rules.d/50-udev-default.rules)

also, what is the output of

stat /dev/kvm

abutanaev · 2024-09-06 14:49:16

Hi,

$ stat /dev/kvm
  File: /dev/kvm
  Size: 0         	Blocks: 0          IO Block: 4096   character special file
Device: 0,6	Inode: 694         Links: 1     Device type: 10,232
Access: (0660/crw-rw----)  Uid: (    0/    root)   Gid: (  108/vboxusers)
Access: 2024-09-06 21:35:58.306666918 +0700
Modify: 2024-09-06 21:35:58.306666918 +0700
Change: 2024-09-06 21:35:58.306666918 +0700
 Birth: 2024-09-06 21:35:49.433333396 +0700

$ sudo grep -r vboxusers /etc/
/etc/group:vboxusers:x:108:
/etc/gshadow:vboxusers:!*::
/etc/group-:vboxusers:x:108:anton
/etc/gshadow-:vboxusers:!*::anton

$ sudo grep -r vboxusers /usr

Nothing about gid 108 or vboxusers in /usr/lib/udev/rules.d/50-udev-default.rules

abutanaev · 2024-09-06 15:32:55

Maybe the problem is that I first installed virtualbox. /dev/kvm somehow was created with group vboxusers? Then I installed qemu, and after that I removed virtualbox (and later manually removed vboxusers group also). Now after reboot I see:

$ ls -l /dev/kvm 
crw-rw---- 1 root 108 10, 232 сен  6 22:25 /dev/kvm

I cannot find any place where this gid 108 is stored. Maybe it's in some image? sudo mkinitcpio -p linux didn't help

abutanaev · 2024-09-06 15:36:52

What is the right way to change /dev/kvm group on boot? I tried to use file /etc/udev/rules.d/65-kvm.rules with this content:

KERNEL=="kvm", NAME="%k", GROUP="kvm", MODE="0660"

didn't help

abutanaev · 2024-09-06 15:38:21

Maybe change kvm group gid to 108?

seth · 2024-09-06 20:06:03

Maybe not.

grep -rE '(kvm|108|vboxusers)' /{etc,usr/lib}/udev/ | grep -v '.hwdb'

abutanaev · 2024-09-07 02:03:04

$ grep -rE '(kvm|108|vboxusers)' /{etc,usr/lib}/udev/ | grep -v '.hwdb'
/usr/lib/udev/rules.d/50-udev-default.rules:SUBSYSTEM=="ptp", ATTR{clock_name}=="KVM virtual PTP", SYMLINK+="ptp_kvm"
/usr/lib/udev/rules.d/50-udev-default.rules:KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"
/usr/lib/udev/rules.d/50-udev-default.rules:KERNEL=="vhost-vsock", GROUP="kvm", MODE="0666", OPTIONS+="static_node=vhost-vsock"
/usr/lib/udev/rules.d/50-udev-default.rules:KERNEL=="vhost-net", GROUP="kvm", MODE="0666", OPTIONS+="static_node=vhost-net"
/usr/lib/udev/rules.d/50-udev-default.rules:KERNEL=="udmabuf", GROUP="kvm"
/usr/lib/udev/rules.d/90-pulseaudio.rules:# CA0108/CA10300
/usr/lib/udev/rc_keymaps/imon_rsc.toml:0x801081 = "KEY_SCREEN"
/usr/lib/udev/rc_keymaps/imon_rsc.toml:0x801082 = "KEY_ZOOM"
/usr/lib/udev/rc_keymaps/imon_rsc.toml:0x801083 = "KEY_MENU"
/usr/lib/udev/rc_keymaps/imon_rsc.toml:0x801084 = "KEY_STOP"
/usr/lib/udev/rc_keymaps/imon_rsc.toml:0x801085 = "KEY_BACKSPACE"
/usr/lib/udev/rc_keymaps/imon_rsc.toml:0x801086 = "KEY_KEYBOARD"
/usr/lib/udev/rc_keymaps/imon_rsc.toml:0x801087 = "KEY_SPACE"
/usr/lib/udev/rc_keymaps/imon_rsc.toml:0x801088 = "KEY_MUTE"
grep: /usr/lib/udev/hwdb.bin: binary file matches
/usr/lib/udev/rc_keymaps/imon_rsc.toml:0x801089 = "KEY_FULL_SCREEN"
/usr/lib/udev/rc_keymaps/x96max.toml:0x108 = "KEY_NUMERIC_6"
/usr/lib/udev/rc_keymaps/kworld_315u.toml:0x6108 = "KEY_NUMERIC_2"

abutanaev · 2024-09-07 04:09:39

Tried to

sudo systemd-hwdb update
sudo udevadm hwdb --update # deprecated

no effect

abutanaev · 2024-09-07 05:49:22

If /dev/kvm group is set up by udev on boot, where it (udev) gets this 108/vboxusers gid? I've grepped all the /usr /etc no leftover from virtualbox seems left...
Also:

/usr/lib/udev/rules.d/50-udev-default.rules:KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"

doesn't work

abutanaev · 2024-09-07 05:58:40

$ grep 'GROUP="kvm"' /usr/lib/udev/rules.d/50-udev-default.rules
KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"
KERNEL=="vhost-vsock", GROUP="kvm", MODE="0666", OPTIONS+="static_node=vhost-vsock"
KERNEL=="vhost-net", GROUP="kvm", MODE="0666", OPTIONS+="static_node=vhost-net"
KERNEL=="udmabuf", GROUP="kvm"

$ ls -l /dev/{kvm,vhost-vsock,vhost-net,udmabuf}
crw-rw---- 1 root vboxusers       10, 232 сен  7 12:09 /dev/kvm
crw-rw---- 1 root kvm       10, 125 сен  7 12:08 /dev/udmabuf
crw-rw---- 1 root vboxusers 10, 238 сен  7 12:08 /dev/vhost-net
crw-rw---- 1 root vboxusers 10, 241 сен  7 12:08 /dev/vhost-vsock

So it seems that udmabuf group is correct but kvm,vhost-vsock,vhost-net gid is wrong

$ grep 'vboxusers\|kvm' /etc/group
kvm:x:992:libvirt-qemu,qemu
vboxusers:x:108:anton

abutanaev · 2024-09-07 07:17:23

$ udevadm test $(udevadm info --query=path --name=/dev/kvm)
This program is for debugging only, it does not run any program
specified by a RUN key. It may show incorrect results, because
some values may be different, or not available at a simulation run.

Using default interface naming scheme 'v255'.
Properties:
  DEVPATH=/devices/virtual/misc/kvm
  DEVNAME=/dev/kvm
  MAJOR=10
  MINOR=232
  ACTION=add
  SUBSYSTEM=misc
  USEC_INITIALIZED=7634191784
Inotify watch:
  disabled
Device node group:
  kvm (gid=992)
Device node permission:
  0666

It seems that udev is ok and something overwrites it afterwards...

seth · 2024-09-07 07:24:54

Please don't bump, edit your previous post if nobody has replied yet.
https://wiki.archlinux.org/title/Audit_ … ies_access
nb. the part where you add the rules to /etc/audit/audit.rules as the system is transient and loses its state w/ a reboot.

Wild guess: tempfile?
https://wiki.archlinux.org/title/System … rary_files

grep -r kvm /{etc,usr/lib}/tmpfiles.d/

/usr/lib/tmpfiles.d/static-nodes-permissions.conf from the systemd package is supposed to set it to kvm

abutanaev · 2024-09-07 07:30:20

$ grep -r kvm /{etc,usr/lib}/tmpfiles.d/
/usr/lib/tmpfiles.d/static-nodes-permissions.conf:z /dev/kvm          0666 - kvm -
/usr/lib/tmpfiles.d/static-nodes-permissions.conf:z /dev/vhost-net    0666 - kvm -
/usr/lib/tmpfiles.d/static-nodes-permissions.conf:z /dev/vhost-vsock  0666 - kvm -

seth · 2024-09-07 07:41:17

That's what you'd expect.
If you can't find anything suspicious in the journal (ie. whatever changes the GID kindly logs it) you'll have to audit the file.

grep kvm /etc/group # just to be sure that file didn't get somehow corrupted and you've two kvm groups or so…

abutanaev · 2024-09-07 07:46:11

$ grep kvm /etc/group*
/etc/group:kvm:x:992:libvirt-qemu,qemu
/etc/group-:kvm:x:992:libvirt-qemu,qemu

Nothing relevant about kvm or vboxusers or 108 in journal

astralc · 2024-09-07 08:30:19

maybe you have some rouge systemd *.service somewhere that change it (or starting vms)? or some /etc/profile shenanigans?
I see that there is 'change' 9 seconds after it was created. In may machine, after booting, change and birth timestamps was the same (and it "change" only when I started vm in libvirtd).

Change: 2024-09-06 21:35:58.306666918 +0700
 Birth: 2024-09-06 21:35:49.433333396 +0700

Also the mode on your machine is 660 instead 666

abutanaev · 2024-09-07 08:51:54

grepped also

grep -r 'vboxusers' .[^.]*

nothing relevant in my home dir. I grepped the whole /etc and /usr/lib for vbosusers, nothing found

Then I installed hook:

sudo cat /etc/libvirt/hooks/qemu
#!/bin/bash

echo $@ | logger -t qemu-hook

if [[ $2 = prepare || $2 = start ]]
then
	chgrp -v kvm /dev/{kvm,vhost-vsock,vhost-net,udmabuf} | logger -t qemu-hook
fi

and now it works though this is a workaround. Interesting thing:

Sep 07 15:39:42 ursul qemu-hook[2362]: ugdev prepare begin -
Sep 07 15:39:42 ursul qemu-hook[2364]: changed group of '/dev/kvm' from vboxusers to kvm
Sep 07 15:39:42 ursul qemu-hook[2364]: changed group of '/dev/vhost-vsock' from vboxusers to kvm
Sep 07 15:39:42 ursul qemu-hook[2364]: changed group of '/dev/vhost-net' from vboxusers to kvm
Sep 07 15:39:42 ursul qemu-hook[2364]: group of '/dev/udmabuf' retained as kvm
Sep 07 15:39:42 ursul systemd[1]: Starting libvirt logging daemon...
Sep 07 15:39:43 ursul systemd[1]: Started libvirt logging daemon.
Sep 07 15:39:43 ursul kernel: tun: Universal TUN/TAP device driver, 1.6
Sep 07 15:39:43 ursul kernel: virbr0: port 1(vnet0) entered blocking state
Sep 07 15:39:43 ursul kernel: virbr0: port 1(vnet0) entered disabled state
Sep 07 15:39:43 ursul kernel: vnet0: entered allmulticast mode
Sep 07 15:39:43 ursul kernel: vnet0: entered promiscuous mode
Sep 07 15:39:43 ursul kernel: virbr0: port 1(vnet0) entered blocking state
Sep 07 15:39:43 ursul kernel: virbr0: port 1(vnet0) entered listening state
Sep 07 15:39:43 ursul charon[730]: 09[KNL] interface vnet0 activated
Sep 07 15:39:43 ursul NetworkManager[532]: <info>  [1725698383.0331] manager: (vnet0): new Tun device (/org/freedesktop/NetworkManager/Devices/11)
Sep 07 15:39:43 ursul systemd-networkd[376]: vnet0: Link UP
Sep 07 15:39:43 ursul systemd-networkd[376]: vnet0: Gained carrier
Sep 07 15:39:43 ursul NetworkManager[532]: <info>  [1725698383.0367] device (vnet0): state change: unmanaged -> unavailable (reason 'connection-assumed', sys-iface-state: 'external')
Sep 07 15:39:43 ursul NetworkManager[532]: <info>  [1725698383.0369] device (vnet0): state change: unavailable -> disconnected (reason 'connection-assumed', sys-iface-state: 'external')
Sep 07 15:39:43 ursul NetworkManager[532]: <info>  [1725698383.0375] device (vnet0): Activation: starting connection 'vnet0' (0c6cf66d-13e8-4291-b229-9c14ef868e70)
Sep 07 15:39:43 ursul NetworkManager[532]: <info>  [1725698383.0378] device (vnet0): state change: disconnected -> prepare (reason 'none', sys-iface-state: 'external')
Sep 07 15:39:43 ursul NetworkManager[532]: <info>  [1725698383.0380] device (vnet0): state change: prepare -> config (reason 'none', sys-iface-state: 'external')
Sep 07 15:39:43 ursul NetworkManager[532]: <info>  [1725698383.0381] device (vnet0): state change: config -> ip-config (reason 'none', sys-iface-state: 'external')
Sep 07 15:39:43 ursul NetworkManager[532]: <info>  [1725698383.0382] device (virbr0): bridge port vnet0 was attached
Sep 07 15:39:43 ursul NetworkManager[532]: <info>  [1725698383.0382] device (vnet0): Activation: connection 'vnet0' attached as port, continuing activation
Sep 07 15:39:43 ursul NetworkManager[532]: <info>  [1725698383.0391] device (vnet0): state change: ip-config -> ip-check (reason 'none', sys-iface-state: 'external')
Sep 07 15:39:43 ursul systemd[1]: Starting Network Manager Script Dispatcher Service...
Sep 07 15:39:43 ursul charon[730]: 07[KNL] interface vnet1 activated
Sep 07 15:39:43 ursul kernel: virbr1: port 1(vnet1) entered blocking state
Sep 07 15:39:43 ursul kernel: virbr1: port 1(vnet1) entered disabled state
Sep 07 15:39:43 ursul kernel: vnet1: entered allmulticast mode
Sep 07 15:39:43 ursul kernel: vnet1: entered promiscuous mode
Sep 07 15:39:43 ursul kernel: virbr1: port 1(vnet1) entered blocking state
Sep 07 15:39:43 ursul kernel: virbr1: port 1(vnet1) entered listening state
Sep 07 15:39:43 ursul NetworkManager[532]: <info>  [1725698383.0528] manager: (vnet1): new Tun device (/org/freedesktop/NetworkManager/Devices/12)
Sep 07 15:39:43 ursul systemd-networkd[376]: vnet1: Link UP
Sep 07 15:39:43 ursul systemd-networkd[376]: vnet1: Gained carrier
Sep 07 15:39:43 ursul NetworkManager[532]: <info>  [1725698383.0564] device (vnet1): state change: unmanaged -> unavailable (reason 'connection-assumed', sys-iface-state: 'external')
Sep 07 15:39:43 ursul NetworkManager[532]: <info>  [1725698383.0567] device (vnet1): state change: unavailable -> disconnected (reason 'connection-assumed', sys-iface-state: 'external')
Sep 07 15:39:43 ursul NetworkManager[532]: <info>  [1725698383.0570] device (vnet1): Activation: starting connection 'vnet1' (7889321d-6735-48ca-b881-63ef430e6a37)
Sep 07 15:39:43 ursul NetworkManager[532]: <info>  [1725698383.0573] device (vnet1): state change: disconnected -> prepare (reason 'none', sys-iface-state: 'external')
Sep 07 15:39:43 ursul NetworkManager[532]: <info>  [1725698383.0576] device (vnet1): state change: prepare -> config (reason 'none', sys-iface-state: 'external')
Sep 07 15:39:43 ursul NetworkManager[532]: <info>  [1725698383.0577] device (vnet1): state change: config -> ip-config (reason 'none', sys-iface-state: 'external')
Sep 07 15:39:43 ursul NetworkManager[532]: <info>  [1725698383.0578] device (virbr1): bridge port vnet1 was attached
Sep 07 15:39:43 ursul NetworkManager[532]: <info>  [1725698383.0578] device (vnet1): Activation: connection 'vnet1' attached as port, continuing activation
Sep 07 15:39:43 ursul NetworkManager[532]: <info>  [1725698383.0583] device (vnet1): state change: ip-config -> ip-check (reason 'none', sys-iface-state: 'external')
Sep 07 15:39:43 ursul qemu-hook[2390]: ugdev start begin -
Sep 07 15:39:43 ursul qemu-hook[2392]: group of '/dev/kvm' retained as kvm
Sep 07 15:39:43 ursul qemu-hook[2392]: group of '/dev/vhost-vsock' retained as kvm
Sep 07 15:39:43 ursul qemu-hook[2392]: changed group of '/dev/vhost-net' from vboxusers to kvm
Sep 07 15:39:43 ursul qemu-hook[2392]: group of '/dev/udmabuf' retained as kvm

Between first and second chgrp, something changed back group of /dev/vhost-net to vboxusers, so second chgrp changed it to kvm again

seth · 2024-09-07 12:09:00

seth wrote:

https://wiki.archlinux.org/title/Audit_ … ies_access
nb. the part where you add the rules to /etc/audit/audit.rules as the system is transient and loses its state w/ a reboot.

Also the vboxusers group is back??

Sidebar, you're at least running NM and networkd in parallel. Probalby disable systemd-networkd, make sure you've no other competing services enabled.

abutanaev · 2024-09-07 13:40:07

Yes the group is back I added it again since it's immortal anyway. About NM and networkd in parallel: I suppose it's hardly the reason, I use networkd to connect on boot (when no user logged in yet). Maybe now NM can connect DHCP wire connection without user login?

seth · 2024-09-07 14:18:45

now NM can connect DHCP wire connection without user login?

It could always do that and while I agree that it's unlikely the cause of the group change (you'll have to audit the file) it's a general source of all sorts of trouble.
Especially on "wire", but also on wifi: http://web.archive.org/web/202408221449 … et_on_boot
(wiki is currently down)

It doesn't matter what service you pick, but pick *ONE*

Arch Linux

#1 2024-09-02 11:09:07

/dev/kvm keeps changing group to vboxusers after reboot

#2 2024-09-02 12:09:46

Re: /dev/kvm keeps changing group to vboxusers after reboot

#3 2024-09-06 14:49:16

Re: /dev/kvm keeps changing group to vboxusers after reboot

#4 2024-09-06 15:32:55

Re: /dev/kvm keeps changing group to vboxusers after reboot

#5 2024-09-06 15:36:52

Re: /dev/kvm keeps changing group to vboxusers after reboot

#6 2024-09-06 15:38:21

Re: /dev/kvm keeps changing group to vboxusers after reboot

#7 2024-09-06 20:06:03

Re: /dev/kvm keeps changing group to vboxusers after reboot

#8 2024-09-07 02:03:04

Re: /dev/kvm keeps changing group to vboxusers after reboot

#9 2024-09-07 04:09:39

Re: /dev/kvm keeps changing group to vboxusers after reboot

#10 2024-09-07 05:49:22

Re: /dev/kvm keeps changing group to vboxusers after reboot

#11 2024-09-07 05:58:40

Re: /dev/kvm keeps changing group to vboxusers after reboot

#12 2024-09-07 07:17:23

Re: /dev/kvm keeps changing group to vboxusers after reboot

#13 2024-09-07 07:24:54

Re: /dev/kvm keeps changing group to vboxusers after reboot

#14 2024-09-07 07:30:20

Re: /dev/kvm keeps changing group to vboxusers after reboot

#15 2024-09-07 07:41:17

Re: /dev/kvm keeps changing group to vboxusers after reboot

#16 2024-09-07 07:46:11

Re: /dev/kvm keeps changing group to vboxusers after reboot

#17 2024-09-07 08:30:19

Re: /dev/kvm keeps changing group to vboxusers after reboot

#18 2024-09-07 08:51:54

Re: /dev/kvm keeps changing group to vboxusers after reboot

#19 2024-09-07 12:09:00

Re: /dev/kvm keeps changing group to vboxusers after reboot

#20 2024-09-07 13:40:07

Re: /dev/kvm keeps changing group to vboxusers after reboot

#21 2024-09-07 14:18:45

Re: /dev/kvm keeps changing group to vboxusers after reboot

Board footer