You are not logged in.

#1 2019-04-01 22:25:34

String_Random
Member
Registered: 2018-10-02
Posts: 8

Issue with systemd 241.67-1

Not sure what I should do here, after doing a routine update where I took Linux 5.0.5 and systemd 241.67-1 I started getting a lot of dmesg spam after a reboot.

The core of the issue was the following:

Linux version 5.0.5-arch1-1-ARCH (builduser@heftig-17705) (gcc version 8.2.1 20181127 (GCC)) #1 SMP PREEMPT Wed Mar 27 17:53:10 UTC 2019
[    0.000000] Command line: BOOT_IMAGE=/vmlinuz-linux root=UUID=72e7f630-335c-4f96-8b12-2d4cc0b5b26a rw amd_iommu=on iommu=pt video=vesafb:off,efifb:off kvm.ignore_msrs=1 default_hugepagesz=1G hugepagesz=1G hugepages=36
......
[   10.476709] ccp 0000:0a:00.2: sev command 0x4 timed out, disabling PSP 
[   10.476757] ccp 0000:0a:00.2: SEV: failed to get status. Error: 0x0
[   10.476795] BUG: unable to handle kernel NULL pointer dereference at 0000000000000040
[   10.476839] #PF error: [normal kernel read fault]
[   10.476869] PGD 0 P4D 0 
[   10.476884] Oops: 0000 [#1] PREEMPT SMP NOPTI
[   10.476907] CPU: 14 PID: 924 Comm: qemu-system-x86 Tainted: G        W         5.0.5-arch1-1-ARCH #1
[   10.476950] Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./X399 Taichi, BIOS P3.30 08/14/2018
[   10.477001] RIP: 0010:sev_ioctl+0x63c/0x8b0 [ccp]
[   10.477025] Code: e8 29 f2 ff ff 48 63 d8 4c 89 f7 e8 8e ad 15 f6 4c 89 ef e8 86 ad 15 f6 e9 dc fb ff ff 45 31 ff 45 31 f6 48 8b 05 dc 8f 00 00 <83> 78 40 01 74 15 48 8d 7c 24 14 e8 54 f5 ff ff 48 63 d8 85 db 0f
[   10.477110] RSP: 0018:ffffb70611663e18 EFLAGS: 00010246
[   10.477136] RAX: 0000000000000000 RBX: ffffffffffffffea RCX: 0000000000000000
[   10.477170] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff9b6bfdc07900
[   10.477203] RBP: 00007ffdbcf65820 R08: 00000000000273a0 R09: ffff9b7bfa53bb40
[   10.477236] R10: ffffde3480c2cc00 R11: ffffde3480c2cc88 R12: ffffb70611663e30
[   10.477269] R13: ffff9b7bfa53bb40 R14: 0000000000000000 R15: 0000000000000000
[   10.477302] FS:  00007f34d816c580(0000) GS:ffff9b7bfeb80000(0000) knlGS:0000000000000000
[   10.477340] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   10.477367] CR2: 0000000000000040 CR3: 000000202d750000 CR4: 00000000003406e0
[   10.477400] Call Trace:
[   10.477419]  do_vfs_ioctl+0xa4/0x630
[   10.477438]  ksys_ioctl+0x60/0x90
[   10.477456]  __x64_sys_ioctl+0x16/0x20
[   10.477476]  do_syscall_64+0x5b/0x170
[   10.477497]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[   10.477522] RIP: 0033:0x7f34d994f80b
[   10.477540] Code: 0f 1e fa 48 8b 05 55 b6 0c 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff c3 66 0f 1f 44 00 00 f3 0f 1e fa b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 25 b6 0c 00 f7 d8 64 89 01 48
[   10.477642] RSP: 002b:00007ffdbcf65808 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
[   10.477674] RAX: ffffffffffffffda RBX: 000000000000000d RCX: 00007f34d994f80b
[   10.477705] RDX: 00007ffdbcf65820 RSI: 00000000c0105300 RDI: 000000000000000d
[   10.477735] RBP: 00007ffdbcf65820 R08: 0000000000000000 R09: 00007f34d7c01b70
[   10.477765] R10: 0000000000000000 R11: 0000000000000246 R12: 00007ffdbcf658c0
[   10.477794] R13: 00007ffdbcf65938 R14: 00007ffdbcf65830 R15: 00007f34d7d97d80
[   10.477826] Modules linked in: dm_mod devlink nct6775 hwmon_vid amdgpu nls_iso8859_1 nls_cp437 vfat fat arc4 kvm iwlmvm mac80211 snd_hda_codec_realtek snd_hda_codec_generic chash ledtrig_audio amd_iommu_v2 snd_hda_codec_hdmi crct10dif_pclmul gpu_sched crc32_pclmul ttm ghash_clmulni_intel btusb btrtl drm_kms_helper snd_hda_intel raid1 btbcm wmi_bmof mxm_wmi btintel iwlwifi snd_usb_audio snd_hda_codec bluetooth drm aesni_intel snd_usbmidi_lib snd_hda_core md_mod aes_x86_64 snd_rawmidi snd_hwdep crypto_simd snd_seq_device agpgart cryptd snd_pcm syscopyarea input_leds cfg80211 glue_helper igb ecdh_generic mousedev snd_timer sysfillrect sp5100_tco joydev pcspkr sysimgblt snd fb_sys_fops ccp i2c_piix4 i2c_algo_bit soundcore k10temp rng_core dca rfkill wmi pinctrl_amd gpio_amdpt pcc_cpufreq evdev mac_hid acpi_cpufreq ip_tables x_tables ext4 crc32c_generic crc16 mbcache jbd2 fscrypto hid_generic usbhid hid ses enclosure sd_mod scsi_transport_sas ahci libahci crc32c_intel libata aacraid xhci_pci
[   10.477875]  xhci_hcd scsi_mod vfio_pci irqbypass vfio_virqfd vfio_iommu_type1 vfio
[   10.478240] CR2: 0000000000000040
[   10.478256] ---[ end trace 2bdcbc0dd2326c6e ]---
[   10.478280] RIP: 0010:sev_ioctl+0x63c/0x8b0 [ccp]
[   10.478300] Code: e8 29 f2 ff ff 48 63 d8 4c 89 f7 e8 8e ad 15 f6 4c 89 ef e8 86 ad 15 f6 e9 dc fb ff ff 45 31 ff 45 31 f6 48 8b 05 dc 8f 00 00 <83> 78 40 01 74 15 48 8d 7c 24 14 e8 54 f5 ff ff 48 63 d8 85 db 0f
[   10.478373] RSP: 0018:ffffb70611663e18 EFLAGS: 00010246
[   10.478395] RAX: 0000000000000000 RBX: ffffffffffffffea RCX: 0000000000000000
[   10.478424] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff9b6bfdc07900
[   10.478454] RBP: 00007ffdbcf65820 R08: 00000000000273a0 R09: ffff9b7bfa53bb40
[   10.478483] R10: ffffde3480c2cc00 R11: ffffde3480c2cc88 R12: ffffb70611663e30
[   10.478512] R13: ffff9b7bfa53bb40 R14: 0000000000000000 R15: 0000000000000000
[   10.478541] FS:  00007f34d816c580(0000) GS:ffff9b7bfeb80000(0000) knlGS:0000000000000000
[   10.478574] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   10.478598] CR2: 0000000000000040 CR3: 000000202d750000 CR4: 00000000003406e0
[   10.493334] kvm: Nested Virtualization enabled

I resolved the issue by first rolling back my Linux package to 5.0.4 and then rolled back systemd to 241.7-2

Is this something one should post a bug to Arch, or should this go upstream?

Update: I moved forward to linux 5.0.5 and the error hasn't happened again, so I'm guessing the issue lies with systemd.

Last edited by String_Random (2019-04-01 22:30:02)

Offline

#2 2019-04-01 23:17:53

loqs
Member
Registered: 2014-03-06
Posts: 17,373

Re: Issue with systemd 241.67-1

Does the issue only occur when using qemu?
Can you bisect between systemd 241.7-2 and systemd 241.67-1 to find the causal commit?

Offline

#3 2019-04-02 00:32:06

String_Random
Member
Registered: 2018-10-02
Posts: 8

Re: Issue with systemd 241.67-1

Issue happens at boot before I even can login — or pretty much simultaneously with the login prompt.

Not sure how I can bisect the systemd package, is there a good guide out there?

I ask because I reverted three different packages associated with systemd, and i’m not sure if I can roll one forward at a time to see which one breaks it, or if it’s an all or nothing kind of change that has to happen.

Last edited by String_Random (2019-04-02 00:36:52)

Offline

#4 2019-04-02 00:41:43

loqs
Member
Registered: 2014-03-06
Posts: 17,373

Re: Issue with systemd 241.67-1

The following assumes the base-devel group and git are installed.

$ git clone git://git.archlinux.org/svntogit/packages.git --single-branch --branch "packages/systemd"
$ cp -r packages/trunk systemd
$ rm -rf packages
$ cd systemd
$ makepkg -rsi #this is to confirm systemd as built on your system has the issue

$ cd systemd/src/systemd-stable
$ git bisect start
$ git bisect bad
$ git reset --hard
$ git checkout cbf14c9500d5e6820fd7d96166ca0bf75c6850df
$ git apply ../../0001-Use-Arch-Linux-device-access-groups.patch
$ rm -rf ../build
$ cd ../..
$ makepkg -ersi #this is to confirm systemd 241.7 as built on your system does not have the issue

$ cd systemd/src/systemd-stable
$ git reset --hard
$ git bisect good
$ rm -rf ../build
$ cd ../..
$ makepkg -ersi #test the bisection point

$ cd systemd/src/systemd
$ git reset --hard
$ git bisect $result #substitute good or bad here
$ git apply ../../0001-Use-Arch-Linux-device-access-groups.patch
$ rm -rf ../build
$ cd ../..
$ makepkg -ersi #repeat these ten lines and test the generated package until git has found the bad commit

See also Bisecting_bugs_with_Git
Edit:
fixup instructions add missing cd systemd, missing git reset --hard and correct build directory location.

Last edited by loqs (2019-04-02 10:28:16)

Offline

#5 2019-04-02 03:07:01

String_Random
Member
Registered: 2018-10-02
Posts: 8

Re: Issue with systemd 241.67-1

Hmm, ran into an issue:

==> Verifying source file signatures with gpg...
    systemd-stable git repo ... FAILED (unknown public key B63B21879C3485B0)
    systemd git repo ... FAILED (unknown public key B63B21879C3485B0)

Offline

#6 2019-04-02 10:23:19

loqs
Member
Registered: 2014-03-06
Posts: 17,373

Re: Issue with systemd 241.67-1

GnuPG#Use_a_keyserver

gpg --recv-keys B63B21879C3485B0

Offline

Board footer

Powered by FluxBB