Sound issues when installing kernel with pacman.

angrynapkin · 2022-09-23 22:05:09

Ive got a Dell XPS 17 9720 that has a strange sound issue.
With default Arch linux kernel I have no sound. I need to modprobe -r the sound module and reload it for it to work.

Working with the folks at SOF project, there have been some patches applied to the kernel which seems to solve it for some users, but this laptop still seems to suffer.
I have compiled the Arch kernel, the SOF testing kernel, the Vanilla kernel, some with patches, others without, but I was able to narrow it down to one thing that is causing the issue (As far as my understanding goes).

When I compile the kernel (Arches, or Vanilla) manually, and install it manually I get working sound.
When I use the Arch Build System to compile the kernel, I get broken sound, but "only" if I install the kernel with pacman.
If I let the Arch Build System compile the kernel and then copy the kernel from the src dir and install all the modules manually, my sound will work.
Something that happens when installing with pacman -U causes my sound driver to hang and never load right, forcing me to have to unload it, and reload it.

Can anyone help shed some light on what might be going on?

Lone_Wolf · 2022-09-24 14:56:36

In _package() function the modules are installed to $pkgdir (and thus end up in the binary tarball installed with pacman -U ) using

  make INSTALL_MOD_PATH="$pkgdir/usr" INSTALL_MOD_STRIP=1 \
    DEPMOD=/doesnt/exist modules_install  # Suppress depmod

Try building without INSTALL_MOD_STRIP=1 to test if the stripping is what causes the module to fail.

angrynapkin · 2022-09-24 16:34:53

This did seem to resolve the issue, tho the install was substantially larger in size.
What does this option actually do?

Lone_Wolf · 2022-09-25 10:28:46

https://www.kernel.org/doc/html/latest/kbuild/kbuild.html wrote:

INSTALL_MOD_STRIP, if defined, will cause modules to be stripped after they are installed. If INSTALL_MOD_STRIP is ‘1’, then the default option –strip-debug will be used.

Debug symbols tend to take lots of space and make things work slower. It's been default setting for archlinux packages for years and considered safe.
The linux PKGBUILD disables stripping by makepkg through options array and uses INSTALL_MOD_STRIP=1 instead of it.

Knowing debug kernel has no issues and also that sound does work if you remove / reload kernel module it strongly suggests you are facing a race condition .
With that I mean the sound module is started before the sound hardware has fully initialised and thus fails .

Please post full lspci -k and dmesg / journal outputs from a boot with repo linux kernel.
Also the MODULES= line of your /etc/mkinitcpio.conf (or equivalent if you use another program then mkinicpio like dracut or booster ).

dmesg & jpournal will likely be large, see https://wiki.archlinux.org/title/List_o … n_services for methods to upload them somewhere public .

angrynapkin · 2022-09-25 13:51:00

Here are the dmesg and Journal logs from repo kernel.
dmesg: http://0x0.st/oWJE.txt
journal: http://0x0.st/oWJ6.txt
lspci -k: http://0x0.st/oWJ0.txt

all I have in MODULES= is MODULES=()

I have submitted these logs to the SOF group on github too before posting here. Based on my logs and others logs they created some patches that made it into the main kernel, but the issue it still present and only seemingly present in arch. If we can help track down where this bug is coming from that wold be awesome.

Lone_Wolf · 2022-09-26 11:02:21

The hang is clearly visible in the log and comes from systemd-udevd .

Some things that stand out :

[   15.912567] snd_hda_intel 0000:00:1f.3: DSP detected with PCI class/subclass/prog-if info 0x040100
[   15.912943] snd_hda_intel 0000:00:1f.3: SoundWire enabled on CannonLake+ platform, using SOF driver

[   16.582290] sof-audio-pci-intel-tgl 0000:00:1f.3: DSP detected with PCI class/subclass/prog-if info 0x040100
[   16.582314] sof-audio-pci-intel-tgl 0000:00:1f.3: SoundWire enabled on CannonLake+ platform, using SOF driver
[   16.582326] sof-audio-pci-intel-tgl 0000:00:1f.3: enabling device (0000 -> 0002)
[   16.582837] sof-audio-pci-intel-tgl 0000:00:1f.3: DSP detected with PCI class/subclass/prog-if 0x040100

[   17.143640] sof-audio-pci-intel-tgl 0000:00:1f.3: use msi interrupt mode
[   17.162092] sof-audio-pci-intel-tgl 0000:00:1f.3: hda codecs found, mask 4
[   17.163915] sof-audio-pci-intel-tgl 0000:00:1f.3: Firmware info: version 2:2:0-57864
[   17.163920] sof-audio-pci-intel-tgl 0000:00:1f.3: Firmware: ABI 3:22:1 Kernel ABI 3:21:0
[   17.163923] sof-audio-pci-intel-tgl 0000:00:1f.3: FW ABI is more recent than kernel
[   17.163928] sof-audio-pci-intel-tgl 0000:00:1f.3: unknown sof_ext_man header type 3 size 0x30

[   17.274867] sof-audio-pci-intel-tgl 0000:00:1f.3: Firmware info: version 2:2:0-57864
[   17.274873] sof-audio-pci-intel-tgl 0000:00:1f.3: Firmware: ABI 3:22:1 Kernel ABI 3:21:0
[   17.274876] sof-audio-pci-intel-tgl 0000:00:1f.3: FW ABI is more recent than kernel

[   21.238733] sof_sdw sof_sdw: snd_soc_register_card failed -517

[   21.347235] sof_sdw sof_sdw: snd_soc_register_card failed -517
[   21.459624] sof_sdw sof_sdw: snd_soc_register_card failed -517
[   21.465116] sof_sdw sof_sdw: snd_soc_register_card failed -517
[   21.472066] sof_sdw sof_sdw: snd_soc_register_card failed -517

I don't like the order in which kernel modules are loaded (f.e. integrated gpu usually needs to finish initialising before the module for the discrete gpu starts) , but there's not enough data yet to set a specific order.

The "unknown sof_ext_man header type 3" looks important ,

Please reboot with this same kernel, remove and reload the snd module.
post the part of dmesg that shows the related kernel messages (last 50 lines or so will prob be enough).

I have submitted these logs to the SOF group on github too before posting here.

Could you post a link to the bug report where that's investigated ?

angrynapkin · 2022-09-26 13:14:12

Here is a dmesg after a fresh failed sound boot.
Waited a few, then unloaded and loaded the module.
http://0x0.st/oW08.txt

Also here is the link to the on going discussion of this issue at thesofproject: https://github.com/thesofproject/linux/issues/3777
Thank for the help on this, its greatly appreciated.

Lone_Wolf · 2022-09-27 13:08:48

The log shows the "unknown sof_ext_man header " error is also present when sound works, so it's unimportant .

working

[  181.150726] sof-audio-pci-intel-tgl 0000:00:1f.3: FW ABI is more recent than kernel
[  181.161100] sof-audio-pci-intel-tgl 0000:00:1f.3: Topology: ABI 3:22:1 Kernel ABI 3:21:0
[  181.161113] sof-audio-pci-intel-tgl 0000:00:1f.3: warn: topology ABI is more recent than kernel
[  181.161557] sof_sdw sof_sdw: ASoC: Parent card not yet available, widget card binding deferred
[  181.397068] sof_sdw sof_sdw: hda_dsp_hdmi_build_controls: no PCM in topology for HDMI converter 3

not working

[   17.274876] sof-audio-pci-intel-tgl 0000:00:1f.3: FW ABI is more recent than kernel
[   21.238733] sof_sdw sof_sdw: snd_soc_register_card failed -517

No clues what causes the difference, but there is a very interesting commit (mentioned multiple times) in the github BR titled ASoC: Intel: sof_sdw: add support for Dell SKU 0AFF

It appears to be a kernel patch .

The commit does mention RT711_JD2 and your logs show messages about rt711-sdca .
Does you system match the DELL SKU 0AFF ?

If unsure, install dmidecode and check its output for that string.

angrynapkin · 2022-09-28 13:39:57

Lone_Wolf wrote:

Does you system match the DELL SKU 0AFF ?

I do not believe it does.
lspci -v show this for my sound:

0000:00:1f.3 Multimedia audio controller: Intel Corporation Alder Lake PCH-P High Definition Audio Controller (rev 01)
Subsystem: Dell Device 0afe
Flags: bus master, fast devsel, latency 64, IRQ 221, IOMMU group 18
Memory at 618f2d0000 (64-bit, non-prefetchable) [size=16K]
Memory at 618f000000 (64-bit, non-prefetchable) [size=1M]
Capabilities: <access denied>
Kernel driver in use: sof-audio-pci-intel-tgl
Kernel modules: snd_hda_intel, snd_sof_pci_intel_tgl

I'm looking at the Dell Device 0afe. Not sure if this is the right thing to look at but I seen 0afe in the same patched code the oaff was in so Im sort of assuming.

Lone_Wolf · 2022-09-29 07:50:15

I agree that looks like another version, but we need to verify at the source.

Please install https://archlinux.org/packages/extra/x86_64/dmidecode/ and post the output .

angrynapkin · 2022-09-29 12:49:09

Here is the output of dmidecode: http://0x0.st/o4S2.txt

Lone_Wolf · 2022-09-30 09:40:57

No sign of "DELL SKU 0AFF" in the dmidecode output.

I'm leaning towards a new bugreport for your specific hardware in kernel bugtracker but feel we need some addtional testing.

Could you try with https://aur.archlinux.org/packages/linux-mainline ?

It builds from upstream source , doesn't apply patches atm and while it does do some stripping it doesn't seem to strip modules.

angrynapkin · 2022-09-30 21:33:32

No luck on that mainline kernel. Same as all the other kernels that get installed thru pacman.
Not sure if its of help, but the sound works without fail when I unload and load the module, and then on boot each time after, only when rebooting.
Its only on cold boots that the drivers fail.

Last edited by angrynapkin (2022-09-30 23:52:06)

seth · 2022-10-01 07:02:18

What if you give snd_hda_intel and i915 a headstart and move it into "MODULES=(i915 snd_hda_intel)"?
(Don't forget to rebuild the initramfs)

If that works you could also try only "MODULES=(i915)"

angrynapkin · 2022-10-01 12:57:21

Adding those to my mkinitcpio.conf did nothing, sorry. Still No sound at boot.

Lone_Wolf · 2022-10-02 11:12:27

The necessary modules may be at a lower level, please post full lsmod before and after sound works with stock kernel.

angrynapkin · 2022-10-02 14:26:24

Here are with and without sound, lsmod's.
without: http://0x0.st/o4Cw.txt
with: http://0x0.st/o4Cx.txt

seth · 2022-10-02 17:45:04

The only difference seems snd_ctl_led…
You could try to blacklist snd_sof_pci_intel_tgl and explicitly load it w/ systemd later on.

angrynapkin · 2022-10-02 19:49:50

seth wrote:

The only difference seems snd_ctl_led…
You could try to blacklist snd_sof_pci_intel_tgl and explicitly load it w/ systemd later on.

Yeah this was the approach others have taken too, but it seems more of a work around than a solution.
I appreciate your help on this. It seems this one might go unsolved, lol.

Is there a way to change the order in which drivers are loaded and some how force this sound stuff to be loaded last, or is the only way to really do that by blacklisting and using systemd?

seth · 2022-10-02 20:38:14

Everything here is a workaround - the solution would be to fix the kernel module to wait for the HW.

The idea of #14 was to pre-load certain modules to maybe have the HW in place, but that didn't cut it - t least not for the modules we tried.
"MODULES=(i915 snd_sof_intel_hda_common snd_sof_pci snd_soc_acpi_intel_match)" might do better (the modules snd_sof_pci_intel_tgl depends on)

Blacklisting and https://wiki.archlinux.org/title/Kernel_module#systemd loading it effectively delays the load.
Alternatively you could alter the install command and add a "/bin/sleep 3" delay there, but I don't see and advantage in this (on the contrary)

angrynapkin · 2022-10-02 21:42:43

Well I just tried with "MODULES=(i915 snd_sof_intel_hda_common snd_sof_pci snd_soc_acpi_intel_match)" also with no success.
Ill post in the SOF issues about looking into making the module wait until the hardware is available.
I still find it odd it works every time on reboots, like the hardware is already ready. Maybe an issue in the firmware, and also it works on custom kernels not installed with pacman. Strange Indeed.

seth · 2022-10-03 06:06:01

the hardware is already ready

That.

works on custom kernels not installed with pacman

https://bbs.archlinux.org/viewtopic.php … 1#p2059081

Lone_Wolf wrote:

Debug symbols tend to take lots of space and make things work slower. It's been default setting for archlinux packages for years and considered safe.
The linux PKGBUILD disables stripping by makepkg through options array and uses INSTALL_MOD_STRIP=1 instead of it.
Knowing debug kernel has no issues and also that sound does work if you remove / reload kernel module it strongly suggests you are facing a race condition .
With that I mean the sound module is started before the sound hardware has fully initialised and thus fails .

angrynapkin · 2022-10-04 01:41:31

Lone_Wolf wrote:

Debug symbols tend to take lots of space and make things work slower.

Thats crazy, the added extra time to boot must be in the fractions of a fraction of a second, lol

Arch Linux

#1 2022-09-23 22:05:09

Sound issues when installing kernel with pacman.

#2 2022-09-24 14:56:36

Re: Sound issues when installing kernel with pacman.

#3 2022-09-24 16:34:53

Re: Sound issues when installing kernel with pacman.

#4 2022-09-25 10:28:46

Re: Sound issues when installing kernel with pacman.

#5 2022-09-25 13:51:00

Re: Sound issues when installing kernel with pacman.

#6 2022-09-26 11:02:21

Re: Sound issues when installing kernel with pacman.

#7 2022-09-26 13:14:12

Re: Sound issues when installing kernel with pacman.

#8 2022-09-27 13:08:48

Re: Sound issues when installing kernel with pacman.

#9 2022-09-28 13:39:57

Re: Sound issues when installing kernel with pacman.

#10 2022-09-29 07:50:15

Re: Sound issues when installing kernel with pacman.

#11 2022-09-29 12:49:09

Re: Sound issues when installing kernel with pacman.

#12 2022-09-30 09:40:57

Re: Sound issues when installing kernel with pacman.

#13 2022-09-30 21:33:32

Re: Sound issues when installing kernel with pacman.

#14 2022-10-01 07:02:18

Re: Sound issues when installing kernel with pacman.

#15 2022-10-01 12:57:21

Re: Sound issues when installing kernel with pacman.

#16 2022-10-02 11:12:27

Re: Sound issues when installing kernel with pacman.

#17 2022-10-02 14:26:24

Re: Sound issues when installing kernel with pacman.

#18 2022-10-02 17:45:04

Re: Sound issues when installing kernel with pacman.

#19 2022-10-02 19:49:50

Re: Sound issues when installing kernel with pacman.

#20 2022-10-02 20:38:14

Re: Sound issues when installing kernel with pacman.

#21 2022-10-02 21:42:43

Re: Sound issues when installing kernel with pacman.

#22 2022-10-03 06:06:01

Re: Sound issues when installing kernel with pacman.

#23 2022-10-04 01:41:31

Re: Sound issues when installing kernel with pacman.

Board footer