How to bisect the kernel using the linux-git PKGBUILD? (Part 2) [SOLV]

frumble · 2022-08-28 23:34:13

Hello, I need to bisect the kernel between versions 5.15.6 and 5.15.7 because 5.15.7 broke SATA Hot Swap on my machine. Last December, I already had to bisect another 5.15.* version and we had a helpful thread in this forum. Now, I’m once again struggling to set up this monstrous process. Just why is there no good explainer for how to bisect the Linux kernel on Arch? Seems like such a common task.
So, I got the PKGBUILD out of the old thread (originally for 5.15.2 but should be fine for 5.15.7, I guess?), changed the line

source=(
  'git+https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git#tag=v5.15.3'

to 5.15.7 and started makepkg.

But makepkg leads to:

…
  LINK    /kernel/src/linux/tools/bpf/resolve_btfids/libbpf/libbpf.a
make[1]: *** [Makefile:72: bpf/resolve_btfids] Fehler 2 [English: error 2]
make: *** [Makefile:1371: tools/bpf/resolve_btfids] Fehler 2 [English: error 2]

Where can I find a proper Arch Linux config file for kernel 5.15.7? Is there something else I need to do?

Last edited by frumble (2022-08-31 12:14:27)

loqs · 2022-08-28 23:51:04

git clone https://github.com/archlinux/svntogit-packages.git --single-branch --branch "packages/linux" linux # checkout the PKGBUILD for the linux package
cd linux/trunk
git log # list the commits to find 5.15.6
git checkout 531196ada61fd9df3f7be4d7616006f5082046cf  # 5.15.6.arch2-1

Then I suggest making some changes to the PKGBUILD to make bisection easier, change the src to stable as you have done previously change $_srcname to match this change, add a pkgver function so the pkgver variable will be updated to follow the bisection.
Drop building the documentation which can take as long as building the kernel.

diff --git a/trunk/PKGBUILD b/trunk/PKGBUILD
index d56c15b..0447a6e 100644
--- a/trunk/PKGBUILD
+++ b/trunk/PKGBUILD
@@ -10,13 +10,12 @@ arch=(x86_64)
 license=(GPL2)
 makedepends=(
   bc kmod libelf pahole cpio perl tar xz
-  xmlto python-sphinx python-sphinx_rtd_theme graphviz imagemagick
   git
 )
 options=('!strip')
-_srcname=archlinux-linux
+_srcname=linux
 source=(
-  "$_srcname::git+https://github.com/archlinux/linux?signed#tag=$_srctag"
+  "$_srcname::git+https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git#tag=v5.15.6"
   config         # the main kernel config file
 )
 validpgpkeys=(
@@ -58,10 +57,15 @@ prepare() {
   echo "Prepared $pkgbase version $(<version)"
 }
 
+pkgver() {
+  cd $_srcname
+
+  git describe | sed -E 's/^v//;s/([^-]*-g)/r\1/;s/-/./g;s/\.rc/rc/'
+}
+
 build() {
   cd $_srcname
   make all
-  make htmldocs
 }
 
 _package() {
@@ -173,26 +177,7 @@ _package-headers() {
   ln -sr "$builddir" "$pkgdir/usr/src/$pkgbase"
 }
 
-_package-docs() {
-  pkgdesc="Documentation for the $pkgdesc kernel"
-
-  cd $_srcname
-  local builddir="$pkgdir/usr/lib/modules/$(<version)/build"
-
-  echo "Installing documentation..."
-  local src dst
-  while read -rd '' src; do
-    dst="${src#Documentation/}"
-    dst="$builddir/Documentation/${dst#output/}"
-    install -Dm644 "$src" "$dst"
-  done < <(find Documentation -name '.*' -prune -o ! -type d -print0)
-
-  echo "Adding symlink..."
-  mkdir -p "$pkgdir/usr/share/doc"
-  ln -sr "$builddir/Documentation" "$pkgdir/usr/share/doc/$pkgbase"
-}
-
-pkgname=("$pkgbase" "$pkgbase-headers" "$pkgbase-docs")
+pkgname=("$pkgbase" "$pkgbase-headers")
 for _p in "${pkgname[@]}"; do
   eval "package_$_p() {
     $(declare -f "_package${_p#$pkgbase}")

frumble · 2022-08-29 00:37:00

Hm… Thanks a lot for this detailed guide!
But sadly, makepkg once again gives me errors with the new config file:

…
make[3]: *** [Makefile:59: /git/src/linux/tools/objtool/libsubcmd-in.o] Fehler 2
make[2]: *** [Makefile:63: /git/src/linux/tools/objtool/libsubcmd.a] Fehler 2
make[2]: *** Es wird auf noch nicht beendete Prozesse gewartet....
…
make[1]: *** [Makefile:69: objtool] Fehler 2
make: *** [Makefile:1371: tools/objtool] Fehler 2
make: *** Es wird auf noch nicht beendete Prozesse gewartet....
…
make[1]: *** [Makefile:72: bpf/resolve_btfids] Fehler 2
make: *** [Makefile:1371: tools/bpf/resolve_btfids] Fehler 2

Am I doing something wrong or is this version just broken?

loqs · 2022-08-29 01:13:39

You ran into https://github.com/torvalds/linux/commi … 2c1a6fde66
Edit:
In prepare after line 40 add

git cherry-pick -n 52a9dab6d892763b2a8334a568bd4e2c1a6fde66

After you have built the package and tested it do

cd linux
git reset --hard
git bisect $ result
git cherry-pick -n 52a9dab6d892763b2a8334a568bd4e2c1a6fde66
cd ..
makepkg -ef

Here are links to packages I built while testing

https://drive.google.com/file/d/1SllhQi … sp=sharing linux-5.15.6-1-x86_64.pkg.tar.zst
https://drive.google.com/file/d/1gx0az_ … sp=sharing linux-headers-5.15.6-1-x86_64.pkg.tar.zst

https://drive.google.com/file/d/1ZomJeJ … sp=sharing linux-5.15.7-1-x86_64.pkg.tar.zst
https://drive.google.com/file/d/1dvH-Bk … sp=sharing linux-headers-5.15.7-1-x86_64.pkg.tar.zst

https://drive.google.com/file/d/1dXtsHp … sp=sharing linux-5.15.6.r104.g5a9afcd827ca-1-x86_64.pkg.tar.zst
https://drive.google.com/file/d/1dP1XK0 … sp=sharing linux-headers-5.15.6.r104.g5a9afcd827ca-1-x86_64.pkg.tar.zst

Last edited by loqs (2022-08-29 01:57:25)

frumble · 2022-08-29 01:39:21

This patch also does not help. Same error.

loqs · 2022-08-29 01:58:11

See my edit, I also linked to some prebuilt packages I used during testing.

frumble · 2022-08-29 21:13:15

Uff! Thanks!
I was able to compile linux-git-5.15.6-1 on myself now, but continuing with the bisect process, I get the same old

make[1]: *** [Makefile:72: bpf/resolve_btfids] Fehler 2
make: *** [Makefile:1371: tools/bpf/resolve_btfids] Fehler 2

error again, even with manually patched tools/lib/subcmd/subcmd-util.h AND your git cherry-pick -n 52a9dab6d892763b2a8334a568bd4e2c1a6fde66.

I could test your binaries though, linux-5.15.6-1 is good like mine, linux-5.15.7-1 is bad and linux-5.15.6.r104.g5a9afcd827ca-1 is also bad.

loqs · 2022-08-29 21:46:16

git bisect bad
Bisecting: 51 revisions left to test after this (roughly 6 steps)
[275827a7dcafe8e02ccd3f6c489bbdcf8ec532b4] net/tls: Fix authentication failure in CCM mode

https://drive.google.com/file/d/1rQjP1T … sp=sharing linux-5.15.6.r52.g275827a7dcaf-1-x86_64.pkg.tar.zst
https://drive.google.com/file/d/145xkzK … sp=sharing linux-headers-5.15.6.r52.g275827a7dcaf-1-x86_64.pkg.tar.zst

frumble · 2022-08-29 22:17:55

Lol, that’s nice of you! This linux-5.15.6.r52.g275827a7dcaf-1 is also bad. And I don’t need the headers.

Last edited by frumble (2022-08-29 22:23:20)

loqs · 2022-08-29 22:34:46

git bisect bad
Bisecting: 25 revisions left to test after this (roughly 5 steps)
[d17d9e935f729c58c57e5183d706f4ca1b0ea022] platform/x86: thinkpad_acpi: Add support for dual fan control

https://drive.google.com/file/d/1F7B4mS … sp=sharing linux-5.15.6.r26.gd17d9e935f72-1-x86_64.pkg.tar.zst
https://drive.google.com/file/d/1GpGuwT … sp=sharing linux-headers-5.15.6.r26.gd17d9e935f72-1-x86_64.pkg.tar.zst

I add the headers as a matter of routine, just ignore those links.

frumble · 2022-08-29 22:42:28

This is a good one!

loqs · 2022-08-29 23:09:42

git bisect good
Bisecting: 12 revisions left to test after this (roughly 4 steps)
[fc7ffa7f10b9454a86369405d9814bf141b30627] ethernet: hisilicon: hns: hns_dsaf_misc: fix a possible array overflow in hns_dsaf_ge_srst_by_port()

https://drive.google.com/file/d/17yvts8 … sp=sharing linux-5.15.6.r39.gfc7ffa7f10b9-1-x86_64.pkg.tar.zst
https://drive.google.com/file/d/1GENlWW … sp=sharing linux-headers-5.15.6.r39.gfc7ffa7f10b9-1-x86_64.pkg.tar.zst

frumble · 2022-08-29 23:14:48

This one is bad.

loqs · 2022-08-29 23:35:13

git bisect bad
Bisecting: 6 revisions left to test after this (roughly 3 steps)
[59219569202156e271c416d17e75ba1b9c552568] scsi: iscsi: Unblock session then wake up error handler

https://drive.google.com/file/d/1Ef8wtT … sp=sharing linux-5.15.6.r32.g592195692021-1-x86_64.pkg.tar.zst
https://drive.google.com/file/d/1X8T9jd … sp=sharing linux-headers-5.15.6.r32.g592195692021-1-x86_64.pkg.tar.zst

frumble · 2022-08-30 00:14:04

A good one.

loqs · 2022-08-30 00:34:08

git bisect good
Bisecting: 3 revisions left to test after this (roughly 2 steps)
[06c6f8f86ec243b89e52f0c3dc7062bcb9de74df] drm/amd/amdkfd: Fix kernel panic when reset failed and been triggered again

https://drive.google.com/file/d/15-H6q9 … sp=sharing linux-5.15.6.r35.g06c6f8f86ec2-1-x86_64.pkg.tar.zst
https://drive.google.com/file/d/12P-hrf … sp=sharing linux-headers-5.15.6.r35.g06c6f8f86ec2-1-x86_64.pkg.tar.zst
Edit:
Likely causes:
https://git.kernel.org/pub/scm/linux/ke … a186549c63
https://git.kernel.org/pub/scm/linux/ke … 196c80efba

Last edited by loqs (2022-08-30 00:36:38)

frumble · 2022-08-30 00:49:53

This one is good.

loqs · 2022-08-30 02:24:02

git bisect good
Bisecting: 1 revision left to test after this (roughly 1 step)
[ab8efdbda76ca5ac01d8e37929f31aa186549c63] ata: ahci: Add Green Sardine vendor ID as board_ahci_mobile

https://drive.google.com/file/d/1hKU4O- … sp=sharing linux-5.15.6.r37.gab8efdbda76c-1-x86_64.pkg.tar.zst
https://drive.google.com/file/d/14pbkEL … sp=sharing linux-headers-5.15.6.r37.gab8efdbda76c-1-x86_64.pkg.tar.zst

frumble · 2022-08-30 02:32:12

This one is bad.

loqs · 2022-08-30 02:39:53

Is the affected system's SATA controller Green Sardine 1022:7901 which would be listed in `lspci -nnk` similar to below

lspci -nnk
...
09:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 51)
	Subsystem: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901]
	Kernel driver in use: ahci
	Kernel modules: ahci

Edit:

git bisect bad
Bisecting: 0 revisions left to test after this (roughly 0 steps)
[75752ada77e0726327adf68018b9f50ae091baeb] drm/amd/amdgpu: fix potential memleak

https://drive.google.com/file/d/1iBbrgS … sp=sharing linux-5.15.6.r36.g75752ada77e0-1-x86_64.pkg.tar.zst
https://drive.google.com/file/d/1URoAS5 … sp=sharing linux-headers-5.15.6.r36.g75752ada77e0-1-x86_64.pkg.tar.zst
Going to assume this kernel is good as the commit is GPU related. So:

git bisect good
ab8efdbda76ca5ac01d8e37929f31aa186549c63 is the first bad commit
commit ab8efdbda76ca5ac01d8e37929f31aa186549c63
Author: Mario Limonciello <mario.limonciello@amd.com>
Date:   Fri Nov 12 14:15:38 2021 -0600

    ata: ahci: Add Green Sardine vendor ID as board_ahci_mobile
    
    [ Upstream commit 1527f69204fe35f341cb599f1cb01bd02daf4374 ]
    
    AMD requires that the SATA controller be configured for devsleep in order
    for S0i3 entry to work properly.
    
    commit b1a9585cc396 ("ata: ahci: Enable DEVSLP by default on x86 with
    SLP_S0") sets up a kernel policy to enable devsleep on Intel mobile
    platforms that are using s0ix.  Add the PCI ID for the SATA controller in
    Green Sardine platforms to extend this policy by default for AMD based
    systems using s0i3 as well.
    
    Cc: Nehal-bakulchandra Shah <Nehal-bakulchandra.Shah@amd.com>
    BugLink: https://bugzilla.kernel.org/show_bug.cgi?id=214091
    Signed-off-by: Mario Limonciello <mario.limonciello@amd.com>
    Signed-off-by: Damien Le Moal <damien.lemoal@opensource.wdc.com>
    Signed-off-by: Sasha Levin <sashal@kernel.org>

 drivers/ata/ahci.c | 1 +
 1 file changed, 1 insertion(+)

Last edited by loqs (2022-08-30 02:50:48)

frumble · 2022-08-30 02:52:12

Yes! I have this controller 1022:7901 four times in my ASUS ROG Strix X570-E Gaming. Are we done? Shouldn’t there be one final step with the commit number of the bad change? Is it ab8efdbda76ca5ac01d8e37929f31aa186549c63?
Edit: Yes, the last one with the GPU code is good.

Last edited by frumble (2022-08-30 03:00:36)

loqs · 2022-08-30 02:59:49

See my edit:
I would investigate Power_management#SATA_Active_Link_Power_Management.
Edit:
Try adding the kernel boot parameter ahci.mobile_lpm_policy=0

Last edited by loqs (2022-08-30 03:08:21)

frumble · 2022-08-30 13:34:43

Wow, ahci.mobile_lpm_policy=0 as boot parameter solves my problem. But the UDEV rule med_power_with_dipm for SATA Link Management makes it start again. What does ahci.mobile_lpm_policy=0 do, exactly? Deactivating all SATA Link Management?
I should still write to the LKML, I suppose?

loqs · 2022-08-30 21:39:21

frumble wrote:

Wow, ahci.mobile_lpm_policy=0 as boot parameter solves my problem. But the UDEV rule med_power_with_dipm for SATA Link Management makes it start again. What does ahci.mobile_lpm_policy=0 do, exactly? Deactivating all SATA Link Management?

modinfo ahci
....
parm:           mobile_lpm_policy:Default LPM policy for mobile chipsets (int)

I assume 0 maps to disable or max performance.

frumble wrote:

I should still write to the LKML, I suppose?

https://access.redhat.com/documentation … guide/alpm has an important note at the end

Setting ALPM to min_power or medium_power will automatically disable the "Hot Plug" feature.

If that is intended standards tracking behavior then it is not a bug.

frumble · 2022-08-30 22:11:32

But it started nonetheless with this kernel version and "cat /sys/class/scsi_host/host*/link_power_management_policy" reports "max_performance", even before I did set anything new, now too. The output hasn’t changed.

Arch Linux

#1 2022-08-28 23:34:13

How to bisect the kernel using the linux-git PKGBUILD? (Part 2) [SOLV]

#2 2022-08-28 23:51:04

Re: How to bisect the kernel using the linux-git PKGBUILD? (Part 2) [SOLV]

#3 2022-08-29 00:37:00

Re: How to bisect the kernel using the linux-git PKGBUILD? (Part 2) [SOLV]

#4 2022-08-29 01:13:39

Re: How to bisect the kernel using the linux-git PKGBUILD? (Part 2) [SOLV]

#5 2022-08-29 01:39:21

Re: How to bisect the kernel using the linux-git PKGBUILD? (Part 2) [SOLV]

#6 2022-08-29 01:58:11

Re: How to bisect the kernel using the linux-git PKGBUILD? (Part 2) [SOLV]

#7 2022-08-29 21:13:15

Re: How to bisect the kernel using the linux-git PKGBUILD? (Part 2) [SOLV]

#8 2022-08-29 21:46:16

Re: How to bisect the kernel using the linux-git PKGBUILD? (Part 2) [SOLV]

#9 2022-08-29 22:17:55

Re: How to bisect the kernel using the linux-git PKGBUILD? (Part 2) [SOLV]

#10 2022-08-29 22:34:46

Re: How to bisect the kernel using the linux-git PKGBUILD? (Part 2) [SOLV]

#11 2022-08-29 22:42:28

Re: How to bisect the kernel using the linux-git PKGBUILD? (Part 2) [SOLV]

#12 2022-08-29 23:09:42

Re: How to bisect the kernel using the linux-git PKGBUILD? (Part 2) [SOLV]

#13 2022-08-29 23:14:48

Re: How to bisect the kernel using the linux-git PKGBUILD? (Part 2) [SOLV]

#14 2022-08-29 23:35:13

Re: How to bisect the kernel using the linux-git PKGBUILD? (Part 2) [SOLV]

#15 2022-08-30 00:14:04

Re: How to bisect the kernel using the linux-git PKGBUILD? (Part 2) [SOLV]

#16 2022-08-30 00:34:08

Re: How to bisect the kernel using the linux-git PKGBUILD? (Part 2) [SOLV]

#17 2022-08-30 00:49:53

Re: How to bisect the kernel using the linux-git PKGBUILD? (Part 2) [SOLV]

#18 2022-08-30 02:24:02

Re: How to bisect the kernel using the linux-git PKGBUILD? (Part 2) [SOLV]

#19 2022-08-30 02:32:12

Re: How to bisect the kernel using the linux-git PKGBUILD? (Part 2) [SOLV]

#20 2022-08-30 02:39:53

Re: How to bisect the kernel using the linux-git PKGBUILD? (Part 2) [SOLV]

#21 2022-08-30 02:52:12

Re: How to bisect the kernel using the linux-git PKGBUILD? (Part 2) [SOLV]

#22 2022-08-30 02:59:49

Re: How to bisect the kernel using the linux-git PKGBUILD? (Part 2) [SOLV]

#23 2022-08-30 13:34:43

Re: How to bisect the kernel using the linux-git PKGBUILD? (Part 2) [SOLV]

#24 2022-08-30 21:39:21

Re: How to bisect the kernel using the linux-git PKGBUILD? (Part 2) [SOLV]

#25 2022-08-30 22:11:32

Re: How to bisect the kernel using the linux-git PKGBUILD? (Part 2) [SOLV]

Board footer