[SOLVED] Network problems with kernel 6.11.5 but 6.10.10 is working

fractalf · 2024-10-31 00:26:55

Hi,
I've been debugging some network problems I've had on an old laptop I'm installing Arch.

Finally (on day 3) I found that the problem I've had fixed itself when I downgraded the kernel.

The problem:
When installing packages (pacman) or downloading large files the network would start out normal and quick, but would gradually slow down and eventually (after about 30-60 sec) die.

But, this problem was only present on my installed system. When I tried to replicate it on the Arch live boot iso, everything was working.
The iso had kernel version 6.10.10, while the installed system have 6.11.5, so obviously something must have happened some place in between here...

I was wondering if anyone have heard of any issues on the kernel regarding this?
If not, maybe I should do some more digging and post an issue..

Thanks
Alf

Last edited by fractalf (2024-10-31 20:27:09)

cryptearth · 2024-10-31 00:29:24

can you provide more details about the system, please? the current post doesn't help anyone without at least the specific NIC

gromit · 2024-10-31 00:32:32

Also testing this against a mirror is not a really good test environment ... Do you maybe have another machine you could test against with iperf or similar?

Additionally, could you test if the latest mainline release also has this issue?

sudo pacman -U https://pkgbuild.com/\~gromit/linux-bisection-kernels/linux-mainline-6.12rc5-1-x86_64.pkg.tar.zst

fractalf · 2024-10-31 00:33:53

Yea sure, sorry I should have added some more info here..

Network controller: Intel corporation PRO/Wireless 4965 AG or AGN [Kedron] Network Connection (rev 61)
Kernel driver in use: iwl4965

cryptearth · 2024-10-31 00:37:16

problem seems 15 years old: https://bbs.archlinux.org/viewtopic.php?id=96947

fractalf · 2024-10-31 00:42:29

gromit wrote:

Also testing this against a mirror is not a really good test environment ... Do you maybe have another machine you could test against with iperf or similar?
Additionally, could you test if the latest mainline release also has this issue?
sudo pacman -U https://pkgbuild.com/\~gromit/linux-bisection-kernels/linux-mainline-6.12rc5-1-x86_64.pkg.tar.zst

I tried this ^, it downloaded fine, I rebootet, but was still using the 6.10.10 kernel. Sorry, I'm a many years Linux user, but haven't done a lot of kernel fiddling..
How to I enable this kernel? ..and do I need some linux-headers also?

Thanks

..it's late here now, but I will do some more debug tomorrow

fractalf · 2024-10-31 00:44:16

cryptearth wrote:

problem seems 15 years old: https://bbs.archlinux.org/viewtopic.php?id=96947

I think its not the same issue. I _have_ network, it's just that it dies after some time while downloading files.
Also this wasn't an issue in kernel 6.10.10, which is not that old.

seth · 2024-10-31 07:31:47

I tried this ^, it downloaded fine, I rebootet, but was still using the 6.10.10 kernel.

Downloaded or installed w/ "pacman -U"? You also have to add the "linux-mainline" to the bootloader config.

But the only major upcoming iwlegacy change seems a gcc14 warning fix.

For 6.11 https://github.com/torvalds/linux/commi … 801112b878 looks sketchy - "Tested on iwl3945 only."…

gromit · 2024-10-31 11:50:45

@seth nice guess, let's see if it's actually part of the issue!
I have built the following with commit 02b682d ("wifi: iwlegacy: do not skip frames with bad FCS") reverted:

sudo pacman -U https://pkgbuild.com/\~gromit/linux-bisection-kernels/linux-mainline-6.12rc5-1.1-x86_64.pkg.tar.zst

So please first test the build in my first comment and then the above ^

progandy · 2024-10-31 12:26:47

gromit wrote:

Also testing this against a mirror is not a really good test environment ... Do you maybe have another machine you could test against with iperf or similar?

There are some sites offering large files specifically for speed tests. That would probably be a better option than using a mirror to test if no local option is possible.

https://www.speedtestx.de/
https://testfile.org/
https://ash-speed.hetzner.com/ https://fsn1-speed.hetzner.com/ https://nbg1-speed.hetzner.com/ https://hel1-speed.hetzner.com/
http://speedtest.tele2.net/
https://www.cs-n.de/internet/speedtest
https://proof.ovh.net/files/

Last edited by progandy (2024-10-31 12:27:23)

seth · 2024-10-31 12:40:55

You can also Get ABBS for international linode servers (100MB and 1GB)

fractalf · 2024-10-31 15:24:29

gromit wrote:

@seth nice guess, let's see if it's actually part of the issue!
I have built the following with commit 02b682d ("wifi: iwlegacy: do not skip frames with bad FCS") reverted:
sudo pacman -U https://pkgbuild.com/\~gromit/linux-bisection-kernels/linux-mainline-6.12rc5-1.1-x86_64.pkg.tar.zst
So please first test the build in my first comment and then the above ^

Ok, so the prober was deactivated in grub, that's why I cound'nt boot it yesterday. Also it was late and my brain wasnt working

Anyway, I tried the first (v1) rc you linked and that didn't work.

However! The second (v1.1) worked!! Horray \o/

Thanks a lot @gromit , great work. I guess this was some kind of regression that will be merged at some point of time?
..until then I will just use the kernel you build.

Much appresiated!

Alf

gromit · 2024-11-04 17:01:16

Hey Alf! Thanks for testing the images.. Would you be willing to write a bugreport to the upstream linux kernel maintainers about this? It would be good to have this fixed for everyone, because as far as I could tell there was no report about this yet.

If you want I could also assist you in writing the report! A brief introduction on how that is supposed to go can be found here: https://docs.kernel.org/admin-guide/rep … sions.html

fractalf · 2024-11-05 09:09:23

@gromit: I would definitly be willing to do that ofc .. but a couple of questions, just so I can fully understand the flow here..

This commit here: https://github.com/torvalds/linux/commi … 801112b878
This is the one that fixes the problem right?

It looks like it has a tag `v6.12-rc6` which I guess is the next kernel to be released?

..so I guess I don't understand something here, because this to me looks like it's already fixed and on the way into the next kernel release?

Thanks for taking the time, introducing me to this world. I got 15+ years of full stack web dev experience, but not a lot of dev xp on this topic here

Last edited by fractalf (2024-11-05 09:09:58)

V1del · 2024-11-05 10:59:52

No this commit is the one that introduces the problem (... it also has a 6.11rc tag so got introduced in the 6.11 kernel series). Reverting it fixes things for you but potentially regresses something for the author of the commit which you'd then have to figure the correct course of action for as part of the bug report

gromit · 2024-11-05 11:40:38

Yeah you just specify what's going wrong, which commit introduced it and CC the right people (and most importantly the regressions list). You can see a recent bugreport that I have filed myself here and use it as a bit of a template: https://lore.kernel.org/all/3cd82004-c5 … @heusel.eu

So what we have already tried is that:
- the error is present in mainline (which is important because that is what the kernel developers work on and care about)
- the mentioned commit causes the issue and reverting it fixes it

You additionally just need to make a good description of your issue and include repro steps, information about your system etc., but all that is missing will also be asked from you! Also please always reply to all, use text formatting (and not HTML) and don't top post.

These are the people you should potentially CC for your mail (obtained via "/scripts/get_maintainer.pl --no-rolestats -f drivers/net/wireless/intel/iwlegacy/3945.c drivers/net/wireless/intel/iwlegacy/4965-mac.c" and the "Signed-off-by: ..." lines in the commit):

Stanislaw Gruszka <stf_xl@wp.pl>
Kalle Valo <kvalo@kernel.org>
Andrii Batyiev <batyiev@gmail.com>
regressions@lists.linux.dev
linux-wireless@vger.kernel.org
linux-kernel@vger.kernel.org

If you have any more questions just post here!

Last edited by gromit (2024-11-05 11:41:31)

fractalf · 2024-11-05 22:56:00

Danke Gromit

For referance:
https://lore.kernel.org/stable/60f752e8 … tmail.com/

fractalf · 2024-11-12 15:16:16

The commit in question was reverted in a patch today:
https://lore.kernel.org/all/20241112142 … ernel.org/

gromit · 2024-11-12 15:23:47

Nice! Now you just need to wait until it is applied and backported to the stable tree

fractalf · 2024-12-01 22:55:23

@gromit
v6.13-rc1 was released today and I think that should contain the bad commit reverted. Have you by any chanse a build for this that I can try?
https://git.kernel.org/pub/scm/linux/ke … =v6.13-rc1

EDIT:
How can one tell if this:
https://git.kernel.org/pub/scm/linux/ke … 81749b3f82
has been merged into the mainline/6.13?

EDIT2:
It kind of looks merged. When browsing the tree of linus and looking at the log for the file in question, the revert is in there:
https://git.kernel.org/pub/scm/linux/ke … 81749b3f82
Right?

Last edited by fractalf (2024-12-01 23:06:03)

gromit · 2024-12-02 00:45:45

@fractalf you can find a prebuilt version of the latest rc here:

sudo pacman -U https://pkgbuild.com/\~gromit/linux-bisection-kernels/linux-mainline-6.12rc3-1-x86_64.pkg.tar.zst

A nice way to check if your change is in mainline and especially in which stable trees it has already landed is this tool: https://kernel.dance/#11597043d74809daf … 81749b3f82

ReDress · 2024-12-02 14:54:13

fractalf wrote:

cryptearth wrote:
problem seems 15 years old: https://bbs.archlinux.org/viewtopic.php?id=96947
I think its not the same issue. I _have_ network, it's just that it dies after some time while downloading files.
Also this wasn't an issue in kernel 6.10.10, which is not that old.

A bug might keep appearing and disappearing as changes are made to the code up until it finally gets fixed.

However unlikely that might sound.

fractalf · 2024-12-03 10:30:45

gromit wrote:

@fractalf you can find a prebuilt version of the latest rc here:
sudo pacman -U https://pkgbuild.com/\~gromit/linux-bisection-kernels/linux-mainline-6.12rc3-1-x86_64.pkg.tar.zst

Pardon me, but that looks like the 6.12 and not 6.13 kernel. What am I missing here ?

gromit wrote:

A nice way to check if your change is in mainline and especially in which stable trees it has already landed is this tool: https://kernel.dance/#11597043d74809daf … 81749b3f82

Nice, thanks!

loqs · 2024-12-03 11:51:58

sudo pacman -U https://pkgbuild.com/\~gromit/linux-bisection-kernels/linux-mainline-6.13rc1-1-x86_64.pkg.tar.zst

ALso queued for 6.12.2 https://git.kernel.org/pub/scm/linux/ke … 9c11995444

Arch Linux

#1 2024-10-31 00:26:55

[SOLVED] Network problems with kernel 6.11.5 but 6.10.10 is working

#2 2024-10-31 00:29:24

Re: [SOLVED] Network problems with kernel 6.11.5 but 6.10.10 is working

#3 2024-10-31 00:32:32

Re: [SOLVED] Network problems with kernel 6.11.5 but 6.10.10 is working

#4 2024-10-31 00:33:53

Re: [SOLVED] Network problems with kernel 6.11.5 but 6.10.10 is working

#5 2024-10-31 00:37:16

Re: [SOLVED] Network problems with kernel 6.11.5 but 6.10.10 is working

#6 2024-10-31 00:42:29

Re: [SOLVED] Network problems with kernel 6.11.5 but 6.10.10 is working

#7 2024-10-31 00:44:16

Re: [SOLVED] Network problems with kernel 6.11.5 but 6.10.10 is working

#8 2024-10-31 07:31:47

Re: [SOLVED] Network problems with kernel 6.11.5 but 6.10.10 is working

#9 2024-10-31 11:50:45

Re: [SOLVED] Network problems with kernel 6.11.5 but 6.10.10 is working

#10 2024-10-31 12:26:47

Re: [SOLVED] Network problems with kernel 6.11.5 but 6.10.10 is working

#11 2024-10-31 12:40:55

Re: [SOLVED] Network problems with kernel 6.11.5 but 6.10.10 is working

#12 2024-10-31 15:24:29

Re: [SOLVED] Network problems with kernel 6.11.5 but 6.10.10 is working

#13 2024-11-04 17:01:16

Re: [SOLVED] Network problems with kernel 6.11.5 but 6.10.10 is working

#14 2024-11-05 09:09:23

Re: [SOLVED] Network problems with kernel 6.11.5 but 6.10.10 is working

#15 2024-11-05 10:59:52

Re: [SOLVED] Network problems with kernel 6.11.5 but 6.10.10 is working

#16 2024-11-05 11:40:38

Re: [SOLVED] Network problems with kernel 6.11.5 but 6.10.10 is working

#17 2024-11-05 22:56:00

Re: [SOLVED] Network problems with kernel 6.11.5 but 6.10.10 is working

#18 2024-11-12 15:16:16

Re: [SOLVED] Network problems with kernel 6.11.5 but 6.10.10 is working

#19 2024-11-12 15:23:47

Re: [SOLVED] Network problems with kernel 6.11.5 but 6.10.10 is working

#20 2024-12-01 22:55:23

Re: [SOLVED] Network problems with kernel 6.11.5 but 6.10.10 is working

#21 2024-12-02 00:45:45

Re: [SOLVED] Network problems with kernel 6.11.5 but 6.10.10 is working

#22 2024-12-02 14:54:13

Re: [SOLVED] Network problems with kernel 6.11.5 but 6.10.10 is working

#23 2024-12-03 10:30:45

Re: [SOLVED] Network problems with kernel 6.11.5 but 6.10.10 is working

#24 2024-12-03 11:51:58

Re: [SOLVED] Network problems with kernel 6.11.5 but 6.10.10 is working

Board footer