[Solved] Server network speeds degradation with kernel 6.1 and higher.

Alexey104 · 2022-12-29 13:12:06

I rent a remote VPS which I use as a personal VPN. It runs ArchLinux, and there are only few services running on this machine, which are ShadowSocks, OpenVPN and WireGuard. This is one CPU core and one Gig of RAM KVM machine - not very powerful, but it is more than enough for my personal needs. I have always had very good speeds with this machine, but latterly I noticed that when uploading a file via SFTP to the server I get at most 1.5-1.7 MB/s, which is very slow (usually I get 45-50 MB/s via SFTP). Speedtests running on the server show 5-6 times slower speeds than usual, and, when sorting mirrors with reflector on the server, there are a lot of timeouts, and most of the mirrors are much slower than they used to be. I've found out that the problem occurs if kernel 6.1 or 6.1.1 is used on the server. Downgrading the kernel to 6.0.12 or lower, or switching to LTS-kernel solves the problem making speeds fast again. My home machine also runs Arch with the latest stable kernel (6.1.1), and network speeds are totally fine with it, but using the same kernel on the server causes huge speed drop.

I wonder if anyone else uses Arch on servers and have same problems with latest kernels.

Last edited by Alexey104 (2023-01-21 21:23:31)

teckk · 2022-12-29 22:25:39

After you change kernels, are you rebooting? Using the same kernel and the modules that go with it?

Alexey104 · 2022-12-29 23:01:19

Yes, I always reboot after kernel updates. Just to illustrate the difference, below are the results of iperf3 tests made with 6.0.12 vs 6.1.1 kernels running on the server:

// 6.0.12

[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  38.5 MBytes   323 Mbits/sec
[  5]   1.00-2.00   sec  61.3 MBytes   514 Mbits/sec
[  5]   2.00-3.00   sec  60.9 MBytes   511 Mbits/sec
[  5]   3.00-4.00   sec  60.8 MBytes   510 Mbits/sec
[  5]   4.00-5.00   sec  61.2 MBytes   514 Mbits/sec
[  5]   5.00-6.00   sec  60.9 MBytes   511 Mbits/sec
[  5]   6.00-7.00   sec  61.2 MBytes   513 Mbits/sec
[  5]   7.00-8.00   sec  61.3 MBytes   514 Mbits/sec
[  5]   8.00-9.00   sec  61.5 MBytes   516 Mbits/sec
[  5]   9.00-10.00  sec  60.3 MBytes   506 Mbits/sec
[  5]  10.00-10.05  sec  2.86 MBytes   509 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-10.05  sec   591 MBytes   493 Mbits/sec                  receiver

//  6.1.1

[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  2.28 MBytes  19.1 Mbits/sec
[  5]   1.00-2.00   sec  1.77 MBytes  14.9 Mbits/sec
[  5]   2.00-3.00   sec  1.28 MBytes  10.7 Mbits/sec
[  5]   3.00-4.00   sec  1.20 MBytes  10.1 Mbits/sec
[  5]   4.00-5.00   sec  1.26 MBytes  10.6 Mbits/sec
[  5]   5.00-6.00   sec  1.15 MBytes  9.65 Mbits/sec
[  5]   6.00-7.00   sec  1.34 MBytes  11.3 Mbits/sec
[  5]   7.00-8.00   sec  1.72 MBytes  14.4 Mbits/sec
[  5]   8.00-9.00   sec  1.61 MBytes  13.5 Mbits/sec
[  5]   9.00-10.00  sec  1.15 MBytes  9.68 Mbits/sec
[  5]  10.00-10.05  sec  99.0 KBytes  17.1 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-10.05  sec  14.9 MBytes  12.4 Mbits/sec                  receiver

Not sure, maybe this is something specific to KVM hypervisor on the hosting side as I don't have problems on my home Arch machine with the latest stable kernel, but just switching the kernel on the server makes such a huge difference.

seth · 2022-12-30 08:06:01

I'd suspect that the specific NIC and the kernel module used for it is more relevant.

Alexey104 · 2023-01-21 21:24:57

The problem is reproducible with kernels 6.1-6.1.6, but everything is fine again with 6.1.7.

Arch Linux

#1 2022-12-29 13:12:06

[Solved] Server network speeds degradation with kernel 6.1 and higher.

#2 2022-12-29 22:25:39

Re: [Solved] Server network speeds degradation with kernel 6.1 and higher.

#3 2022-12-29 23:01:19

Re: [Solved] Server network speeds degradation with kernel 6.1 and higher.

#4 2022-12-30 08:06:01

Re: [Solved] Server network speeds degradation with kernel 6.1 and higher.

#5 2023-01-21 21:24:57

Re: [Solved] Server network speeds degradation with kernel 6.1 and higher.

Board footer