Random ping spikes after upgrading the kernel from 4.18.10 to 4.20.3

Skitter · 2019-01-22 12:00:58

Hi,

I'm having a weird problem with my integrated ethernet card. Since the full system update, ping times stretch by hundreds of milliseconds at random. Here's an example:

PING 8.8.8.8 (8.8.8.8) 1234(1262) bytes of data.
1242 bytes from 8.8.8.8: icmp_seq=1 ttl=124 time=7.30 ms
1242 bytes from 8.8.8.8: icmp_seq=2 ttl=124 time=7.21 ms
1242 bytes from 8.8.8.8: icmp_seq=3 ttl=124 time=398 ms
1242 bytes from 8.8.8.8: icmp_seq=4 ttl=124 time=448 ms
1242 bytes from 8.8.8.8: icmp_seq=5 ttl=124 time=7.26 ms
1242 bytes from 8.8.8.8: icmp_seq=6 ttl=124 time=7.28 ms
1242 bytes from 8.8.8.8: icmp_seq=7 ttl=124 time=7.29 ms
1242 bytes from 8.8.8.8: icmp_seq=8 ttl=124 time=7.16 ms
1242 bytes from 8.8.8.8: icmp_seq=9 ttl=124 time=1022 ms
1242 bytes from 8.8.8.8: icmp_seq=10 ttl=124 time=187 ms
1242 bytes from 8.8.8.8: icmp_seq=11 ttl=124 time=38.3 ms
1242 bytes from 8.8.8.8: icmp_seq=12 ttl=124 time=7.30 ms
1242 bytes from 8.8.8.8: icmp_seq=13 ttl=124 time=7.28 ms

Of course, it doesn't matter which IP address I ping, even my default gateway responds with ~0.200ms on avarage and randomly spikes to 1000ms. My colleague has the exact same hardware and OS and he's also experiencing this problem.
A moment ago I installed the 4.18.10 kernel and headers, rebooted my laptop and tried to ping something. As expected, the ping times came back to normal. I installed the latest kernel again and the problem started occurring, so I'm 100% sure that something has broken between versions 4.18.10 and 4.20.3. I even plugged in an external USB ethernet card and everything was working fine when I used it instead. I think it's a problem with r8169 module.

02:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 0c)
	Subsystem: Hewlett-Packard Company RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
	Flags: bus master, fast devsel, latency 0, IRQ 17
	I/O ports at 3000 [size=256]
	Memory at d0700000 (64-bit, non-prefetchable) [size=4K]
	Memory at d0400000 (64-bit, prefetchable) [size=16K]
	Capabilities: [40] Power Management version 3
	Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
	Capabilities: [70] Express Endpoint, MSI 01
	Capabilities: [b0] MSI-X: Enable+ Count=4 Masked-
	Capabilities: [d0] Vital Product Data
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [140] Virtual Channel
	Capabilities: [160] Device Serial Number 01-00-00-00-68-4c-e0-00
	Capabilities: [170] Latency Tolerance Reporting
	Kernel driver in use: r8169
	Kernel modules: r8169

The card is installed in HP Probook 450g1 and possibly some other models too. [EDIT] If this post should be in the Laptop forum section, please move it. [/EDIT]

Last edited by Skitter (2019-01-22 12:06:14)

pigiron · 2019-01-23 18:47:45

I think it's a problem with r8169 module.

Possible, but not probable. That's a pretty popular chip. In fact, I have two boxes at home with the same chip and neither exhibit that symptom while running the 4.20.3-arch1-1-ARCH kernel.

If you really want to diagnose this, then keep it simple and don't go beyond boxes on the same subnet. So while pinging the gateway was a good idea, pinging the google was not. Run a traceroute 8.8.8.8 command to see what I mean.

One suggestion would be to find another box on the same subnet and have them ping each other. Try to sync the time on both boxes to be as close as possible. Then, capture the symptom by simultaneously running tcpdump -i XXX -w <some-file-name> on each box.

Here comes the hard part. Bring both files over to a box. Fire up a Wireshark job on each output file, filter on "icmp", and examine/compare the ping packets. Look for delays from when a packet was sent to when it arrived; how long it took for a box to "turn around" a ping packet; etc; etc.

This will hopefully narrow the scope of the problem to the box(es) or the network. My total guess is it's the network as I've seen this happen when gateways/routers (and sometimes switches) are under stress.

seth · 2019-01-23 19:11:30

r8169 covers a wider range of chips, so ensure that you actually have the exact same chip.

@Skitter, you could also try using https://aur.archlinux.org/packages/r8168-dkms/ instead to narrow the cause.

Arch Linux

#1 2019-01-22 12:00:58

Random ping spikes after upgrading the kernel from 4.18.10 to 4.20.3

#2 2019-01-23 18:47:45

Re: Random ping spikes after upgrading the kernel from 4.18.10 to 4.20.3

#3 2019-01-23 19:11:30

Re: Random ping spikes after upgrading the kernel from 4.18.10 to 4.20.3

Board footer