[Solved] Ping spikes and unusable internet when connected to eduroam

caddar · 2024-12-19 09:52:46

Hi all,

after trying to solve the issues for quite a while and even switching from wpa_supplicant to iwd without success, I decided to reach out for help here. I'm using eduroam at university and since some time in the summer (unfortunately I did not note down the exact time) eduroam started to have connectivity issues on my work laptop (but not on my Android phone + tablet or the laptops of other colleagues running different Linux distributions). Using a different wifi that is also broadcasted from the same physical access points, but not a 8021.x-wifi like eduroam, works.

The phenomena is that I get ping spikes every couple of seconds, that last for multiple seconds, and it that time all packets seem to get lost and connectivity (both to public and internal servers) is completely broken.

I've found https://bbs.archlinux.org/viewtopic.php?id=260133 which has similar symptoms, but the suggestions from that thread did not help to improve the situation (disabling IPv6).

If anyone has any idea what the problem could be I would be very grateful. I did not yet involve our eduroam admins because I seem to be the only one with the issue right now. But since it worked and started after some system update, I could imagine that the infrastructure here is using some configs that does not like whatever new versions of libraries sent in my packets o_O

Thank you!

Pinging for example google looks like this (same happens for internal services):

❯ ping google.de
PING google.de (216.58.206.35) 56(84) bytes of data.
64 bytes from lcfraa-aa-in-f3.1e100.net (216.58.206.35): icmp_seq=1 ttl=59 time=9.59 ms
64 bytes from lcfraa-aa-in-f3.1e100.net (216.58.206.35): icmp_seq=2 ttl=59 time=9.65 ms
64 bytes from lcfraa-aa-in-f3.1e100.net (216.58.206.35): icmp_seq=3 ttl=59 time=11.0 ms
64 bytes from lcfraa-aa-in-f3.1e100.net (216.58.206.35): icmp_seq=4 ttl=59 time=11.6 ms
64 bytes from lcfraa-aa-in-f3.1e100.net (216.58.206.35): icmp_seq=5 ttl=59 time=13.7 ms
64 bytes from lcfraa-aa-in-f3.1e100.net (216.58.206.35): icmp_seq=18 ttl=59 time=966 ms          # <---------
64 bytes from lcfraa-aa-in-f3.1e100.net (216.58.206.35): icmp_seq=19 ttl=59 time=9.16 ms
64 bytes from lcfraa-aa-in-f3.1e100.net (216.58.206.35): icmp_seq=20 ttl=59 time=11.3 ms
64 bytes from lcfraa-aa-in-f3.1e100.net (216.58.206.35): icmp_seq=21 ttl=59 time=10.9 ms
64 bytes from lcfraa-aa-in-f3.1e100.net (216.58.206.35): icmp_seq=22 ttl=59 time=10.8 ms
64 bytes from lcfraa-aa-in-f3.1e100.net (216.58.206.35): icmp_seq=23 ttl=59 time=11.4 ms
64 bytes from lcfraa-aa-in-f3.1e100.net (216.58.206.35): icmp_seq=24 ttl=59 time=9.70 ms
64 bytes from lcfraa-aa-in-f3.1e100.net (216.58.206.35): icmp_seq=25 ttl=59 time=9.64 ms
64 bytes from lcfraa-aa-in-f3.1e100.net (216.58.206.35): icmp_seq=26 ttl=59 time=9.70 ms
64 bytes from lcfraa-aa-in-f3.1e100.net (216.58.206.35): icmp_seq=51 ttl=59 time=2619 ms          # <---------
64 bytes from lcfraa-aa-in-f3.1e100.net (216.58.206.35): icmp_seq=52 ttl=59 time=1607 ms          # <---------
64 bytes from lcfraa-aa-in-f3.1e100.net (216.58.206.35): icmp_seq=53 ttl=59 time=593 ms          # <---------
64 bytes from lcfraa-aa-in-f3.1e100.net (216.58.206.35): icmp_seq=54 ttl=59 time=9.79 ms
64 bytes from lcfraa-aa-in-f3.1e100.net (216.58.206.35): icmp_seq=55 ttl=59 time=10.8 ms

Using MTR for both TCP and UDP will show me a huge rate of lost packets during these spikes:

I also tried to move from NetworkManager + wpa_supplicant to NetworkManager + iwd (since I found some forum posts mentioning that iwd might work better with eduroam), but it shows the exact same behavior.

dmesg and syslog are not showing any entry when the spikes happen.

I've tried to include useful information about my system here, but if you are interested in something else let me know and I'll post it.

❯ neofetch
work@laptop
--------------------------------
OS: Arch Linux x86_64
Host: 21HES1YC00 ThinkPad T14 Gen 4
Kernel: 6.12.4-arch1-1
Uptime: 18 hours, 3 mins
Packages: 1671 (pacman)
Shell: zsh 5.9
Resolution: 3840x2160
DE: Hyprland
WM: sway
Theme: Adwaita [GTK2/3]
Icons: Adwaita [GTK2/3]
Terminal: alacritty
CPU: 13th Gen Intel i7-1355U (12) @ 5.000GHz
GPU: Intel Raptor Lake-P [Iris Xe Graphics]
Memory: 8970MiB / 31758MiB

❯  lsmod | grep iwlwifi
iwlwifi               598016  1 iwlmvm
cfg80211             1396736  3 iwlmvm,iwlwifi,mac80211

iw dev wlan0 info
Interface wlan0
	ifindex 4
	wdev 0x2
	addr 74:13:ea:a2:7f:24
	type managed
	wiphy 0
	txpower 20.00 dBm
	multicast TXQ:
		qsz-byt	qsz-pkt	flows	drops	marks	overlmt	hashcol	tx-bytes	tx-packets
		0	0	0	0	0	0	0	0	0

❯ iw dev wlan0 link
Connected to cc:db:93:69:b1:8c (on wlan0)
	SSID: eduroam
	freq: 5660.0
	RX: 6991 bytes (35 packets)
	TX: 3768 bytes (37 packets)
	signal: -51 dBm
	rx bitrate: 400.0 MBit/s VHT-MCS 9 40MHz short GI VHT-NSS 2
	tx bitrate: 108.0 MBit/s VHT-MCS 3 40MHz VHT-NSS 2
	bss flags: short-slot-time
	dtim period: 1
	beacon int: 100

❯ systemctl list-unit-files --state enabled
UNIT FILE                          STATE   PRESET
cups.path                          enabled disabled
bluetooth.service                  enabled disabled
cups.service                       enabled disabled
docker.service                     enabled disabled
getty@.service                     enabled enabled
NetworkManager-dispatcher.service  enabled disabled
NetworkManager-wait-online.service enabled disabled
NetworkManager.service             enabled disabled
reflector.service                  enabled disabled
systemd-timesyncd.service          enabled enabled
tlp.service                        enabled disabled
cups.socket                        enabled disabled
remote-fs.target                   enabled enabled
borgmatic.timer                    enabled disabled

14 unit files listed.

❯ systemctl status iwd
● iwd.service - Wireless service
     Loaded: loaded (/usr/lib/systemd/system/iwd.service; disabled; preset: disabled)
     Active: active (running) since Wed 2024-12-18 16:22:43 CET; 18h ago
 Invocation: 5b8cff2752d648b28a7bccce052f539b
       Docs: man:iwd(8)
             man:iwd.config(5)
             man:iwd.network(5)
             man:iwd.ap(5)
   Main PID: 1256 (iwd)
      Tasks: 1 (limit: 37907)
     Memory: 2.6M (peak: 3.3M)
        CPU: 1.150s
     CGroup: /system.slice/iwd.service
             └─1256 /usr/lib/iwd/iwd

Dec 19 10:34:06 laptop..de iwd[1256]: event: connect-info, ssid: eduroam, bss: cc:db:93:69:b1:8c, signal: -53, load: 2/255
Dec 19 10:34:06 laptop..de iwd[1256]: event: state, old: disconnected, new: connecting
Dec 19 10:34:06 laptop..de iwd[1256]: EAP completed with eapSuccess
Dec 19 10:34:06 laptop..de iwd[1256]: event: state, old: connecting, new: connected
Dec 19 10:34:33 laptop..de iwd[1256]: event: state, old: connected, new: disconnecting
Dec 19 10:34:33 laptop..de iwd[1256]: event: state, old: disconnecting, new: disconnected
Dec 19 10:34:33 laptop..de iwd[1256]: event: connect-info, ssid: eduroam, bss: cc:db:93:69:b1:8c, signal: -52, load: 2/255
Dec 19 10:34:33 laptop..de iwd[1256]: event: state, old: disconnected, new: connecting
Dec 19 10:34:33 laptop..de iwd[1256]: EAP completed with eapSuccess
Dec 19 10:34:33 laptop..de iwd[1256]: event: state, old: connecting, new: connected

Last edited by caddar (2025-12-03 15:32:26)

caddar · 2025-12-03 15:32:09

Hey - for future searchers: I finally managed to find the root cause for this problem. One hint was that when connected to a VPN (via EduVPN) the eduroam connection continued to work, while without the VPN it would show the described problem above. I started to dig into it and finally realized what's wrong on my system:

The reason was a routing conflict / subnet conflict between Docker networks and the Eduroam subnet used by my university. My university uses 172.23.0.0/16 for Eduroam, and Docker starts assigning bridge networks at 172.17.0.0/16. If you have many docker networks, such as in my case due to running many different docker compose student projects, the subnets count upwards and at some point the overlap happens. This also explains why the problem was non-existent after re-installation of Arch and came back just during the student-grading-phase...

After I've cleaned up docker networks and changed the docker default subnet to 10.200.0.0/16 (via /etc/docker/daemon.json) my eduroam works without issues.

What I still don't understand why it worked for the first few seconds and then somehow decided to switch to the other route, but since this change it just works again. At some point I'll ask some colleague who is doing networks to understand this

172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 
172.18.0.0/16 dev br-bfb6586b9046 proto kernel scope link src 172.18.0.1 linkdown 
172.19.0.0/16 dev br-a775551f5568 proto kernel scope link src 172.19.0.1 linkdown 
172.20.0.0/16 dev br-58e892a77e9c proto kernel scope link src 172.20.0.1 linkdown 
172.21.0.0/16 dev br-62352c4ad7d7 proto kernel scope link src 172.21.0.1 linkdown 
172.22.0.0/16 dev br-f0338fa90713 proto kernel scope link src 172.22.0.1 linkdown 
172.23.0.0/16 dev br-0cd30d396cf5 proto kernel scope link src 172.23.0.1 linkdown 
172.24.0.0/16 dev br-c566b93f77da proto kernel scope link src 172.24.0.1 linkdown 
172.25.0.0/16 dev br-f270599d7ebf proto kernel scope link src 172.25.0.1 linkdown 
172.26.0.0/16 dev br-999a9881388a proto kernel scope link src 172.26.0.1 linkdown

For reference, the /etc/docker/daemon.json file:

{
  "default-address-pools": [
    {
      "base": "10.200.0.0/16",
      "size": 24
    }
  ]
}

Last edited by caddar (2025-12-03 15:33:51)

Arch Linux

#1 2024-12-19 09:52:46

[Solved] Ping spikes and unusable internet when connected to eduroam

#2 2025-12-03 15:32:09

Re: [Solved] Ping spikes and unusable internet when connected to eduroam

Board footer