r8169 ethernet driver takes a long time to stabilize network link

austinleroy · 2023-03-24 21:00:16

Hello! This is an odd issue that's been plaguing me ever since I swapped the NIC on my machine to a card that uses the r8169 driver. It regularly takes 30+ seconds after boot (or re-plugging the cable) for the link to stabilize and for my machine to route correctly on the network. This wasn't an issue before - the previous device used the igb driver and was able to get up and running within 1-2 seconds. I'm not well versed enough in networking to figure this out on my own, so I thought I'd post here to see if anyone would be able to help. Thanks in advance!

Here's some information that may or may not be helpful:

General network topography:

NIC is connected to a wire with tagged VLAN traffic (Ids 1, 2, & 3).
VLAN 3 is the default, uses the interface `enp3s0.3`
My install just uses `systemd-networkd` for managing interfaces/networking.
No RX/TX errors logged in my router (which is on the other end of the wire) which leads me to believe this issue is local to my machine.

Device info from `lspci -k`:

03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 1e)
	Subsystem: Realtek Semiconductor Co., Ltd. Device 0123
	Kernel driver in use: r8169
	Kernel modules: r8169

Journal output filtered to interface - `journalctl -b | grep enp3`:

In this log you can see that it takes 46 seconds (15:15:34 - 15:16:20) from the network being configured until an ipv4 address is assigned. This is the full output, too - there aren't any further "enp3" journal entries up til now (currently about 15:50).

Mar 24 15:15:32 austin-arch kernel: r8169 0000:03:00.0 enp3s0: renamed from eth0
Mar 24 15:15:34 austin-arch systemd-networkd[476]: enp3s0: Configuring with /etc/systemd/network/10-enp3s0.network.
Mar 24 15:15:34 austin-arch systemd-networkd[476]: enp3s0: Link UP
Mar 24 15:15:34 austin-arch kernel: r8169 0000:03:00.0 enp3s0: Link is Down
Mar 24 15:15:34 austin-arch systemd-networkd[476]: enp3s0.2: netdev ready
Mar 24 15:15:34 austin-arch systemd-networkd[476]: enp3s0.1: netdev ready
Mar 24 15:15:34 austin-arch systemd-networkd[476]: enp3s0.3: netdev ready
Mar 24 15:15:34 austin-arch systemd-networkd[476]: enp3s0.2: Configuring with /etc/systemd/network/20-enp3s0.2.network.
Mar 24 15:15:34 austin-arch systemd-networkd[476]: enp3s0.1: Configuring with /etc/systemd/network/20-enp3s0.1.network.
Mar 24 15:15:34 austin-arch systemd-networkd[476]: enp3s0.3: Configuring with /etc/systemd/network/20-enp3s0.3.network.
Mar 24 15:15:34 austin-arch systemd-networkd[476]: enp3s0.3: Link UP
Mar 24 15:15:38 austin-arch systemd-networkd[476]: enp3s0: Gained carrier
Mar 24 15:15:38 austin-arch kernel: r8169 0000:03:00.0 enp3s0: Link is Up - 1Gbps/Full - flow control rx/tx
Mar 24 15:15:38 austin-arch kernel: IPv6: ADDRCONF(NETDEV_CHANGE): enp3s0: link becomes ready
Mar 24 15:15:38 austin-arch kernel: IPv6: ADDRCONF(NETDEV_CHANGE): enp3s0.3: link becomes ready
Mar 24 15:15:38 austin-arch systemd-networkd[476]: enp3s0.3: Gained carrier
Mar 24 15:15:39 austin-arch kernel: r8169 0000:03:00.0 enp3s0: Link is Down
Mar 24 15:15:39 austin-arch systemd-networkd[476]: enp3s0: Gained IPv6LL
Mar 24 15:15:39 austin-arch systemd-networkd[476]: enp3s0.3: Gained IPv6LL
Mar 24 15:15:40 austin-arch systemd-networkd[476]: enp3s0: Lost carrier
Mar 24 15:15:40 austin-arch systemd-networkd[476]: enp3s0: DHCPv6 lease lost
Mar 24 15:15:40 austin-arch systemd-networkd[476]: enp3s0.3: Lost carrier
Mar 24 15:15:40 austin-arch systemd-networkd[476]: enp3s0.3: DHCPv6 lease lost
Mar 24 15:15:42 austin-arch kernel: r8169 0000:03:00.0 enp3s0: Link is Up - 1Gbps/Full - flow control rx/tx
Mar 24 15:15:42 austin-arch systemd-networkd[476]: enp3s0: Gained carrier
Mar 24 15:15:42 austin-arch systemd-networkd[476]: enp3s0.3: Gained carrier
Mar 24 15:15:43 austin-arch kernel: r8169 0000:03:00.0 enp3s0: Link is Down
Mar 24 15:15:44 austin-arch systemd-networkd[476]: enp3s0: Lost carrier
Mar 24 15:15:44 austin-arch systemd-networkd[476]: enp3s0: DHCPv6 lease lost
Mar 24 15:15:44 austin-arch systemd-networkd[476]: enp3s0.3: Lost carrier
Mar 24 15:15:44 austin-arch systemd-networkd[476]: enp3s0.3: DHCPv6 lease lost
Mar 24 15:15:46 austin-arch systemd-networkd[476]: enp3s0: Gained carrier
Mar 24 15:15:46 austin-arch kernel: r8169 0000:03:00.0 enp3s0: Link is Up - 1Gbps/Full - flow control rx/tx
Mar 24 15:15:46 austin-arch systemd-networkd[476]: enp3s0.3: Gained carrier
Mar 24 15:15:48 austin-arch kernel: r8169 0000:03:00.0 enp3s0: Link is Down
Mar 24 15:15:48 austin-arch systemd-networkd[476]: enp3s0: Lost carrier
Mar 24 15:15:48 austin-arch systemd-networkd[476]: enp3s0: DHCPv6 lease lost
Mar 24 15:15:48 austin-arch systemd-networkd[476]: enp3s0.3: Lost carrier
Mar 24 15:15:48 austin-arch systemd-networkd[476]: enp3s0.3: DHCPv6 lease lost
Mar 24 15:15:51 austin-arch systemd-networkd[476]: enp3s0: Gained carrier
Mar 24 15:15:51 austin-arch kernel: r8169 0000:03:00.0 enp3s0: Link is Up - 100Mbps/Full - flow control rx/tx
Mar 24 15:15:51 austin-arch systemd-networkd[476]: enp3s0.3: Gained carrier
Mar 24 15:15:52 austin-arch kernel: r8169 0000:03:00.0 enp3s0: Link is Down
Mar 24 15:15:52 austin-arch systemd-networkd[476]: enp3s0: Lost carrier
Mar 24 15:15:52 austin-arch systemd-networkd[476]: enp3s0: DHCPv6 lease lost
Mar 24 15:15:52 austin-arch systemd-networkd[476]: enp3s0.3: Lost carrier
Mar 24 15:15:52 austin-arch systemd-networkd[476]: enp3s0.3: DHCPv6 lease lost
Mar 24 15:15:55 austin-arch systemd-networkd[476]: enp3s0: Gained carrier
Mar 24 15:15:55 austin-arch kernel: r8169 0000:03:00.0 enp3s0: Link is Up - 100Mbps/Full - flow control rx/tx
Mar 24 15:15:55 austin-arch systemd-networkd[476]: enp3s0.3: Gained carrier
Mar 24 15:15:56 austin-arch kernel: r8169 0000:03:00.0 enp3s0: Link is Down
Mar 24 15:15:56 austin-arch systemd-networkd[476]: enp3s0: Lost carrier
Mar 24 15:15:56 austin-arch systemd-networkd[476]: enp3s0: DHCPv6 lease lost
Mar 24 15:15:56 austin-arch systemd-networkd[476]: enp3s0.3: Lost carrier
Mar 24 15:15:56 austin-arch systemd-networkd[476]: enp3s0.3: DHCPv6 lease lost
Mar 24 15:15:59 austin-arch systemd-networkd[476]: enp3s0: Gained carrier
Mar 24 15:15:59 austin-arch kernel: r8169 0000:03:00.0 enp3s0: Link is Up - 100Mbps/Full - flow control rx/tx
Mar 24 15:15:59 austin-arch systemd-networkd[476]: enp3s0.3: Gained carrier
Mar 24 15:16:00 austin-arch kernel: r8169 0000:03:00.0 enp3s0: Link is Down
Mar 24 15:16:00 austin-arch systemd-networkd[476]: enp3s0: Lost carrier
Mar 24 15:16:00 austin-arch systemd-networkd[476]: enp3s0: DHCPv6 lease lost
Mar 24 15:16:00 austin-arch systemd-networkd[476]: enp3s0.3: Lost carrier
Mar 24 15:16:00 austin-arch systemd-networkd[476]: enp3s0.3: DHCPv6 lease lost
Mar 24 15:16:03 austin-arch systemd-networkd[476]: enp3s0: Gained carrier
Mar 24 15:16:03 austin-arch kernel: r8169 0000:03:00.0 enp3s0: Link is Up - 1Gbps/Full - flow control rx/tx
Mar 24 15:16:03 austin-arch systemd-networkd[476]: enp3s0.3: Gained carrier
Mar 24 15:16:04 austin-arch kernel: r8169 0000:03:00.0 enp3s0: Link is Down
Mar 24 15:16:05 austin-arch systemd-networkd[476]: enp3s0: Lost carrier
Mar 24 15:16:05 austin-arch systemd-networkd[476]: enp3s0: DHCPv6 lease lost
Mar 24 15:16:05 austin-arch systemd-networkd[476]: enp3s0.3: Lost carrier
Mar 24 15:16:05 austin-arch systemd-networkd[476]: enp3s0.3: DHCPv6 lease lost
Mar 24 15:16:07 austin-arch systemd-networkd[476]: enp3s0: Gained carrier
Mar 24 15:16:07 austin-arch kernel: r8169 0000:03:00.0 enp3s0: Link is Up - 1Gbps/Full - flow control rx/tx
Mar 24 15:16:07 austin-arch systemd-networkd[476]: enp3s0.3: Gained carrier
Mar 24 15:16:08 austin-arch kernel: r8169 0000:03:00.0 enp3s0: Link is Down
Mar 24 15:16:09 austin-arch systemd-networkd[476]: enp3s0: Lost carrier
Mar 24 15:16:09 austin-arch systemd-networkd[476]: enp3s0: DHCPv6 lease lost
Mar 24 15:16:09 austin-arch systemd-networkd[476]: enp3s0.3: Lost carrier
Mar 24 15:16:09 austin-arch systemd-networkd[476]: enp3s0.3: DHCPv6 lease lost
Mar 24 15:16:12 austin-arch systemd-networkd[476]: enp3s0: Gained carrier
Mar 24 15:16:12 austin-arch kernel: r8169 0000:03:00.0 enp3s0: Link is Up - 1Gbps/Full - flow control rx/tx
Mar 24 15:16:12 austin-arch systemd-networkd[476]: enp3s0.3: Gained carrier
Mar 24 15:16:13 austin-arch kernel: r8169 0000:03:00.0 enp3s0: Link is Down
Mar 24 15:16:14 austin-arch systemd-networkd[476]: enp3s0: Lost carrier
Mar 24 15:16:14 austin-arch systemd-networkd[476]: enp3s0: DHCPv6 lease lost
Mar 24 15:16:14 austin-arch systemd-networkd[476]: enp3s0.3: Lost carrier
Mar 24 15:16:14 austin-arch systemd-networkd[476]: enp3s0.3: DHCPv6 lease lost
Mar 24 15:16:16 austin-arch kernel: r8169 0000:03:00.0 enp3s0: Link is Up - 100Mbps/Full - flow control rx/tx
Mar 24 15:16:16 austin-arch systemd-networkd[476]: enp3s0: Gained carrier
Mar 24 15:16:16 austin-arch systemd-networkd[476]: enp3s0.3: Gained carrier
Mar 24 15:16:17 austin-arch kernel: r8169 0000:03:00.0 enp3s0: Link is Down
Mar 24 15:16:17 austin-arch systemd-networkd[476]: enp3s0: Lost carrier
Mar 24 15:16:17 austin-arch systemd-networkd[476]: enp3s0: DHCPv6 lease lost
Mar 24 15:16:17 austin-arch systemd-networkd[476]: enp3s0.3: Lost carrier
Mar 24 15:16:17 austin-arch systemd-networkd[476]: enp3s0.3: DHCPv6 lease lost
Mar 24 15:16:20 austin-arch systemd-networkd[476]: enp3s0: Gained carrier
Mar 24 15:16:20 austin-arch kernel: r8169 0000:03:00.0 enp3s0: Link is Up - 100Mbps/Full - flow control rx/tx
Mar 24 15:16:20 austin-arch systemd-networkd[476]: enp3s0.3: Gained carrier
Mar 24 15:16:20 austin-arch systemd-networkd[476]: enp3s0.3: DHCPv4 address 192.168.3.189/24, gateway 192.168.3.1 acquired from 192.168.3.1

Output of `ifconfig`:

enp3s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::ea4e:6ff:fea5:1c60  prefixlen 64  scopeid 0x20<link>
        ether e8:4e:06:a5:1c:60  txqueuelen 1000  (Ethernet)
        RX packets 22112  bytes 19528914 (18.6 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 14792  bytes 2849183 (2.7 MiB)
        TX errors 5  dropped 0 overruns 0  carrier 0  collisions 0

enp3s0.3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.3.189  netmask 255.255.255.0  broadcast 192.168.3.255
        inet6 fd00:b1a:1c35:0:ea4e:6ff:fea5:1c60  prefixlen 64  scopeid 0x0<global>
        inet6 fe80::ea4e:6ff:fea5:1c60  prefixlen 64  scopeid 0x20<link>
        ether e8:4e:06:a5:1c:60  txqueuelen 1000  (Ethernet)
        RX packets 19556  bytes 19063321 (18.1 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 14753  bytes 2845977 (2.7 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

seth · 2023-03-24 22:15:16

Journal output filtered to interface - `journalctl -b | grep enp3`:

Please post your complete system journal for the boot:

sudo journalctl -b | curl -F 'file=@-' 0x0.st

30s is suspiciously close to the 25s dbus timeout and the link goes frequently up and down which might suggest concurrent services trying to configure it - or if this is a kernel module issue, that got stashed by the filter.

austinleroy · 2023-03-24 23:06:01

Here's the link to the output: http://0x0.st/HoB_.txt

Thanks for taking a look!

seth · 2023-03-24 23:20:56

The link goes up & down w/ 100MB and 1GB and no indication as to why what means you should look at the cable, remember that the plugs are part of the cable and that they on both ends and then check the cable again. Sometimes the latch protection collides w/ the case and impedes a complete insertion, sometimes a cat is chewing on the cable, sometimes it's interference and sometimes it's just broken.

If you want to insist on the delusion that it's not the cable, try a different kernel (lts) and software stack (dhcpcd, preferably w/o the VLANs) or live distro on the NIC.
But that's a waste of time, because rule #1:
It's always the cable.

SysGhost · 2025-03-17 20:49:11

I was recently beating my head at this problem with a few HP EliteBook laptops using this Realtek interface.
I was sure it wasn't the cables because the cables work flawlessly on other network equipment. I even tested the cables I knew were good for a few days with simulated traffic. There was no packet loss whatsoever. That made me certain it wasn't the cables that was the problem.

I then went on to check with different kernel versions, dkms package, trying other modules left and right. ... to no avail.
The interface kept going down and up on a random basis. It was never able to keep a stable connection for more than a minute.
After beating my head at it for a while, about to give up and just about to get myself a bunch of USB 3 Ethernet adapters I thought to myself:

-"Fine! Let's make one more test with a different kind of cable."

I then grabbed the best CAT 6 cable I had lying around, as well as the best quality CAT 6 RJ45 plugs I had—all shielded.
I grabbed my tools and made one cable to the best abilities I could muster.

Plugged the expensive fancy cable in just to prove to myself that "it's not the cable". ...it works flawlessly...

WHAT... NOO? This... no... this just can't be... WHY?
The other cables that fail here... work flawlessly everywhere else... Why is that?

Is it that...Realtek ...is more finicky about cable quality ... ?

... well... now that I think about it... it's the likely explanation. Realtek might simply require much better, more expensive cables than everyone else.
I'm not throwing the supposedly "bad" cables out., They works fine everywhere else. Just not that well with realtek.

Writing this just to tell others who google this up. IT IS THE CABLE. Stop fooling yourself.

----

TLDR:

IT IS THE CABLE.
Realtek is more demanding than others. Go expensive! Make the cable yourself with the best RJ45 connectors you got.

seth · 2025-03-17 20:59:05

Bad plug fit?
The most fucked up cable thing I encountered was the latch protection colliding with the case, resulting in a loose connection (but no way to insert it "more" so probably just normal…)
Takeaway: if you can wiggle an rj45 plug, it's not in.

SysGhost · 2025-03-18 09:49:25

seth wrote:

Bad plug fit?
The most fucked up cable thing I encountered was the latch protection colliding with the case, resulting in a loose connection (but no way to insert it "more" so probably just normal…)
Takeaway: if you can wiggle an rj45 plug, it's not in.

It was one of the first thing I tried. And yeah, these laptops have a spring-hinged RJ45 port so it's not that far off.

But then I also happen to have a stationary computer with a Realtek r8169-based interface with the same problem. Needs a much higher quality cable to stay stable.

To be honest I probably could reterminate these "bad" cables with my set of good RJ45 connectors and I bet they'll be fine.
The difference between the "bad" cables and the good ones I have is that the bad ones are premanufactured. The good ones I've terminated myself.

Arch Linux

#1 2023-03-24 21:00:16

r8169 ethernet driver takes a long time to stabilize network link

#2 2023-03-24 22:15:16

Re: r8169 ethernet driver takes a long time to stabilize network link

#3 2023-03-24 23:06:01

Re: r8169 ethernet driver takes a long time to stabilize network link

#4 2023-03-24 23:20:56

Re: r8169 ethernet driver takes a long time to stabilize network link

#5 2025-03-17 20:49:11

Re: r8169 ethernet driver takes a long time to stabilize network link

#6 2025-03-17 20:59:05

Re: r8169 ethernet driver takes a long time to stabilize network link

#7 2025-03-18 09:49:25

Re: r8169 ethernet driver takes a long time to stabilize network link

Board footer