DHCP stopped working

andreas.karlsson.se · 2016-10-20 11:07:19

After updating kernel to 4.8.2-1 systemd unit dhcpcd@enp1s0f0 failed to start during boot. There were also other issues during boot and barley getting to login-screen my system was unusable with one processor stuck at 100%. I booted from usb-stick and did a full downgrade to earlier date (16 oct) getting me back to kernel 4.7.6-1.

However, dhcp for my ethernet still fails. I've tried this at two different locations (work and home) and other computers get a lease on the same cable.

Another strange thing is that the same thing now happens when i boot from usb-stick. I.e. dhcpcd fails for the device during boot, irrespective if the cable is plugged in or not.

Could the kernel upgrade and/or following malfunction somehow mess at such a low level that the problem persist when booting to different installations? Or is it a hardware failure?

Full dmesg

sudo dhcpcd --debug -1 enp1s0f0 gives (also says "timeout" on stderr at exit):

dhcpcd-6.11.3 starting
enp1s0f0: executing `/usr/lib/dhcpcd/dhcpcd-run-hooks' PREINIT
enp1s0f0: executing `/usr/lib/dhcpcd/dhcpcd-run-hooks' CARRIER
DUID 00:01:00:01:1f:86:92:de:10:dd:b1:c5:16:b5
enp1s0f0: IAID b1:c5:16:b5
enp1s0f0: delaying IPv6 router solicitation for 0.4 seconds
enp1s0f0: delaying IPv4 for 0.5 seconds
enp1s0f0: soliciting an IPv6 router
enp1s0f0: sending Router Solicitation
enp1s0f0: soliciting a DHCP lease
enp1s0f0: sending DISCOVER (xid 0xc0616d94), next in 4.4 seconds
enp1s0f0: xid 0xe4394708 is for hwaddr 00:02:9b:96:2e:93:00:00:00:00:00:00:00:00:00:00
enp1s0f0: sending Router Solicitation
enp1s0f0: sending DISCOVER (xid 0xc0616d94), next in 8.8 seconds
enp1s0f0: sending Router Solicitation
enp1s0f0: xid 0xe4394708 is for hwaddr 00:02:9b:96:2e:93:00:00:00:00:00:00:00:00:00:00
enp1s0f0: sending Router Solicitation
enp1s0f0: no IPv6 Routers available
enp1s0f0: sending DISCOVER (xid 0xc0616d94), next in 16.3 seconds
enp1s0f0: xid 0xe4394708 is for hwaddr 00:02:9b:96:2e:93:00:00:00:00:00:00:00:00:00:00
enp1s0f0: xid 0xe4394708 is for hwaddr 00:02:9b:96:2e:93:00:00:00:00:00:00:00:00:00:00
enp1s0f0: sending DISCOVER (xid 0xc0616d94), next in 32.1 seconds
dhcpcd exited

I've also tried running dhcpcd with /dev/null as config, but not luck. Here is my /etc/dhcpcd.conf

I'm on a macbook pro, and I've tried resetting NVRAM and SMC.

What else can I try? Any insights welcomed!

Last edited by andreas.karlsson.se (2016-10-20 11:10:14)

seth · 2016-10-20 15:20:10

rebooted the router?

andreas.karlsson.se · 2016-10-20 16:50:21

Yes, I've tried that, as well as not using my own router. Have also tried it on another network, same symptoms. Other devices get IP:s on my LAN so that shouldn't be the problem.

seth · 2016-10-20 17:01:58

lease hikkup - and it's the common cause ;-)

If you try -4 (ie. skip ipv6)?

Any other services operating on the device? (netctl, nm, systemd-networkd)

/usr/lib/dhcpcd/dhcpcd-run-hooks is stock of the dhcpcd package?

andreas.karlsson.se · 2016-10-20 19:21:36

As far as I can understand no other service should be in effect.
systemd-network, netctl and systemd-networkd are all inactive

What is nm? NetworkManager? I stopped it and tried dhcpcd with IPv4 only:

➜  ~ sudo dhcpcd --debug -1 -4 -t50 enp1s0f0
dhcpcd-6.11.3 starting
enp1s0f0: executing `/usr/lib/dhcpcd/dhcpcd-run-hooks' PREINIT
enp1s0f0: executing `/usr/lib/dhcpcd/dhcpcd-run-hooks' CARRIER
DUID 00:01:00:01:1f:86:92:de:10:dd:b1:c5:16:b5
enp1s0f0: IAID b1:c5:16:b5
enp1s0f0: delaying IPv4 for 0.3 seconds
enp1s0f0: soliciting a DHCP lease
enp1s0f0: sending DISCOVER (xid 0x86d0f04d), next in 4.3 seconds
enp1s0f0: xid 0xe4394708 is for hwaddr 00:02:9b:96:2e:93:00:00:00:00:00:00:00:00:00:00
enp1s0f0: sending DISCOVER (xid 0x86d0f04d), next in 8.6 seconds
enp1s0f0: xid 0xe4394708 is for hwaddr 00:02:9b:96:2e:93:00:00:00:00:00:00:00:00:00:00
enp1s0f0: sending DISCOVER (xid 0x86d0f04d), next in 16.1 seconds
enp1s0f0: xid 0xe4394708 is for hwaddr 00:02:9b:96:2e:93:00:00:00:00:00:00:00:00:00:00
enp1s0f0: xid 0xe4394708 is for hwaddr 00:02:9b:96:2e:93:00:00:00:00:00:00:00:00:00:00
enp1s0f0: wrong xid 0x7b4c2b7a (expecting 0x86d0f04d) from 178.78.215.129
enp1s0f0: xid 0xe4394708 is for hwaddr 00:02:9b:96:2e:93:00:00:00:00:00:00:00:00:00:00
enp1s0f0: sending DISCOVER (xid 0x86d0f04d), next in 32.0 seconds
enp1s0f0: xid 0xe4394708 is for hwaddr 00:02:9b:96:2e:93:00:00:00:00:00:00:00:00:00:00
enp1s0f0: xid 0xe4394708 is for hwaddr 00:02:9b:96:2e:93:00:00:00:00:00:00:00:00:00:00
enp1s0f0: xid 0xe4394708 is for hwaddr 00:02:9b:96:2e:93:00:00:00:00:00:00:00:00:00:00
timed out
dhcpcd exited

...to no avail.

For a while I also got my hopes up by this section on load order of kernel modules for old Broadcom NICs https://wiki.archlinux.org/index.php/Ne … m_BCM57780 but that didn't change anything either.

andreas.karlsson.se · 2016-10-20 19:32:00

Isn't it strange that I now, since kernel problems and downgrade, also have these problems when booting from install usb? Ethernet used to work from there..

What other than hw failure can be the reason for that??

seth · 2016-10-20 19:59:00

Did you reboot after stopping NM?
Did you btw. also ensure dhcpcd@enp1s0f0 to be stopped before issuing the test call?

Yes, sudden failure on other systems would point HW issues, but there're several reports of *random* dhcp failure.
Is there still an OS installation - does the device work there?

andreas.karlsson.se · 2016-10-20 21:28:30

No reboot after stopping NM, should I have? I'm running gnome and i could see that after stopping NM there weren't any more "trying to connect" icons in task bar etc. even though I plugged/unplugged.

Yes, dhcpcd and dhcpcd@enp1s0f0 were defiantly stopped when I performed manual test. I also looked for any process with dhcp in their name..

Is there any non-volatile memory in these types of devices? Can there be some state changed on the card? MAC adress and what not?

Also, my ISP uses static IPs though it's configured through dhcp for convenience. I checked what IP and gateway my router had, and set them manually on my computer (through gnome network settings) and plugged the cable straight to my computer. That didn't work either, couldn't ping 8.8.8.8 or get any other reaction..

seth · 2016-10-20 22:21:04

No reboot after stopping NM, should I have?

To clear the stage, yes. (But you'll have to also disable it, oc. - otherwise it will just restart)

MAC's are usually faked on the OS layer - I doubt some ethernet chip has non volatile ram (except for an unintended fuse...)

Can you ping the router on the static lease? Does the enp1s0f0 MAC show up there as connected device?
You should also "sudo tcpdump -i enp1s0f0" and check whether there's "unexpected" traffic.

andreas.karlsson.se · 2016-10-21 10:47:01

So, after some well needed sleep I'm back at work. And now at the office, dhcp works again! I must admit this now looks as a gnarly case of different problems causing similar symptoms:

I was at work when I made the malfunctioning kernel upgrade. I could boot my computer, with lots of errors, but dhcpcd was the only service failing. When I got back home I did the downgrade and now it was booting properly but dhcpcd was still failing. Since other (linux) devices didn't have a problem with dhcp in my home network I drew the conclusion that the failing dhcp was still somehow attributed to the kernel upgrade/downgrade.

As soon as I get home I'll try again. Either the problem has spuriously disappeared altogether or I'll continue investigating it as a config/compatibility problem with my home network.

Arch Linux

#1 2016-10-20 11:07:19

DHCP stopped working

#2 2016-10-20 15:20:10

Re: DHCP stopped working

#3 2016-10-20 16:50:21

Re: DHCP stopped working

#4 2016-10-20 17:01:58

Re: DHCP stopped working

#5 2016-10-20 19:21:36

Re: DHCP stopped working

#6 2016-10-20 19:32:00

Re: DHCP stopped working

#7 2016-10-20 19:59:00

Re: DHCP stopped working

#8 2016-10-20 21:28:30

Re: DHCP stopped working

#9 2016-10-20 22:21:04

Re: DHCP stopped working

#10 2016-10-21 10:47:01

Re: DHCP stopped working

Board footer