Arch Linux

kszonek

Member

Registered: 2011-10-12

Posts: 13

Systemd-networkd doesn't connect to DHCP

I have Linode VPS with Arch on it. It had few months uptime, but decided to power down and migrate to cheaper plan. After migration my system did not even try to get IP from DHCP. I have no idea if it has something to do with migration.

$ systemctl status systemd-networkd.service

● systemd-networkd.service - Network Service
   Loaded: loaded (/usr/lib/systemd/system/systemd-networkd.service; enabled)
   Active: active (running) since Sun 2014-07-06 23:26:15 UTC; 16min ago
     Docs: man:systemd-networkd.service(8)
Main PID: 1649 (systemd-network)
   Status: "Processing requests..."
   CGroup: /system.slice/systemd-networkd.service
           └─1649 /usr/lib/systemd/systemd-networkd

I have created second VPS with stock Arch setup, same command:

● systemd-networkd.service - Network Service
   Loaded: loaded (/usr/lib/systemd/system/systemd-networkd.service; enabled)
   Active: active (running) since Sun 2014-07-06 21:39:18 UTC; 2h 4min ago
     Docs: man:systemd-networkd.service(8)
Main PID: 1424 (systemd-network)
   Status: "Processing requests..."
   CGroup: /system.slice/systemd-networkd.service
           └─1424 /usr/lib/systemd/systemd-networkd

Jul 06 21:39:18 localhost systemd[1]: Started Network Service.
Jul 06 21:39:18 localhost systemd-networkd[1424]: eth0: link is up
Jul 06 21:39:18 localhost systemd-networkd[1424]: eth0: carrier on
Jul 06 21:39:18 localhost systemd-networkd[1424]: eth0: DHCPv4 address 176.58.106.115/24 via 176.58.106.1
Jul 06 21:39:18 localhost systemd-networkd[1424]: eth0: link configured

I looked around for few hours, but nothing comes up with google. I learned a lot about systemd, but issue is still unresolved. "dhcpcd eth0" gives me my IP in matter of seconds after manually loging in, but this is just temporary solution.

Any hints?

kszonek

Moviuro

Member

Registered: 2012-06-03

Posts: 73

Re: Systemd-networkd doesn't connect to DHCP

Do you have the /etc/systemd/network/eth0.network file with:

[Match]
Name=eth0

[Network]
DHCP=yes

?

---

bspwm, BTRFS over LUKS
Archlinux a lot, FreeBSD more and more...
Murphy's rule: The day you need a backup, you tell yourself you should have created some.

kszonek

Member

Registered: 2011-10-12

Posts: 13

Re: Systemd-networkd doesn't connect to DHCP

Do you have the /etc/systemd/network/eth0.network file with:

[Match]
Name=eth0

[Network]
DHCP=yes

?

I have "/etc/systemd/network/10-dhcp.network":

[Match]
Name=eth0

[Network]
DHCP=yes

[DHCPv4]
UseHostname=false

I have applied an workaround and enabled dhcpcd with "systemctl enable dhcpcd@eth0.service", but it just makes my connection come up without manual intervention. All daemons depending on systemd-networkd-wait-online (like VPN). I am not keen on doing workarounds, it should work as it did with systemd-networkd...

kszonek

Runiq

Member

From: Germany

Registered: 2008-10-29

Posts: 1,053

Re: Systemd-networkd doesn't connect to DHCP

What version of systemd is Linode using?

Edit: Well, that was stupid, as it's Arch. Do you disable IPv6, by any chance? If so, that might be the culprit. Does it work if you use "DHCP=v4" instead of "DHCP=yes"?

Last edited by Runiq (2014-07-07 08:02:34)

---

Dotfiles

kszonek

Member

Registered: 2011-10-12

Posts: 13

Re: Systemd-networkd doesn't connect to DHCP

What version of systemd is Linode using?

Edit: Well, that was stupid, as it's Arch. Do you disable IPv6, by any chance? If so, that might be the culprit. Does it work if you use "DHCP=v4" instead of "DHCP=yes"?

Whole system is up to date

DHCP=v4 changed nothing. I can't remember doing anything about IPv6. Whole problem is strange.

PS. Congrats on your 1000 posts Runiq Your contribution to this was very helpful debugging my problem, a lot of useful information about systemd. Unfortunately I still can't figure out what is wrong.

Runiq

Member

From: Germany

Registered: 2008-10-29

Posts: 1,053

Re: Systemd-networkd doesn't connect to DHCP

DHCP=v4 changed nothing. I can't remember doing anything about IPv6. Whole problem is strange.

Any details about this cheaper plan of yours? Do you have any stray config files or systemd services that may be in conflict with systemd-networkd (dhcpcd, dhclient, etc.)? What does "journalctl -b -u systemd-networkd.service" say?

PS. Congrats on your 1000 posts Runiq

I hadn't even noticed, haha. Thanks.

Your contribution to this was very helpful debugging my problem, a lot of useful information about systemd. Unfortunately I still can't figure out what is wrong.

Apparently it wasn't all that helpful or else you wouldn't have the problem anymore.

---

Dotfiles

kszonek

Member

Registered: 2011-10-12

Posts: 13

Re: Systemd-networkd doesn't connect to DHCP

DHCP=v4 changed nothing. I can't remember doing anything about IPv6. Whole problem is strange.

Any details about this cheaper plan of yours? Do you have any stray config files or systemd services that may be in conflict with systemd-networkd (dhcpcd, dhclient, etc.)? What does "journalctl -b -u systemd-networkd.service" say?

PS. Congrats on your 1000 posts Runiq

I hadn't even noticed, haha. Thanks.

Your contribution to this was very helpful debugging my problem, a lot of useful information about systemd. Unfortunately I still can't figure out what is wrong.

Apparently it wasn't all that helpful or else you wouldn't have the problem anymore.

Cheaper plan is just single core instead of dual, hand half the RAM for half the price. Shouldn't inflict network whatsoever. Maybe network card MAC address changed? Don't now, didn't write it down.

"journalctl -b -u systemd-networkd.service" says nothing, but "journalctl -u systemd-networkd.service" outputs several of:

May 23 13:04:27 aquila systemd[1]: Starting Network Service...
May 23 13:04:27 aquila systemd-networkd[2207]: eth0: link is up
May 23 13:04:27 aquila systemd-networkd[2207]: eth0: carrier on
May 23 13:04:27 aquila systemd[1]: Started Network Service.
May 23 13:04:27 aquila systemd-networkd[2207]: eth0: DHCPv4 address 85.159.XXX.XXX/24 via 85.159.209.1
May 23 13:04:27 aquila systemd-networkd[2207]: eth0: link configured

Last one is 2 months old, so I might have messed something up in the meanwhile - for example while setting up VPN. I am very cautious messing with system settings, but everything is possible. Migration made me power down, I regret I didn't boot it up again before migrating - now I don't know if it is related.

kszonek

Member

Registered: 2011-10-12

Posts: 13

Re: Systemd-networkd doesn't connect to DHCP

I have noticed something else - I have redmine instance running on the server and it can not deliver emails to users via smtp. Last send email was just before shutdown. Is that helpful somehow? Any direction I should look into?

kszonek

Member

Registered: 2011-10-12

Posts: 13

Re: Systemd-networkd doesn't connect to DHCP

Little update. I am tiring to get Linode's Support assistance with this one, since there is something wrong with theirs Arch deployment image.

I tried various scenarios involving migration and it appears it is not a cause of the problem. I added new VPS to my account and upgraded fresh deployment with 'pacman -Suy'. After reboot systemd-networkd.service didn't configure eth0 interface.

Pacman updated following packages:

Packages (66): bash-4.3.018-3  cronie-1.4.11-2  curl-7.37.0-1  dbus-1.8.4-1
               device-mapper-2.02.106-2  dhcpcd-6.4.0-1
               dnssec-anchors-20140629-1  dnsutils-9.9.2.P2-2
               e2fsprogs-1.42.10-1  elfutils-0.158-3  file-5.19-1
               filesystem-2014.06-2  findutils-4.4.2-6  gawk-4.1.1-1
               gcc-libs-4.9.0-5  gettext-0.19.1-1  glibc-2.19-5  gnupg-2.0.25-1
               gpgme-1.5.0-1  grep-2.20-1  htop-1.0.3-2  hwids-20140602-1
               kmod-18-1  libdbus-1.8.4-1  libffi-3.1-2  libgpg-error-1.13-1
               libsystemd-214-2  libusb-1.0.19-1  libutil-linux-2.24.2-1
               licenses-20140629-1  linux-api-headers-3.14.1-1  lvm2-2.02.106-2
               lzo2-2.08-1  man-pages-3.69-1  mdadm-3.3.1-2  mpfr-3.1.2.p10-1
               nano-2.2.6-3  netctl-1.8-1  ntp-4.2.7.p446-1  openssl-1.0.1.h-1
               pacman-4.1.2-6  pacman-mirrorlist-20140706-1  pam-1.1.8-5
               perl-5.20.0-5  procps-ng-3.3.9-3  psmisc-22.21-2
               python2-2.7.8-1  readline-6.3.006-1  rsync-3.1.1-2
               s-nail-14.7.1-1  screen-4.2.1-2  shadow-4.2.1-1  sqlite-3.8.5-1
               sudo-1.8.10.p3-1  sysfsutils-2.1.0-9  sysstat-11.0.0-1
               systemd-214-2  systemd-sysvcompat-214-2  tzdata-2014e-1
               unzip-6.0-7  util-linux-2.24.2-1  vi-1:050325-4  vim-7.4.335-4
               vim-runtime-7.4.335-4  whois-5.1.3-1  xfsprogs-3.2.0-1

So I guess it is related to update of systemd to v214. It works like a charm on all my computers, so Linode deployment image must have something what makes it special Probably many of LinodeArch users experience this problem, but I was first to hit a wall.

Some additional info:
I was unable to send emails from Redmine with TLS enabled SMTP accounts. It did work on another setup in the same datacenter. It worked via telnet, but not with ruby'ies ActionMailer. Setting static IP with netctl was a solution to this one. I don't know what dhcpcd was doing, but my best guess is that it messed DNSs up.

regards
kszonek

rsmarples

Member

Registered: 2009-05-12

Posts: 287

Re: Systemd-networkd doesn't connect to DHCP

I don't know what dhcpcd was doing, but my best guess is that it messed DNSs up.

If you're using systemd for DHCP (which your config and logs indicate) then dhcpcd shouldn't be doing anything at all. Nothing in your config or logs indicate dhcpcd activity so it's could be dormant and not doing anything, so why apply the blame?

If it is doing something then you have your answer - don't run two different DHCP clients at the same time.

Last edited by rsmarples (2014-07-10 08:33:16)

kszonek

Member

Registered: 2011-10-12

Posts: 13

Re: Systemd-networkd doesn't connect to DHCP

Maybe I need to clarify some things:
1. I was using systemd-networkd for some time,
2. It stopped working, so I look for an answer (logs I posted),
3. I disabled systemd-networkd because it didn't work anymore,
4. I enabled dhcpcd@eth0.service,
5. I blame dhcpcd for messing up DNS / IPv6 / whatever, which is cousing some other network services to fail.

Now I use netcfg with static IP and didn't find any problems (just yet).

kszonek

Last edited by kszonek (2014-07-10 09:05:03)

rsmarples

Member

Registered: 2009-05-12

Posts: 287

Re: Systemd-networkd doesn't connect to DHCP

Maybe I need to clarify some things:
1. I was using systemd-networkd for some time,
2. It stopped working, so I look for an answer (logs I posted),
3. I disabled systemd-networkd because it didn't work anymore,
4. I enabled dhcpcd@eth0.service,
5. I blame dhcpcd for messing up DNS / IPv6 / whatever, which is cousing some other network services to fail.

Now I use netcfg with static IP and didn't find any problems (just yet).

You missed out the part explaining how dhcpcd messed it up
Every dhcpcd release I make works for me at least - can you explain how it didn't work for you so I can attempt to address the problem in future release?

kszonek

Member

Registered: 2011-10-12

Posts: 13

Re: Systemd-networkd doesn't connect to DHCP

It would be hard to reproduce right now, since i moved to static IP and am not very keen on taking my server down again, since it all works.

Redmine developer pointed out that there is a chance that it involves iIPv6 - may be. Linode support is looking into this issue, I will let you know if I get their response, Dhcpcd is messing some configuration, but maybe it is because some issues on the other side. Thats my bet - it all works on my other machines.

asb

Member

Registered: 2014-07-11

Posts: 3

Re: Systemd-networkd doesn't connect to DHCP

The issue is a regression in upstream systemd-networkd. I found this via investigation (well, backing out a patch on a hunch). It seems broadcast dhcp responses don't make it through to the VM and systemd-networkd now requests a broadcst response. See my bug report for more info https://bugs.freedesktop.org/show_bug.cgi?id=81225

matth

Member

Registered: 2011-11-09

Posts: 24

Re: Systemd-networkd doesn't connect to DHCP

I ran into this issue today as well. My workaround was to switch systemd-networkd to use a static IPs instead of DHCP.
This allowed me to stick with systemd-networkd instead of switching to netctl or netcfg.

From my testing it was getting the IPv6 address through DHCP but failing on the IPv4 address.

asb

Member

Registered: 2014-07-11

Posts: 3

Re: Systemd-networkd doesn't connect to DHCP

To update this issue, the bug has now been fixed upstream and broadcast is now opt-in. I don't know how long we'll have to wait for systemd-216, so I've filed https://bugs.archlinux.org/task/41215 in the hope the fix can be applied to Arch's systemd-215 package.

asb

Member

Registered: 2014-07-11

Posts: 3

Re: Systemd-networkd doesn't connect to DHCP

Looks like we have to wait for systemd-216. "Reverting isn't an option" though I'm not sure I follow why not.