Server Down - netctl troubles

florck · 2014-03-09 23:30:45

Dear all,

I own a dedicated server hosted by OVH.
This has been installed almost a year ago and was working perfectly well until today. I did some upgrades. After a reboot, unable to get even a ping on that server. I booted to rescue-mode and inspected journalctl and here is what I found.

Mar 10 00:09:10 ks3100448 systemd[1]: Started Create Volatile Files and Directories.
Mar 10 00:09:10 ks3100448 systemd[1]: Starting Update UTMP about System Reboot/Shutdown...
Mar 10 00:09:10 ks3100448 systemd[1]: Started Update UTMP about System Reboot/Shutdown.
Mar 10 00:09:11 ks3100448 systemd-udevd[3266]: Could not apply link config to dummy0
Mar 10 00:10:39 ks3100448 systemd[1]: Job dev-sda3.device/start timed out.
Mar 10 00:10:39 ks3100448 systemd[1]: Timed out waiting for device dev-sda3.device.
Mar 10 00:10:39 ks3100448 systemd[1]: Dependency failed for /dev/sda3.
Mar 10 00:10:39 ks3100448 systemd[1]: Dependency failed for Swap.
Mar 10 00:10:39 ks3100448 systemd[1]: Job sys-subsystem-net-devices-enp1s0.device/start timed out.
Mar 10 00:10:39 ks3100448 systemd[1]: Timed out waiting for device sys-subsystem-net-devices-enp1s0.
Mar 10 00:10:39 ks3100448 systemd[1]: Dependency failed for dhcpcd on enp1s0.
Mar 10 00:10:39 ks3100448 systemd[1]: Starting Network.
Mar 10 00:10:39 ks3100448 systemd[1]: Reached target Network.
Mar 10 00:10:39 ks3100448 systemd[1]: Starting System Initialization.
Mar 10 00:10:39 ks3100448 systemd[1]: Reached target System Initialization.
Mar 10 00:10:39 ks3100448 systemd[1]: Starting Daily Cleanup of Temporary Directories.
Mar 10 00:10:39 ks3100448 systemd[1]: Started Daily Cleanup of Temporary Directories.
Mar 10 00:10:39 ks3100448 systemd[1]: Starting Timers.
Mar 10 00:10:39 ks3100448 systemd[1]: Reached target Timers.

I expect the line Mar 10 00:10:39 ks3100448 systemd[1]: Timed out waiting for device sys-subsystem-net-devices-enp1s0. to be the relevant one.

It has been for 4 hours I try everything I can alternate between rescue mode and standard mode, I even changed, because at the beginning it was an interface named eth0, I activated the "new way naming", still nothing. Now I have a file named /etc/netctl/ovh_net_eth0 (didn't changed the name) that contains the following :

[root@rescue /]# ls /etc/udev/rules.d/
[root@rescue /]#

# IPv4 Setup
Description='A basic static ethernet connection'
Interface=enp1s0
Connection=ethernet
IP=static
Address=('37.59.62.xxx/24')
Gateway=('37.59.62.254')
DNS=('213.186.33.99')

# IPv6 Setup
IP6=static
Address6=('2001:xxxx:8:79df::1/64')
IPCustom=('-6 route add 2001:xxxx:x:79ff:ff:ff:ff:ff dev eth0' '-6 route add default via 2001:xxxx:x:79ff:ff:ff:ff:ff')

it is activated through netctl enable

I scheduled by a cron some command lines to be launched in standard mode and inspected the result in rescue mode, I can't explain why the interface is mentionned as "down".

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: bond0: <BROADCAST,MULTICAST,MASTER> mtu 1500 qdisc noop state DOWN mode DEFAULT group default
    link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
3: dummy0: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN mode DEFAULT group default
    link/ether 76:f6:22:3c:cf:bb brd ff:ff:ff:ff:ff:ff
4: enp1s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 00:22:4d:87:8e:5f brd ff:ff:ff:ff:ff:ff
5: tunl0: <NOARP> mtu 1480 qdisc noop state DOWN mode DEFAULT group default
    link/ipip 0.0.0.0 brd 0.0.0.0
6: sit0: <NOARP> mtu 1480 qdisc noop state DOWN mode DEFAULT group default
    link/sit 0.0.0.0 brd 0.0.0.0
7: ip6tnl0: <NOARP> mtu 1452 qdisc noop state DOWN mode DEFAULT group default
    link/tunnel6 :: brd ::

It is out of my ability scope, then I kindly look for your help!

Thank you a lot,

Florck

Gcool · 2014-03-10 07:37:57

Mar 10 00:10:39 ks3100448 systemd[1]: Dependency failed for dhcpcd on enp1s0.

Since you're using a static ip setup (obviously), could you perhaps try disabling dhcpcd.service and see what happens if it doesn't interfere?

florck · 2014-03-10 08:32:04

I'm sorry, I made a mistake I posted the log after a reboot when I tried to deactivate the static connection to activate dhcp.
When dhcp is unactivated, I get :

Mar 10 08:29:45 ks3100448 systemd[1]: Job sys-subsystem-net-devices-enp1s0.device/start timed out.
Mar 10 08:29:45 ks3100448 systemd[1]: Timed out waiting for device sys-subsystem-net-devices-enp1s0.device.
Mar 10 08:29:45 ks3100448 systemd[1]: Dependency failed for A basic static ethernet connection.

in journalctl

Thank you for your help

Centzilius · 2014-03-10 14:51:03

Try

netctl disable <networkname>
netctl enable <networkname>

or

rm '/etc/systemd/system/multi-user.target.wants/netctl@<networkname>.service'
ln -s '/etc/systemd/system/netctl@<networkname>.service' '/etc/systemd/system/multi-user.target.wants/netctl@<networkname>.service'

I have no idea why but it fixed my problem some days ago when modifying my netctl file.

florck · 2014-03-10 15:30:53

I tried this unsuccessfully.

I've solved temporarly my issue by downgrading systemd to the previous version. But this is not satisfying !!

The Compiler · 2014-03-12 12:40:47

A workmate of mine has had the same problem, with an OVH server as well (unfortunately he solved it by switching to Debian...).

I updated two OVH servers and didn't reboot yet; not sure if I should try.

One suspicion I have is the fact a normal OVH configuration netboots their kernel with your own rootfs, and maybe their kernel is tinkered with/outdated/... enough to cause something weird with systemd 210.

The Compiler · 2014-03-13 06:52:18

I've now rebooted both of my servers, without problems.

Although I don't use netctl (just a simple service file for static networking), and especially I don't use OVH's weird Arch image (where they did some weird stuff and some backdoors for themselves) but use a vanilla arch installed myself.

Last edited by The Compiler (2014-03-13 06:52:32)

WonderWoofy · 2014-03-13 07:07:31

@The Compiler, since you use static IPv6, you should take a look at the new systemd-networkd. It is quite neat and might fit your needs quite well. There is no DHCP6 as of yet, which is why I say that your use of a static IP is fitting. The configuration syntax is in simple INI style formatting, and learning about it will also teach you about how network interface renaming is handled in the newest versions.

mattage · 2014-03-13 20:31:17

You might be suffering from the same issue I was having. I've posted an answer already in another topic: https://bbs.archlinux.org/viewtopic.php … 7#p1391147

Arch Linux

#1 2014-03-09 23:30:45

Server Down - netctl troubles

#2 2014-03-10 07:37:57

Re: Server Down - netctl troubles

#3 2014-03-10 08:32:04

Re: Server Down - netctl troubles

#4 2014-03-10 14:51:03

Re: Server Down - netctl troubles

#5 2014-03-10 15:30:53

Re: Server Down - netctl troubles

#6 2014-03-12 12:40:47

Re: Server Down - netctl troubles

#7 2014-03-13 06:52:18

Re: Server Down - netctl troubles

#8 2014-03-13 07:07:31

Re: Server Down - netctl troubles

#9 2014-03-13 20:31:17

Re: Server Down - netctl troubles

Board footer