Arch Linux

dawnofman

Member

Registered: 2019-07-26

Posts: 140

[solved]cannot manage to set multiple interfaces with systemd

My /etc/systemd/network/ directory has 4 files named:

11-BroadCom-BCM-5720.link
12-BroadCom-BCM-5720.link
13-BroadCom-BCM-5720.link
14-BroadCom-BCM-5720.link

where each of the above files is like following:

[Match]
MACAddress=##:##:##:##:##:##

[Link]
Name=[switch1|server2|server3|server4]

which upon server restart brings me 4 interfaces named accordingly instead of the default eno(1-4) devices; so far so good.

My former /etc/systemd/system/network.service configured for JUST ONE interface was like following:

[Unit]
Wants=network.target
Before=network.target
BindsTo=sys-subsystem-net-devices-switch1.device
After=sys-subsystem-net-devices-switch1.device

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/bin/ip link set dev switch1 up
ExecStart=/usr/bin/ip route add default via 192.168.77.9 dev switch1
ExecStart=/usr/bin/ip address add 192.168.77.11/29 dev switch1
ExecStop=/usr/bin/ip address flush dev switch1
ExecStop=/usr/bin/ip link set dev switch1 down

[Install]
WantedBy=multi-user.target

It works as expected.

Now I want to also use another interface (a second one as dedicated link between the servers without touching the switch) so for starters I did the following:

[Unit]
Wants=network.target
Before=network.target
BindsTo=sys-subsystem-net-devices-switch1.device
BindsTo=sys-subsystem-net-devices-server2.device
After=sys-subsystem-net-devices-switch1.device
After=sys-subsystem-net-devices-server2.device

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/bin/ip link set dev switch1 up
ExecStart=/usr/bin/ip link set dev server2 up
ExecStart=/usr/bin/ip route add default via 192.168.77.9 dev switch1
ExecStart=/usr/bin/ip address add 192.168.77.11/29 dev switch1
ExecStart=/usr/bin/ip address add 192.168.78.11/29 dev server2
ExecStop=/usr/bin/ip address flush dev switch1
ExecStop=/usr/bin/ip address flush dev server2
ExecStop=/usr/bin/ip link set dev switch1 down
ExecStop=/usr/bin/ip link set dev server2 down

[Install]
WantedBy=multi-user.target

soon cleaned-up as following:

[Unit]
Wants=network.target
Before=network.target
BindsTo=sys-subsystem-net-devices-switch1.device
BindsTo=sys-subsystem-net-devices-server2.device
After=sys-subsystem-net-devices-switch1.device
After=sys-subsystem-net-devices-server2.device

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/bin/ip link set dev switch1 up
ExecStart=/usr/bin/ip link set dev server2 up
ExecStop=/usr/bin/ip address flush dev switch1
ExecStop=/usr/bin/ip address flush dev server2
ExecStop=/usr/bin/ip link set dev switch1 down
ExecStop=/usr/bin/ip link set dev server2 down

[Install]
WantedBy=multi-user.target

while placing the address info on two dedicated files:

/etc/systemd/network/30-servers-default
/etc/systemd/network/40-servers-link-dedicated

[Match]
Name=switch1

[Network]
Address=192.168.77.11/29
Gateway=192.168.77.9

[Match]
Name=server2

[Network]
Address=192.168.78.11/29

Problem is it is not working as expected. The switch1 and server2 interfaces show when I do ip address show but they are not assigned the addresses I specify on the associated files. Obviously each time I change something I do systemctl stop network; systemctl daemon-reload; systemctl start network; and even I tried systemctl disable network; systemctl enable network; systemctl start network;

What am I doing wrong ?

The goal is to bond the 3 server(2-4) interfaces on each server for a 3GE aggregated and dedicated link between two servers while all normal traffic will keep going through the switch1 interface; think file sharing and the like (eventually moving the 29 mask on the dedicated links to 30 for a two-point connection).

Last edited by dawnofman (2022-04-17 17:34:21)

dawnofman

Member

Registered: 2019-07-26

Posts: 140

Re: [solved]cannot manage to set multiple interfaces with systemd

This is a reply to myself to clarify some things.

I managed to assign the IP addresses as required with the following network.service configuration because I cannot manage to replicate this behavior with the stand-alone files on /etc/systemd/network/:

[Unit]
Wants=network.target
Before=network.target
BindsTo=sys-subsystem-net-devices-switch1.device
BindsTo=sys-subsystem-net-devices-server2.device
After=sys-subsystem-net-devices-switch1.device
After=sys-subsystem-net-devices-server2.device

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/bin/ip address flush dev switch1 scope global
ExecStart=/usr/bin/ip address flush dev server2 scope global
ExecStart=/usr/bin/ip address add local 192.168.77.11/29 dev switch1
ExecStart=/usr/bin/ip address add local 192.168.78.11/29 dev server2
ExecStart=/usr/bin/ip link set dev switch1 up
ExecStart=/usr/bin/ip link set dev server2 up
ExecStart=/usr/bin/ip route add default via 192.168.77.9 dev switch1
ExecStop=/usr/bin/ip link set dev switch1 down
ExecStop=/usr/bin/ip link set dev server2 down

[Install]
WantedBy=multi-user.target

Question is:

How can I configure a point-to-point connection with a 30 mask ?

eg: 192.168.78.11/30 should be the same address as 192.168.78.10./30 right ? (30 mask = all ones)

Should I use ip address add peer instead of the default "local" or something like this ?

dawnofman

Member

Registered: 2019-07-26

Posts: 140

Re: [solved]cannot manage to set multiple interfaces with systemd

self-solved:

server one primary interface (aka switch1) (default) is 192.168.78.10 255.255.255.248 (ie mask=29) gw 192.168.78.9 via switch1
server two primary interface (aka switch1) (default) is 192.168.78.11 255.255.255.248 (ie mask=29) gw 192.168.78.9 via switch1

server one secondary interface (aka server2) (dedicated-link) is 192.168.79.10 255.255.255.254 (ie mask=31) gw 192.168.79.11 via server2
server two secondary interface (aka server2) (dedicated-link) is 192.168.79.11 255.255.255.254 (ie mask=31) gw 192.168.79.10 via server2

[Unit]
Wants=network.target
Before=network.target
BindsTo=sys-subsystem-net-devices-switch1.device
BindsTo=sys-subsystem-net-devices-server2.device
After=sys-subsystem-net-devices-switch1.device
After=sys-subsystem-net-devices-server2.device

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/bin/ip address flush dev switch1 scope global
ExecStart=/usr/bin/ip address flush dev server2 scope global
{computer-specific-block-A}
ExecStart=/usr/bin/ip link set dev switch1 up
ExecStart=/usr/bin/ip link set dev server2 up
{computer-specific-block-B}
ExecStart=/usr/bin/ip route add default via 192.168.77.9 dev switch1
ExecStop=/usr/bin/ip link set dev switch1 down
ExecStop=/usr/bin/ip link set dev server2 down

[Install]
WantedBy=multi-user.target

{computer-specific-block-A} for server one:

ExecStart=/usr/bin/ip address add local 192.168.78.10/31 dev server2
ExecStart=/usr/bin/ip address add local 192.168.77.10/29 dev switch1

{computer-specific-block-A} for server two:

ExecStart=/usr/bin/ip address add local 192.168.78.11/31 dev server2
ExecStart=/usr/bin/ip address add local 192.168.77.11/29 dev switch1

{computer-specific-block-B} for server one:

ExecStart=/usr/bin/ip route add 192.168.78.11 via 192.168.78.10 dev server2
ExecStart=/usr/bin/ip route add default via 192.168.77.9 dev switch1

{computer-specific-block-B} for server two:

ExecStart=/usr/bin/ip route add 192.168.78.10 via 192.168.78.11 dev server2
ExecStart=/usr/bin/ip route add default via 192.168.77.9 dev switch1

Besides previously getting some things wrong like the mask for a two-point connection (classless) as 30 instead of 31 the key for this configuration are the routes to reach the other server via the non-default interface, in my case the server2 interface.

Since my servers still have two more interfaces to play-with, this configuration can be improved bonding the server2 with the other two to make a 3GE bonded-interface.

If the switch/router goes down for whatever reason (obviously cutting access to the outside world) the servers can still talk to each other; think backup processes and the like. A plus.

Everything working fine.

PS: I would love to place all these ip commands on proper unit files under /etc/systemd/network/ but that is something I cannot managed to do, albeit a low priority task for the time being. But if anyone can tip me on the subject ...