Arch Linux

Salkay

Member

Registered: 2014-05-22

Posts: 619

[SOLVED] systemd user unit to trigger on network connection

I'm attempted to trigger a script when the network connects. Apparently, the following should work.

[Unit]
Description=Test on connect
Wants=network-online.target
After=network-online.target

[Service]
Type=oneshot
ExecStart=/home/salkay/tmp/test

(Here ~/tmp/test just writes ping output to a temporary file.)

However, in order for user units to reference system units, I'd have to create a user version of the system network-online.target.

$ systemctl --user link /usr/lib/systemd/system/network-online.target
Created symlink from /home/salkay/.config/systemd/user/network-online.target to /usr/lib/systemd/system/network-online.target.

I start the new unit:

$ systemctl --user start network-online.target

After disconnecting and reconnecting my ethernet connection with NetworkManager's plasmoid, nothing seems to happen. Am I doing this correctly?

Last edited by Salkay (2017-05-08 22:57:24)

Awebb

Member

Registered: 2010-05-06

Posts: 6,286

Re: [SOLVED] systemd user unit to trigger on network connection

I'm not sure if the user session is aware of the system targets.

EDIT: Mixed results here: https://bbs.archlinux.org/viewtopic.php?id=185908

EDIT1: I'm sorry, I read your post during my first cup of coffee and didn't realize you already cloned the target.

EDIT2: http://www.freedesktop.org/wiki/Softwar … orkTarget/

network-online.target is a target that actively waits until the nework is "up", where the definition of "up" is defined by the network management software. Usually it indicates a configured, routable IP address of some kind. It's primary purpose is to actively delay activation of services until the network is set up. It is an active target, meaning that is may be pulled in by the services requiring the network to be up, but is not pulled in by the network management service itself. By default all remote mounts defined in /etc/fstab pull this service in, in order to make sure the network is up before it is attempted to connect to a network share. Note that normally, if no service requires it, and if not remote mount point is configured this target is not pulled into the boot, thus avoiding any delays during boot should the network not be available. It is strongly recommended not to pull in this target too liberally: for example network server software should generally not pull this in (since server software generally is happy to accept local connections even before any routable network interface is up), it's primary purpose is network client software that cannot operate without network.

Now we need to find out, how exactly networkmanager defines "up".

EDIT3: Have you verified the general state of your systemd user session? Can you work with other, more generic --user units?

Last edited by Awebb (2015-11-02 08:37:54)

Salkay

Member

Registered: 2014-05-22

Posts: 619

Re: [SOLVED] systemd user unit to trigger on network connection

Thank you for the reply. Firstly, I do have several systemd user services running; they are all working fine.

Secondly, I'm not sure if I'm using the right thing. I'm wanting to trigger a script every single time the network reconnects. I'm using mutt with mbsync. Normally, I use an imapnotify user systemd service to trigger a mailbox sync on new email. However, this won't work immediately after a suspend/resume cycle. Hence, I'd like to trigger a mailbox sync every time after resuming. I've read a little bit, but I'm still not sure if network-online.target triggers after the network is "up" only once after boot, or every subsequent time it goes "up" after being "down".

Awebb

Member

Registered: 2010-05-06

Posts: 6,286

Re: [SOLVED] systemd user unit to trigger on network connection

You can test this. Write a unit that restarts something big on network-online.target (I always use the display manager, you WILL notice :-)) and pull the plug a few times.

This could also be simply Networkmanager not properly starting the target. Have you tried starting the target manually? systemctl start... and then check the status with... status.

EDIT: try start/stop/restart combinations, to avoid state limbo.

Last edited by Awebb (2015-11-02 09:36:36)

Salkay

Member

Registered: 2014-05-22

Posts: 619

Re: [SOLVED] systemd user unit to trigger on network connection

I did try to test some of this. The script that the service calls is ~/tmp/test

#!/usr/bin/bash
/usr/bin/ping 8.8.8.8 -c 5 >> /home/salkay/tmp/ping_on_connect

If I manually start with

systemctl --user start test_on_connect.service

The file is written correctly. Further, the status seems normal enough

$ systemctl --user status test_on_connect.service 
● test_on_connect.service - Test on connect
   Loaded: loaded (/home/salkay/.config/systemd/user/test_on_connect.service; static; vendor preset: enabled)
   Active: inactive (dead)

Nov 02 20:38:19 salkay-XPS-17 systemd[1346]: Starting Test on connect...
Nov 02 20:38:23 salkay-XPS-17 systemd[1346]: Started Test on connect.

Awebb

Member

Registered: 2010-05-06

Posts: 6,286

Re: [SOLVED] systemd user unit to trigger on network connection

I mean have you tried starting the system network-online.target manually, to see if it's systemd not triggering the user unit or if it is Networkmanager not triggering network-online.target?

Salkay

Member

Registered: 2014-05-22

Posts: 619

Re: [SOLVED] systemd user unit to trigger on network connection

Ah. Yes, I also tried this, but this is the limit of my systemd understanding. I suppose in light of your question that this is failing; I guess the target is not triggering my custom script?

$ systemctl --user start network-online.target
$ systemctl --user status network-online.target
● network-online.target - Network is Online
   Loaded: loaded (/usr/lib/systemd/system/network-online.target; linked; vendor preset: enabled)
   Active: active since Mon 2015-11-02 12:18:38 AEDT; 8h ago
     Docs: man:systemd.special(7)
           http://www.freedesktop.org/wiki/Software/systemd/NetworkTarget

Nov 02 12:18:38 salkay-XPS-17 systemd[1346]: Reached target Network is Online.
Nov 02 12:21:00 salkay-XPS-17 systemd[1346]: Reached target Network is Online.
Nov 02 20:57:31 salkay-XPS-17 systemd[1346]: Reached target Network is Online.

And the custom script is not triggered.

Awebb

Member

Registered: 2010-05-06

Posts: 6,286

Re: [SOLVED] systemd user unit to trigger on network connection

Try restart or start/stop as well, because I'm not sure whether starting an already reached target will reach the target again.

Salkay

Member

Registered: 2014-05-22

Posts: 619

Re: [SOLVED] systemd user unit to trigger on network connection

[Restarting], or [stopping then starting] network-online.target has no effect on my custom script.

Awebb

Member

Registered: 2010-05-06

Posts: 6,286

Re: [SOLVED] systemd user unit to trigger on network connection

I'd propose you test your custom unit as a system unit instead of a user unit, to see if it's just a problem with user sessions. I'm running out of ideas. :-(

EDIT: If this target does not work, we can figure out a way to check for a running network connection after a resume (those targets work relatively well in my experience) to solve your actual problem.

Last edited by Awebb (2015-11-02 13:13:00)

firecat53

Member

From: Lake Stevens, WA, USA

Registered: 2007-05-14

Posts: 1,542

Website

Re: [SOLVED] systemd user unit to trigger on network connection

If you are already aware of this alternate solution, then ignore this and I apologize for the noise! It appears you are using NetworkManager and typically the "best" way to scripts network-based events is using NetworkManager dispatcher scripts. For example, I have this script in /etc/NetworkManager/dispatcher.d to restart my SSH tunnels when the network and/or VPN connection is restarted:

#!/bin/bash
### Restart autossh tunnels

if [[ "$2" == 'dhcp4-change' || "$2" == 'up' || "$2" == 'vpn-up' ]]; then
	echo "Restarting SSH tunnels."
	pkill -x --signal 10 autossh  # Restart autossh tunnels. Sends SIGUSR1 to autossh.
fi

Scott

---

Github

Salkay

Member

Registered: 2014-05-22

Posts: 619

Re: [SOLVED] systemd user unit to trigger on network connection

I'd propose you test your custom unit as a system unit instead of a user unit, to see if it's just a problem with user sessions. I'm running out of ideas. :-(

EDIT: If this target does not work, we can figure out a way to check for a running network connection after a resume (those targets work relatively well in my experience) to solve your actual problem.

I created a system-level service at `/etc/systemd/system/test_system_connect.service`. It works fine if run manually, i.e. with `sudo systemctl start test_system_connect.service`. However, it doesn't do anything when disconnecting/reconnecting NetworkManager, nor with restarting or stop/starting network-online.target. Is there anything else I need to do to enable test_system_connect.service, or is it automatically "activated"? I tried enabling it, but I guess it fails because it's linked to network-online.target.

$ sudo systemctl enable test_system_connect.service
The unit files have no [Install] section. They are not meant to be enabled
using systemctl.
Possible reasons for having this kind of units are:
1) A unit may be statically enabled by being symlinked from another unit's
   .wants/ or .requires/ directory.
2) A unit's purpose may be to act as a helper for some other unit which has
   a requirement dependency on it.
3) A unit may be started when needed via activation (socket, path, timer,
   D-Bus, udev, scripted systemctl call, ...).

If you are already aware of this alternate solution, then ignore this and I apologize for the noise! It appears you are using NetworkManager and typically the "best" way to scripts network-based events is using NetworkManager dispatcher scripts. For example, I have this script in /etc/NetworkManager/dispatcher.d to restart my SSH tunnels when the network and/or VPN connection is restarted:

#!/bin/bash
### Restart autossh tunnels

if [[ "$2" == 'dhcp4-change' || "$2" == 'up' || "$2" == 'vpn-up' ]]; then
	echo "Restarting SSH tunnels."
	pkill -x --signal 10 autossh  # Restart autossh tunnels. Sends SIGUSR1 to autossh.
fi

Scott

Thank you for the suggestion. However, I've have tried this before, but ran into problems accessing gnome-keyring. Because this script is activated with root privileges, it's non-trivial to access the keyring. Also, I sometimes find NetworkManager a bit flakey. My computer seems to know when it's connected to the network. When it isn't, `ping` gives `connect: Network is unreachable`. However, NetworkManager can think I'm not connected, at least according to the plasmoid. If NetworkManager is confused, then conceivably the triggers won't fire correctly.

Salkay

Member

Registered: 2014-05-22

Posts: 619

Re: [SOLVED] systemd user unit to trigger on network connection

Bump. Thanks for the help so far. It seems that no one really knows how to trigger an action on every network connection, so perhaps I should investigate systemd units to trigger on resuming from suspend instead. If so, I guess I could test for network connection in a loop.

#EDIT
Ugh, seems like this is non-trivial too. I created a (system) systemd unit as suggested in the wiki. It triggers fine on resume with a test script. However, it fails to access gnome-keyring, exactly as before. So I tried the same thing as my first post. I created a systemd user unit that triggered on resume, at ~/.config/systemd/user/resume.service

[Unit]
Description=User resume actions
After=suspend.target

[Service]
Type=simple
ExecStart=/home/salkay/.local/bin/resume-hook

[Install]
WantedBy=suspend.target

When called manually with `systemctl --user start resume.service`, it accesses the gnome-keyring fine. I then created the user version of the system suspend.target.

$ systemctl --user link /usr/lib/systemd/system/suspend.target
Created symlink from /home/salkay/.config/systemd/user/suspend.target to /usr/lib/systemd/system/suspend.target.

I tried to start the new unit, but got errors, so linked the missing unit dependencies too:

$ systemctl --user start suspend.target
Failed to start suspend.target: Unit systemd-suspend.service failed to load: No such file or directory.

$ systemctl --user link /usr/lib/systemd/system/systemd-suspend.service
Created symlink from /home/salkay/.config/systemd/user/systemd-suspend.service to /usr/lib/systemd/system/systemd-suspend.service.

$ systemctl --user start suspend.target
Failed to start suspend.target: Unit sleep.target failed to load: No such file or directory.

$ systemctl --user link /usr/lib/systemd/system/sleep.target
Created symlink from /home/salkay/.config/systemd/user/sleep.target to /usr/lib/systemd/system/sleep.target.

$ systemctl --user start suspend.target
A dependency job for suspend.target failed. See 'journalctl -xe' for details.

At the end, I seemed to run into permission problems.

Nov 15 21:01:25 salkay-XPS-17 systemd[608]: Reached target Sleep.
-- Subject: Unit UNIT has finished start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit UNIT has finished starting up.
-- 
-- The start-up result is done.
Nov 15 21:01:25 salkay-XPS-17 systemd[608]: Starting Suspend...
-- Subject: Unit UNIT has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit UNIT has begun starting up.
Nov 15 21:01:25 salkay-XPS-17 systemd-sleep[4517]: Failed to open /sys/power/state: Permission denied
Nov 15 21:01:25 salkay-XPS-17 systemd[608]: systemd-suspend.service: Main process exited, code=exited, status=1/FAILURE
Nov 15 21:01:25 salkay-XPS-17 systemd[608]: Failed to start Suspend.
-- Subject: Unit UNIT has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit UNIT has failed.
-- 
-- The result is failed.
Nov 15 21:01:25 salkay-XPS-17 systemd[608]: Dependency failed for Suspend.
-- Subject: Unit UNIT has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit UNIT has failed.
-- 
-- The result is dependency.
Nov 15 21:01:25 salkay-XPS-17 systemd[608]: suspend.target: Job suspend.target/start failed with result 'dependency'.
Nov 15 21:01:25 salkay-XPS-17 systemd[608]: sleep.target: Unit not needed anymore. Stopping.
Nov 15 21:01:25 salkay-XPS-17 systemd[608]: systemd-suspend.service: Unit entered failed state.
Nov 15 21:01:25 salkay-XPS-17 systemd[608]: systemd-suspend.service: Failed with result 'exit-code'.
Nov 15 21:01:25 salkay-XPS-17 systemd[608]: Stopped target Sleep.
-- Subject: Unit UNIT has finished shutting down
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit UNIT has finished shutting down.
Nov 15 21:01:25 salkay-XPS-17 systemd[608]: Started User resume actions.
-- Subject: Unit UNIT has finished start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit UNIT has finished starting up.
-- 
-- The start-up result is done.

I tried a suspend-resume cycle anyway, but unsurprisingly it failed to trigger on resume.

Last edited by Salkay (2015-11-15 10:06:24)

Raynman

Member

Registered: 2011-10-22

Posts: 1,539

Re: [SOLVED] systemd user unit to trigger on network connection

Some pointers:

• Read the page about the network target linked by Awebb in #2.

• You can also

  $ systemctl cat network-online.target

  to see there is hardly anything in there -- that is not magically going to start your service. Targets are little more than named synchronization points, used together with other units for ordering or grouping.

• Similiarly, check out NetworkManager-wait-online.service. It uses nm-online; perhaps you can use that.

• Now you explicitly say something needs to happen on every network connection. (It wouldn't hurt to fully describe your actual goal.) If you've read the page, you know the network targets are not meant for that. For a NM user, the dispatcher mechanism is interesting, especially for system-wide (and usually running as root) scripts. If you want to run something as a user, you can try to make it work, or you can connection to NM dbus signals to create a user-level dispatcher. I believe you can find some python scripts online that take care of most of the work of connecting to these events.

Last edited by Raynman (2015-11-15 10:39:10)

Salkay

Member

Registered: 2014-05-22

Posts: 619

Re: [SOLVED] systemd user unit to trigger on network connection

Thanks Raynman. I must have missed the details in the link earlier. You are very correct. I should describe my goal, and prevent this from being an XY problem.

I've moved to mutt as my email client. I am using imapnotify to trigger an Inbox synchronisation. Hence, when new email arrives in the server, imapnotify triggers mbsync to download the new emails, and I am notified locally. However, after suspending (and/or network reconnection), imapnotify often loses the connection, then stalls. It takes 10 minutes for it to reconnect, and I suspect it doesn't notify for emails that arrive when it was down anyway. Hence, I find myself manually syncing my inbox every time I resume from suspend.

As per my previous posts, I've tried a few system-level solutions (systemd and NetworkManager dispatcher), but all of them fail to access gnome-keyring, where my email password is stored. The user-level systemd solutions can access the keyring, but don't seem to be triggered.

From your post, it seems that systemd network-based solutions are not the correct method, since they are not triggered on every reconnect. Conceivably, NetworkManager could work, if I can somehow create a user-level dispatcher, but I suspect that this may be quite difficult. (Setting $DISPLAY and $USER hasn't been enough to access gnome-keyring.) I suspect that user-level systemd resume services might be the simplest solution, if I could get them to trigger.

Awebb

Member

Registered: 2010-05-06

Posts: 6,286

Re: [SOLVED] systemd user unit to trigger on network connection

X/Y problem, I'd say. We danced around NM and systemd targets, when the culprit is a broken imapnotify.

Here, line 70-94 of imapnotify:

  connection.once('error', function(error) {
    self.logger.error({box: box}, 'Error registered');
    self.logger.error({box: box}, error);
    if (! self.errors[box]) {
      self.logger.info({box: box}, 'Restarting immediately')
      self.errors[box] = {'time': Date.now(), 'count': 1 }
      connection.emit('shouldReconnect')
    } else {
      var delta = Date.now() - self.errors[box]['time']
        , errCount = self.errors[box]['count'];


      if (self.errors[box]['count'] == 3) {
        throw Error('Max retry limit reached')
      }

      var restartTimeout = errCount*3000
      self.errors[box]['count'] = self.errors[box]['count'] + 1
      self.logger.info({box: box}, 'Scheduling restart in ' + restartTimeout)
      setTimeout(function() {
        connection.emit('shouldReconnect')
      }, restartTimeout);

    }
  })

I don't speak node, but I'd say it tries three times and then waits 3*3000 time units. I'm not sure if that's seconds, but I bet decreasing this number will reduce the timeout until the next attempt to connect.

Short check list:

* network-online.target actually does something
* you got the NM dispatcher to work

Apropos dispatcher, have you tried a dispatcher, that runs as the user and calls systemd --user? Like a "su $USER -c systemctl --user restart myfancy.service"?

Salkay

Member

Registered: 2014-05-22

Posts: 619

Re: [SOLVED] systemd user unit to trigger on network connection

I bet decreasing this number will reduce the timeout until the next attempt to connect.

Thank you for digging into the code, Awebb. Changing that number does indeed seem like a valid fix. I've been monitoring the imapnotify process via `systemctl status`. Previously, it would often choke and take 10 minutes to restart. However I'm not seeing this now at all. I'll keep monitoring it, but it looks like this would be the best solution if it chokes again. I also checked, and emails that are received when my computer is suspended are still triggered in imapnotify on resume, so that sorts out that concern.

Apropos dispatcher, have you tried a dispatcher, that runs as the user and calls systemd --user? Like a "su $USER -c systemctl --user restart myfancy.service"?

Huh… that is a fantastic idea! Alternatively, I could also use the system-level systemd unit to trigger the user-level systemd unit. It's probably moot now, but I'll keep both options in mind.

Thank you for all your help.