You are not logged in.

#1 2016-03-23 22:30:21

oniram
Member
Registered: 2010-04-01
Posts: 5

restart systemd service on process death [SOLVED]

I wrote a systemd service to start a license server. The license server is lmgrd by FLEXnet and is needed by IDL. I run the service as a systemd user service, it gets started at login and things work nicely.
However, the license server usually dies after a few suspend/resume cycles. The problem is that during early suspend my wireless card MAC address shows as 00:00:....:00, which confuses the license server. The MAC address is used to identify my laptop and match it against the license. The license server thinks the license doesn't match the machine and quits.

Here's my service file:

idl-lmgrd.service
[Unit]
Description=IDL License Manager
ConditionPathExists=/opt/exelis/idl85/bin/lmgrd
After=network.target

[Service]
Type=simple
WorkingDirectory=/opt/exelis/license/
StandardOutput=null
ExecStart=/opt/exelis/idl85/bin/lmgrd -local -z -c license.dat 
ExecStop=/opt/exelis/idl85/bin/lmdown -q -c license.dat 
# lmgrd quits with exitcode 15
SuccessExitStatus=15
Restart=on-failure

[Install]
WantedBy=default.target

I specify that systemd should restart the service on failure, but this never happens. To fully understand why let me explain how the license manager seems to operate. The main lmgrd process spawns a child idl_lmgrd which performs the actual license magic. So lmgrd is just a manager that monitors the license process idl_lmgrd.

$ systemctl --user status idl-lmgrd
● idl-lmgrd.service - IDL License Manager
   Loaded: loaded (/home/xxxxx/.config/systemd/user/idl-lmgrd.service; enabled; vendor preset: enabled)
   Active: active (running) since Wed 2016-03-23 09:10:10 MDT; 55min ago
 Main PID: 781 (lmgrd)
   CGroup: /user.slice/user-1000.slice/user@1000.service/idl-lmgrd.service
           ├─781 /opt/exelis/idl85/bin/bin.linux.x86_64/lmgrd -local -z -c license.dat
           └─787 idl_lmgrd -T hostname 11.12 3 -c license.dat .......

Mar 23 09:10:10 hostname systemd[771]: Started IDL License Manager.

When we run into trouble during resume, idl_lmgrd quits claiming that the license does not match the machine. The process lmgrd notices this and logs something like: invalid license, won't try to restart idl_lmgrd. But lmgrd just sits there like a dummy, doing nothing.
To summarize, whenever things break we have this situation:

  • idl_lmgrd quits, but systemd doesn't know about this process, so nothing happens

  • lmgrd sits happily doing nothing, systemd doesn't see anything wrong and nothing happens

I would like systemd to automatically restart the service whenever this situation is encountered.
So here's my question: is there anyway to do this? To make systemd restart the service whenever the child (idl_lmgrd) dies?
Notice that lmgrd does not save the PID of its child anywhere.

Last edited by oniram (2016-03-31 18:23:21)

Offline

#2 2016-03-24 10:05:05

berbae
Member
From: France
Registered: 2007-02-12
Posts: 1,302

Re: restart systemd service on process death [SOLVED]

Could this https://wiki.archlinux.org/index.php/Po … th_systemd help you?
Particularly the 'Suspend/resume service files'.
A search on the web with the terms 'systemd suspend resume' could also provide useful infos.

The idea would be to stop your idl-lmgrd.service on suspend, and restart it on resume using systemd service files as shown in the wiki.

Offline

#3 2016-03-31 18:20:29

oniram
Member
Registered: 2010-04-01
Posts: 5

Re: restart systemd service on process death [SOLVED]

I finally got this working, I'm gonna post here what I did in case it helps somebody else. Thanks for your suggestion @berbae, it sent me on the right track to figure this out.

The basic idea is to use a sleep hook to stop/start my idl-lmgrd.service when the laptop suspends/resumes. I'm using the combined Suspend/Resume file suggested by the wiki: https://wiki.archlinux.org/index.php/Po … rvice_file
This is my sleep hook file:

~/.config/systemd/user/sleep-hook@.service
[Unit]
Description=User Sleep Hook for %i.service
Before=user-sleep.target
StopWhenUnneeded=yes

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=-/usr/bin/systemctl --user stop %i
ExecStop=-/usr/bin/systemctl --user start %i

[Install]
WantedBy=user-sleep.target

It's enabled by:

systemctl --user enable sleep-hook@idl-lmgrd

The hook runs before 'sleep.target' (actually user-sleep.target but bear with me for a bit), which stops the license manager before suspend. After resume, the hook is stopped, which restarts the license manager.
However, notice that the hook is a user service and can't depend on the system wide 'sleep.target'. As the wiki says:

systemd --user runs as a separate process from the systemd --system process. User units can not reference or depend on system units.

So after much looking around I found this forum post that explains how to propagate a system wide target to a systemd --user session. What we need is:

  • a system wide service that propagates the system wide sleep.target to the user instance

  • a user defined user-sleep.target

The system wide "propagator":

/etc/systemd/system/user-sleep@.service
[Unit]
Description=Propagate sleep target to user session
Before=sleep.target
StopWhenUnneeded=yes

[Service]
Type=oneshot
Environment=DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/%I/bus
User=%I
RemainAfterExit=yes
ExecStart=/usr/bin/systemctl --user start user-sleep.target
ExecStop=/usr/bin/systemctl --user stop user-sleep.target

[Install]
WantedBy=sleep.target

The user sleep target:

~/.config/systemd/user/user-sleep.target
[Unit]
Description=User Sleep Target

That's why the sleep hook above depends on 'user-sleep.target'.
The only thing left to do is to enable the system wide "propagator" service (as root):

systemctl enable user-sleep@1000

Notice you have to use your user ID instead of your user name.

To recap, let me copy the systemd logs that show what's going on behind the scenes in a suspend/resume cycle:

-- Suspend Starts --
Mar 31 11:19:26 xxxx systemd[1]: Starting Propagate sleep target to user session...
Mar 31 11:19:26 xxxx systemd[618]: Starting User Sleep Hook for idl-lmgrd.service...
Mar 31 11:19:26 xxxx systemd[618]: Stopping IDL License Manager...
Mar 31 11:19:27 xxxx systemd[618]: Stopped IDL License Manager.
Mar 31 11:19:27 xxxx systemd[618]: Started User Sleep Hook for idl-lmgrd.service.
Mar 31 11:19:27 xxxx systemd[618]: Reached target User Sleep Target.
Mar 31 11:19:27 xxxx systemd[1]: Started Propagate sleep target to user session.
Mar 31 11:19:27 xxxx systemd[1]: Reached target Sleep.
-- Resume Starts ---
Mar 31 11:19:39 xxxx systemd[1]: sleep.target: Unit not needed anymore. Stopping.
Mar 31 11:19:39 xxxx systemd[1]: Stopped target Sleep.
Mar 31 11:19:39 xxxx systemd[1]: user-sleep@1000.service: Unit not needed anymore. Stopping.
Mar 31 11:19:39 xxxx systemd[1]: Stopping Propagate sleep target to user session...
Mar 31 11:19:39 xxxx systemd[618]: Stopped target User Sleep Target.
Mar 31 11:19:39 xxxx systemd[618]: sleep-hook@idl-lmgrd.service: Unit not needed anymore. Stopping.
Mar 31 11:19:39 xxxx systemd[618]: Stopping User Sleep Hook for idl-lmgrd.service...
Mar 31 11:19:39 xxxx systemd[1]: Stopped Propagate sleep target to user session.
Mar 31 11:19:39 xxxx systemd[618]: Started IDL License Manager.
Mar 31 11:19:39 xxxx systemd[618]: Stopped User Sleep Hook for idl-lmgrd.service.

The process "systemd[618]" is the systemd --user process.

Offline

Board footer

Powered by FluxBB