profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

skuggo · 2015-09-06 20:33:15

I can see that the service is in there with your command. However this is the outputs:

☁  ~  psd p
Profile-sync-daemon v6.00 on Arch Linux

 Systemd service is currently failed.
 Systemd timer is currently active.
 Overlayfs v23 is currently active.

Psd will manage the following per /home/skuggo/.psd/psd.conf:

 browser/psname:  chromium/chromium
 owner/group id:  skuggo/1000
 sync target:     /home/skuggo/.config/chromium
 tmpfs dir:       /tmp/skuggo-chromium
 profile size:    60M
 overlayfs size:  
 recovery dirs:   none


☁  ~  systemctl --user start psd.service
Job for psd.service failed because the control process exited with error code. See "systemctl status psd.service" and "journalctl -xe" for details.
☁  ~  systemctl --user status psd.service
● psd.service - Profile-sync-daemon
   Loaded: loaded (/usr/lib/systemd/user/psd.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Sun 2015-09-06 22:29:57 CEST; 664ms ago
     Docs: man:psd(1)
           man:profile-sync-daemon(1)
           https://wiki.archlinux.org/index.php/Profile-sync-daemon
  Process: 23063 ExecStart=/usr/bin/profile-sync-daemon resync (code=exited, status=1/FAILURE)
 Main PID: 23063 (code=exited, status=1/FAILURE)

Thanks!

graysky · 2015-09-06 20:44:09

Do you have a browser open while you try to start it? To get more output why the service fails:

journalctl --user-unit psd

skuggo · 2015-09-06 20:57:59

Hehe well that is probably the culprit, figured I was starting the daemon just so it would run at boot at which point no browsers are open. systemctl --user enable psd didn't do the trick to get it running at boot. Had to shut it all down and systemctl --user start psd beforehand.

Thanks alot, works fine now

Appreciate it!

graysky · 2015-09-06 21:09:08

You should see more verbose output from that command; here is an example I setup with chromium open before I ran the daemon:

Sep 06 16:07:31 stripe systemd[677]: Starting Profile-sync-daemon...
Sep 06 16:07:31 stripe profile-sync-daemon[26658]: Refusing to start; chromium is running by facade!
Sep 06 16:07:31 stripe systemd[677]: psd.service: Main process exited, code=exited, status=1/FAILURE
Sep 06 16:07:31 stripe systemd[677]: Failed to start Profile-sync-daemon.
Sep 06 16:07:31 stripe systemd[677]: psd.service: Unit entered failed state.
Sep 06 16:07:31 stripe systemd[677]: psd.service: Failed with result 'exit-code'.

n125 · 2015-09-07 02:20:59

Since updating to the latest version, I get the following message in my journal every time psd synchronizes. I am not sure what it means nor what I should do about it, but everything appears to be working fine otherwise (as far as I can tell, anyway).

systemd[955]: local-fs.target: Cannot add dependency job, ignoring: Unit local-fs.target failed to load: No such file or directory.
systemd[955]: Started Profile-sync-daemon.

My set up isn't fancy: It's just synchronizing my Chromium profile and I have the overlayfs option enabled. The overlay module is successfully loaded, and I have made the necessary additions for mount and umount to my sudoers file as per the wiki.

Last edited by n125 (2015-09-07 08:09:48)

Markus00000 · 2015-09-07 07:34:49

How to adjust the sync timer in version 6.00?

Someone marked that part on the PSD wiki page.

graysky · 2015-09-07 09:30:54

@n125 - I haven't seen that before nor do I understand why you would. Many services provided by systemd use it (see below). Does `journalctl --user-unit psd` give any clues?

/usr/lib/systemd/system/rsh@.service:3:After=local-fs.target
/usr/lib/systemd/system/systemd-journal-catalog-update.service:13:After=local-fs.target
/usr/lib/systemd/system/systemd-remount-fs.service:15:Before=local-fs-pre.target local-fs.target shutdown.target
/usr/lib/systemd/system/systemd-remount-fs.service:16:Wants=local-fs-pre.target
/usr/lib/systemd/system/systemd-quotacheck.service:13:Before=local-fs.target shutdown.target
/usr/lib/systemd/system/systemd-machine-id-commit.service:14:After=local-fs.target
/usr/lib/systemd/system/systemd-fsck@.service:13:After=%i.device systemd-fsck-root.service local-fs-pre.target
/usr/lib/systemd/system/systemd-tmpfiles-setup-dev.service:14:Before=sysinit.target local-fs-pre.target systemd-udevd.service shutdown.target
/usr/lib/systemd/system/systemd-update-done.service:13:After=local-fs.target
/usr/lib/systemd/system/cgmanager.service:5:After=local-fs.target
/usr/lib/systemd/system/sysinit.target:12:Wants=local-fs.target swap.target
/usr/lib/systemd/system/sysinit.target:13:After=local-fs.target swap.target emergency.service emergency.target
/usr/lib/systemd/system/tmp.mount:15:Before=local-fs.target umount.target
/usr/lib/systemd/system/systemd-tmpfiles-setup.service:13:After=local-fs.target systemd-sysusers.service
/usr/lib/systemd/system/systemd-fsck-root.service:12:Before=local-fs.target shutdown.target
/usr/lib/systemd/system/quotaon.service:13:Before=local-fs.target shutdown.target
/usr/lib/systemd/system/systemd-hibernate-resume@.service:13:Wants=local-fs-pre.target
/usr/lib/systemd/system/systemd-hibernate-resume@.service:15:Before=local-fs-pre.target
/usr/lib/systemd/system/dm-event.service:6:Before=local-fs.target
/usr/lib/systemd/system/modprobed-db@.service:6:Wants=local-fs.target modprobed-db@.timer
/usr/lib/systemd/system/local-fs.target:13:After=local-fs-pre.target
/usr/lib/systemd/system/telnet@.service:3:After=local-fs.target
/usr/lib/systemd/system/systemd-tmpfiles-clean.service:13:After=local-fs.target time-sync.target
/usr/lib/systemd/system/rlogin@.service:3:After=local-fs.target

@Markus - Yes, I marked it because I don't know how 100% to do this with user timers. Was hoping someone who did would edit.

Markus00000 · 2015-09-07 09:58:32

@graysky, I missed that it was you who marked it. I tried this:

~/.config/systemd/user/psd.timer.d/frequency.conf
-------------------------------------------------
[Unit]
Description=Custom timer for Profile-sync-daemon
PartOf=psd.service

[Timer]
OnBootSec=
OnBootSec=1min
OnUnitActiveSec=
OnUnitActiveSec=1min

But it seems to only run once:

NEXT                          LEFT     LAST PASSED UNIT      ACTIVATES
Mon 2015-09-07 12:00:18 CEST  16s left n/a  n/a    psd.timer psd.service

NEXT                          LEFT   LAST                          PASSED  UNIT      ACTIVATES
Mon 2015-09-07 12:00:18 CEST  3s ago Mon 2015-09-07 12:00:21 CEST  1ms ago psd.timer psd.service

NEXT                          LEFT         LAST                          PASSED      UNIT      ACTIVATES
Mon 2015-09-07 12:00:18 CEST  1min 10s ago Mon 2015-09-07 12:00:21 CEST  1min 7s ago psd.timer psd.service

I noticed that NEXT also stays the same for the included timer (/usr/lib/systemd/user/psd.timer). This doesn’t seem to be right. Are you sure that the unit activates correctly so that OnUnitActiveSec= works?

Last edited by Markus00000 (2015-09-07 12:22:25)

n125 · 2015-09-07 13:15:59

@graysky, At first glance journalctl --user-unit psd seemed normal, but upon closer inspection, psd only seems to run only one time after starting. I don't know if this is related to that local-fs.target message in my journal, but it could be a separate issue.

-- Reboot --
Sep 07 03:25:54 arch-laptop systemd[1020]: Starting Profile-sync-daemon...
Sep 07 03:25:54 arch-laptop systemd[1020]: Started Profile-sync-daemon.
Sep 07 04:22:36 arch-laptop systemd[1020]: Started Profile-sync-daemon.

As you can see, after a reboot psd started, and one hour later it ran again as it should. As of this post, it is now a lot more than an hour after the last sync. When I look farther back in the log, this pattern persists since updating to 6.0x.

I can't really pinpoint anything wrong:

$ systemctl --user status -l psd.timer
● psd.timer - Timer for Profile-sync-daemon - 1Hour
   Loaded: loaded (/usr/lib/systemd/user/psd.timer; static; vendor preset: enabled)
   Active: active (running) since Mon 2015-09-07 03:25:54 PDT; 2h 34min ago

Sep 07 03:25:54 arch-laptop systemd[1020]: Started Timer for Profile-sync-daemon - 1Hour

All good here, and here:

$ systemctl --user status -l psd.service
● psd.service - Profile-sync-daemon
   Loaded: loaded (/usr/lib/systemd/user/psd.service; enabled; vendor preset: enabled)
   Active: active (exited) since Mon 2015-09-07 03:25:54 PDT; 2h 35min ago
     Docs: man:psd(1)
           man:profile-sync-daemon(1)
           https://wiki.archlinux.org/index.php/Profile-sync-daemon
 Main PID: 8191 (code=exited, status=0/SUCCESS)

Sep 07 03:25:54 arch-laptop systemd[1020]: Starting Profile-sync-daemon...
Sep 07 03:25:54 arch-laptop systemd[1020]: Started Profile-sync-daemon.
Sep 07 04:22:36 arch-laptop systemd[1020]: Started Profile-sync-daemon.

Good here too:

 $ psd p
Profile-sync-daemon v6.01 on Arch Linux

 Systemd service is currently active.
 Systemd timer is currently active.
 Overlayfs v23 is currently active.

Psd will manage the following per /home/anthony/.psd/.anthony@arch-laptop.pid.conf:

 browser/psname:  chromium/chromium
 owner/group id:  anthony/1000
 sync target:     /home/anthony/.config/chromium
 tmpfs dir:       /run/user/1000/anthony-chromium
 profile size:    97M
 overlayfs size:  13M
 recovery dirs:   none

graysky · 2015-09-07 16:43:45

Indeed the 6.00 and 6.01 timer did not get called. Fixed now in 6.02.
Note - You do not enable any timer. Just psd.service. Doing that will call the psd-resync.{service,timer}. Try it and report back here please.

Still not sure about your local-fs.target messages...

Last edited by graysky (2015-09-07 16:48:18)

Markus00000 · 2015-09-07 17:02:51

@graysky, you fixed it! I think this can be added to the wiki as an example:

~/.config/systemd/user/psd-resync.timer.d/frequency.conf
--------------------------------------------------------
[Unit]
Description=Timer for Profile-sync-daemon
PartOf=psd.service

[Timer]
OnBootSec=
OnBootSec=10m
OnUnitActiveSec=
OnUnitActiveSec=10m

graysky · 2015-09-07 18:53:33

@Markus - Thank you. Edited.

phoenix · 2015-09-07 19:28:56

can we have the option to run psd as a system service again?
it was much more convenient that way, with no need to modify sudoers or put 1777 on /tmp, potentially weakening security.

Markus00000 · 2015-09-07 19:31:10

@graysky, thanks. Interestingly, I added a comment to my pre-6.0.0 frequency.conf that the timer first had to be cleared like this:

[Timer]
OnUnitActiveSec=
OnUnitActiveSec=10m

Yet clearing seems to be unnecessary as I correctly get just the one timer from frequency.conf.

graysky · 2015-09-07 20:10:42

@phoenix - The only extra step is the optional and one-time modification of sudoers. There is no need to mess with tmpfs at all since the default is to use the secured $XDG_RUNTIME_DIR. No security weak link there. As well, psd should now support encrypted $HOME.

n125 · 2015-09-08 06:57:20

Alright, after updating to 6.02 the timer is properly looping and as far as I can tell, the program is operating as expected. Thank you. :) I still get the local-fs.target message in my journal, but only when psd.service is started for the first time, and it doesn't seem to affect anything.

I noticed that when stopping psd.service (systemctl --user stop psd), Chromium or whatever browser specified to sync is closed ungracefully. Is this expected behavior?

graysky · 2015-09-08 07:33:59

Still unsure what to make of the warning text
Yes, if you stop the daemon, the browsers must be force closed to guarantee profile integrity since we are moving a symlink to a dir. It is by design. See the 'kill_browsers' function in /usr/bin/psd for the code.

kozaki · 2015-09-09 13:27:38

Having updated from profile-sync-daemon pre-6 to v6.04-1, given my user the mount, umount rights, psd p is not happy with my /etc/sudoers.

EDIT: Works fine after I moved the sudoers' line for /usr/bin/{mount,umount} after the 'group' instructions :}

Last edited by kozaki (2015-09-09 14:28:38)

graysky · 2015-09-09 13:57:23

kozaki wrote:

EDIT: Works fine after I moved the suders' line for /usr/bin/{mount,umount} after the group instructions :} Don't bother with what follows!
...

@kozaki - Can you edit your post removing the rest of it since google hits these forums, we want to maximize the signal-to-noise ratio

kozaki · 2015-09-09 14:51:45

@Graysky: Done. And you're right the information till that suffices for other visudo stars as I was

BTW I'm benchmarking Firefox with & without tmpfs on a netbook, with a 5.4K HDD, 1GB RAM DDR2-800 atm and a 667 MHz FSB. Your work helps a lot Graysky Now I'm wondering why the little differences between the spinning and the ram storage. Eg where does the %iowait comes from when psd (and asd for Firefox's cache)?

A) Default, ie On Disk only I/O, first run after boot:

14:38:52        CPU     %user     %nice   %system   %iowait    %steal     %idle
14:38:57        all      3,21      0,00      0,30      0,00      0,00     96,49
14:39:02        all     11,11      0,00      1,01     41,52      0,00     46,36   # Choosing profile
14:39:07        all     33,13      0,00      2,12     14,14      0,00     50,61
14:39:12        all     37,36      0,10      1,51     19,13      0,00     41,89
14:39:17        all     33,80      0,00      0,81     21,19      0,00     44,20   # UI appears
14:39:22        all     41,21      0,00      1,31     18,18      0,00     39,29
14:39:27        all     50,25      0,00      1,21      2,22      0,00     46,32
14:39:32        all     65,93      0,00      2,43      9,50      0,00     22,14   # UI usable
14:39:37        all     75,98      0,00      2,62      0,20      0,00     21,19
14:39:42        all     83,97      0,00      2,52      0,50      0,00     13,00
14:39:47        all     96,88      0,00      3,12      0,00      0,00      0,00
14:39:52        all     87,81      0,20      3,22      2,11      0,00      6,65   # tabs available and active one loaded
14:39:57        all     17,69      0,00      0,70      0,80      0,00     80,80
Average:        all     49,09      0,02      1,76      9,95      0,00     39,17

32 sec to UI usable, 52 sec from launching to session restored, pffwoow!

B) Profile and Cache On tmpfs (PSD with overlayfs, ASD), first run after boot:

16:22:47        CPU     %user     %nice   %system   %iowait    %steal     %idle
16:22:52        all      3,42      0,00      0,50     11,17      0,00     84,91
16:22:57        all     15,76      0,10      1,11     36,46      0,00     46,57   # Choosing profile
16:23:02        all     40,93      0,00      2,32     13,61      0,00     43,15
16:23:07        all     37,41      0,00      1,11     20,73      0,00     40,75   # UI appears
16:23:12        all     38,22      0,30      1,31     17,80      0,00     42,37
16:23:17        all     50,00      0,00      1,11      1,21      0,00     47,69
16:23:22        all     47,17      0,00      1,72      8,91      0,00     42,21   # Ui usable
16:23:27        all     91,54      0,00      3,22      1,31      0,00      3,93
16:23:32        all     73,08      0,00      1,51      0,60      0,00     24,80
16:23:37        all     96,88      0,00      3,12      0,00      0,00      0,00
16:23:42        all     95,27      0,00      3,63      0,30      0,00      0,81   # tabs available and active one loaded
16:23:47        all     26,83      0,00      1,51      0,10      0,00     71,56
^C
Average:        all     49,34      0,03      1,77      8,88      0,00     39,98

25 sec to UI usable, 45 sec from launching to session restored First run. A moderate 9% decreased time vs no PSD.

What may be the bottleneck? Between "UI appears" (it's all grey) and "is usable", you'll see both Disk and CPU aren't even maxed out (respectively %iowait and %idle columns). After that when restoring the session, seems than that low power CPU is the main bottleneck (not the 5.4K HDD).

EDIT: (+) using overlayfs, precisions on the benchmark result.

Last edited by kozaki (2015-09-09 23:19:24)

jean_no · 2015-09-09 15:34:34

Hi

Can you correct the psd.install file.
Line 11: $ HOME / .psd / psd.conf
Line 35: ~ / .psd / psd.conf
and line 6 (typo): byt => but

Thank you for your work

graysky · 2015-09-09 17:37:08

@kozaki - Not sure. I would think the speed advantages might come from calls to the profile history or the like. Also recommend you try the overlayfs option to decrease startup times.
@jean_no - Thanks fixed.

kozaki · 2015-09-09 23:24:04

Thanks Graysky; already using overlayfs Also the question if I can refine is: Why nearly as much I/O with and without PSD & ASD?

9tales · 2015-09-10 17:16:19

Hi,

My browser(s) won't start after a reboot, as they're symlinked to a location ( /run/user/1000/ap-chromium ) that has nothing. The profiles are stored with a -backup prefix in the config folders. I have to manually remove the symlink and rename the folder, post which the browser opens correctly.

Relevant outputs :

 % psd p
Profile-sync-daemon v6.04 on Arch Linux

 Systemd service is currently active.
 Systemd resync-timer is currently active.
 Overlayfs v23 is currently active.

Psd will manage the following per /home/ap/.config/psd/.psd.conf:

 browser/psname:  chromium/chromium
 owner/group id:  ap/100
 sync target:     /home/ap/.config/chromium
 tmpfs dir:       /run/user/1000/ap-chromium
 profile size:    489M
 overlayfs size:
 recovery dirs:   none

relevant part from `journalctl --user`

Sep 10 20:19:33 lip systemd[403]: local-fs.target: Cannot add dependency job, ignoring: Unit local-fs.target failed to load: No such file or directory.
Sep 10 20:19:33 lip systemd[403]: Reached target Sockets.
Sep 10 20:19:33 lip systemd[403]: Reached target Paths.
Sep 10 20:19:33 lip systemd[403]: Started Timer for profile-sync-daemon - 1Hour.
Sep 10 20:19:33 lip systemd[403]: Created slice -.slice.
Sep 10 20:19:33 lip systemd[403]: Reached target Timers.
Sep 10 20:19:33 lip systemd[403]: Reached target Basic System.
Sep 10 20:19:33 lip systemd[403]: Starting Profile-sync-daemon...
Sep 10 20:19:33 lip systemd[403]: Started Profile-sync-daemon.
Sep 10 20:19:33 lip systemd[403]: Reached target Default.
Sep 10 20:19:33 lip systemd[403]: Starting Timed resync...
Sep 10 20:19:33 lip profile-sync-daemon[418]: /home/ap/.config/chromium does not exist or is a broken symlink! Is /run/user/1000 unmounted?
Sep 10 20:19:33 lip systemd[403]: psd-resync.service: Main process exited, code=exited, status=1/FAILURE
Sep 10 20:19:33 lip systemd[403]: Failed to start Timed resync.
Sep 10 20:19:33 lip systemd[403]: Startup finished in 1.007s.
Sep 10 20:19:33 lip systemd[403]: psd-resync.service: Unit entered failed state.
Sep 10 20:19:33 lip systemd[403]: psd-resync.service: Failed with result 'exit-code'.
Sep 10 20:19:46 lip systemd-coredump[496]: Process 1 (systemd) of user 1000 dumped core.
Sep 10 20:19:50 lip systemd-coredump[516]: Process 1 (systemd) of user 1000 dumped core.

~/.config/psd/psd.conf

USE_OVERLAYFS="yes"
BROWSERS="chromium"
USE_BACKUPS="yes"

etc/modules-load.d/load_me.conf

overlay

/etc/sudoers (last line of the file)

ap ALL=NOPASSWD: /usr/bin/mount,/usr/bin/umount

systemctl status

 % systemctl --user status psd
● psd.service - Profile-sync-daemon
   Loaded: loaded (/usr/lib/systemd/user/psd.service; enabled; vendor preset: enabled)
   Active: active (exited) since Thu 2015-09-10 20:19:33 IST; 2h 15min ago
     Docs: man:psd(1)
           man:profile-sync-daemon(1)
           https://wiki.archlinux.org/index.php/Profile-sync-daemon
  Process: 407 ExecStart=/usr/bin/profile-sync-daemon sync (code=exited, status=0/SUCCESS)
 Main PID: 407 (code=exited, status=0/SUCCESS)
   CGroup: /user.slice/user-1000.slice/user@1000.service/psd.service

Sep 10 20:19:33 lip systemd[403]: Starting Profile-sync-daemon...
Sep 10 20:19:33 lip systemd[403]: Started Profile-sync-daemon.

The problem persists even when I've disabled overlayfs support.

I've also run `systemctl --user enable psd` before, so service starts on boot.

What am I missing here?

Last edited by 9tales (2015-09-10 17:19:32)

graysky · 2015-09-10 19:20:07

@9tales - The core dump is something that psd shouldn't do since it is just rsyncing files... in reality psd is just a glorified rsync wrapper script. How much memory does your machine have? I see your profile is about 500 MB.

Arch Linux

#601 2015-09-06 20:33:15

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#602 2015-09-06 20:44:09

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#603 2015-09-06 20:57:59

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#604 2015-09-06 21:09:08

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#605 2015-09-07 02:20:59

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#606 2015-09-07 07:34:49

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#607 2015-09-07 09:30:54

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#608 2015-09-07 09:58:32

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#609 2015-09-07 13:15:59

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#610 2015-09-07 16:43:45

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#611 2015-09-07 17:02:51

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#612 2015-09-07 18:53:33

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#613 2015-09-07 19:28:56

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#614 2015-09-07 19:31:10

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#615 2015-09-07 20:10:42

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#616 2015-09-08 06:57:20

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#617 2015-09-08 07:33:59

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#618 2015-09-09 13:27:38

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#619 2015-09-09 13:57:23

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#620 2015-09-09 14:51:45

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#621 2015-09-09 15:34:34

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#622 2015-09-09 17:37:08

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#623 2015-09-09 23:24:04

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#624 2015-09-10 17:16:19

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#625 2015-09-10 19:20:07

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

Board footer