profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

9tales · 2015-09-10 19:31:37

4 GB. More than 2 free (from free -m).

graysky · 2015-09-10 19:55:26

I can't figure what would cause a core dump... anything useful in `systemctl --user status -l psd`

9tales · 2015-09-10 21:38:29

Nope. Same as the one my original post.

You think it might be an issue with using overlay? Also, is my chromium profile abnormally large? I've tried using profile-cleaner from the AUR to cut it down to size, but it never seems to reduce it by much.

At shutdown, there's a stray "[FAILED] failed to unmount ..." message. Don't know where to look for that particular message. journalctl should have it, but I can't seem to find it.

graysky · 2015-09-11 18:49:30

9tales wrote:

Nope. Same as the one my original post.
You think it might be an issue with using overlay? Also, is my chromium profile abnormally large? I've tried using profile-cleaner from the AUR to cut it down to size, but it never seems to reduce it by much.
At shutdown, there's a stray "[FAILED] failed to unmount ..." message. Don't know where to look for that particular message. journalctl should have it, but I can't seem to find it.

@9tales - Mine is around 112 M. Didn't you say that you experience the same problem without overlay enabled? Try disabling it and rebooting. Does it sync on start?

klenamenis · 2015-09-11 20:52:52

Upgrading from 5.97 to 6.05 broke psd complaining about missing /usr/share/doc/psd so I completely removed psd from my system and reinstalled it again. Now I get "ls: cannot access /usr/share/doc/psd/browsers/: No such file or directory" everytime running "psd parse". My config (moved from /etc/ to ~/.config/psd/) has been working all the time, I'm using Icecat on Parabola here. Sudo rights have been granted as suggested, any hints on what's going wrong/why /usr/share/doc/psd doesn't exist on my system after a fresh install of psd?

graysky · 2015-09-11 21:07:29

@klenamenis - It is part of the package for sure. Do you run anything that deletes /usr/share/docs by chance?

EDIT: Fixed the $DOCDIR check in 6.06-1 but again, that is not what's wrong with your system's install.

Plz update to 6.06 and try again. If it works, again I would ask if you are running anything that would delete /usr/share/docs

Last edited by graysky (2015-09-11 21:16:26)

klenamenis · 2015-09-11 21:18:06

@graysky - Oh, indeed. I forgot to remove "!docs" again from OPTIONS= in /etc/makepkg.conf after having it set yesterday for testing purposes. My fault.

graysky · 2015-09-11 21:28:59

Ah, I was unaware of that makepkg option. Perhaps I should rethink using /usr/share/docs/psd as the dir for these shared files.

nasedo · 2015-09-17 07:10:44

I started having some problems after the transition to user units. I didn't notice it at first because it's not happening every time.

I have linger enabled for my user (loginctl enable-linger). What it means is that my user manager and services are started at boot and kept until the system shuts down. The consequence of that is that psd not always can sync back last changes and sometimes even leaves me with an empty profile (with no crash recovery backups, even though I didn't disable them).

Another thing, is local-fs.target in Wants really needed? AFAICT there is no local-fs.target for user managers. Similar situation with RequiresMountsFor=/home/, I don't think a user can log in with his home directory not mounted.

graysky · 2015-09-17 08:20:48

nasedo wrote:

I started having some problems after the transition to user units. I didn't notice it at first because it's not happening every time.
I have linger enabled for my user (loginctl enable-linger). What it means is that my user manager and services are started at boot and kept until the system shuts down. The consequence of that is that psd not always can sync back last changes and sometimes even leaves me with an empty profile (with no crash recovery backups, even though I didn't disable them).
Another thing, is local-fs.target in Wants really needed? AFAICT there is no local-fs.target for user managers. Similar situation with RequiresMountsFor=/home/, I don't think a user can log in with his home directory not mounted.

A few questions: how large is your profile/profiles and are you using an overlayfs mount (post `psd p`)?

I don't know about the implications of a linger... seems that so long as systemd calls the service to run the execstop command all should be well. Can you post the output of `journalctl --user-init psd` from directly BEFORE one of these situations? Also, is the media-bound backup present in these situations? For chromium, it would be 'chromium-backup' which is different from a recovery snapshot which is only there if psd starts and the symlink is dead.

I can't remember why I had the local-fs target in there... let me investigate.

The RequiresMountsFor= is needed for users with encrypted home directories so that psd will wait to start for them to be decrypted and then stop when they get re-encrypted.

EDIT: OK. You discovered a bug which I just fixed in 6.10 relating to the crashrecovery check under version 6. Please try 6.10 which I just pushed to the AUR.

Last edited by graysky (2015-09-17 09:14:12)

flannelhead · 2015-09-18 08:12:32

Occasionally, I'm having this happen at shutdown:

syys 18 10:58:49 stroemsoe umount[2388]: umount: /run/user/1000: target is busy
syys 18 10:58:49 stroemsoe umount[2388]: (In some cases useful info about processes that
syys 18 10:58:49 stroemsoe umount[2388]: use the device is found by lsof(8) or fuser(1).)
syys 18 10:58:49 stroemsoe systemd[1]: run-user-1000.mount: Mount process exited, code=exited status=32
syys 18 10:58:49 stroemsoe systemd[1]: Failed unmounting /run/user/1000.
syys 18 10:58:49 stroemsoe sudo[2390]:   sakari : TTY=unknown ; PWD=/home/sakari ; USER=root ; COMMAND=/usr/sbin/psd-overlay-helper -v 23 -l /home/sakari/.mozilla/fir
syys 18 10:58:49 stroemsoe sudo[2390]: pam_unix(sudo:session): session opened for user root by (uid=0)
syys 18 10:58:49 stroemsoe mkinitcpio[2290]: ==> Build complete.
syys 18 10:58:49 stroemsoe systemd[1]: Started Generate shutdown-ramfs.
syys 18 10:58:49 stroemsoe sudo[2390]: pam_unix(sudo:session): session closed for user root
syys 18 10:58:49 stroemsoe profile-sync-daemon[2291]: Sync successful
syys 18 10:58:49 stroemsoe sudo[2444]:   sakari : TTY=unknown ; PWD=/home/sakari ; USER=root ; COMMAND=/usr/sbin/psd-overlay-helper -d /run/user/1000/sakari-firefox-u
syys 18 10:58:49 stroemsoe sudo[2444]: pam_unix(sudo:session): session opened for user root by (uid=0)
syys 18 10:58:49 stroemsoe sudo[2444]: pam_unix(sudo:session): session closed for user root
syys 18 10:58:49 stroemsoe profile-sync-daemon[2291]: Unsync successful
syys 18 10:58:49 stroemsoe sudo[2466]:   sakari : TTY=unknown ; PWD=/home/sakari ; USER=root ; COMMAND=/usr/sbin/psd-overlay-helper -d /run/user/1000/sakari-google-ch
syys 18 10:58:49 stroemsoe sudo[2466]: pam_unix(sudo:session): session opened for user root by (uid=0)
syys 18 10:58:49 stroemsoe sudo[2466]: pam_unix(sudo:session): session closed for user root
syys 18 10:58:49 stroemsoe profile-sync-daemon[2291]: Unsync successful
syys 18 10:58:49 stroemsoe profile-sync-daemon[2291]: Unsync successful
syys 18 10:58:49 stroemsoe systemd[379]: Stopped Profile-sync-daemon.

It seems systemd is trying to unmount /run/user/1000 too early. This happens on about every other shutdown.

Here's the output of `psd p` for reference:

 Systemd service is currently active.
 Systemd resync-timer is currently active.
 Overlayfs v23 is currently active.

Psd will manage the following per /home/sakari/.config/psd/psd.conf:

 browser/psname:  firefox/firefox
 owner/group id:  sakari/1000
 sync target:     /home/sakari/.mozilla/firefox/unad8r2o.default
 tmpfs dir:       /run/user/1000/sakari-firefox-unad8r2o.default
 profile size:    42M
 overlayfs size:  20M
 recovery dirs:   none

 browser/psname:  google-chrome/chrome
 owner/group id:  sakari/1000
 sync target:     /home/sakari/.config/google-chrome
 tmpfs dir:       /run/user/1000/sakari-google-chrome
 profile size:    2,3M
 overlayfs size:  0
 recovery dirs:   none

graysky · 2015-09-18 09:14:56

Does it go down cleanly, just delayed or do you lose data?

flannelhead · 2015-09-18 10:04:46

graysky wrote:

Does it go down cleanly, just delayed or do you lose data?

Haven't seen any data loss so far. Aside of the error, the shutdown process seems to be clean. Also, no noticeable delay due to the error.

Edit: quite predictably, adding a drop-in file for psd containing

[Unit]
RequiresMountsFor=/home/ $XDG_RUNTIME_DIR

seems to mitigate the errors, and as per journalctl logs from several shutdowns, /run/user/1000 is unmounted after psd.service has stopped.
Edit 2: I saw the error again, with the above fix in place. Still no data loss.

Last edited by flannelhead (2015-09-18 13:29:09)

graysky · 2015-09-18 18:51:35

@flannelhead - That is unusual. I don't understand the timing of what looks like mkinitcpio output with the same timestamp. DId you run an update while shutting down or...??? In any case, no data loss is a good thing.

graysky · 2015-09-18 18:58:21

@flannelhead - That is unusual. I don't understand the timing of what looks like mkinitcpio output with the same timestamp. DId you run an update while shutting down or...??? In any case, no data loss is a good thing.

Try this:
1) Stop psd
2) As root, run `sed -i 's/ local-fs.target//' /usr/lib/systemd/user/psd.service`
3) Run as your user `systemctl --user daemon-reload`
4) Inspect your firefox and chrome profiles looking for a recursive symlink target as described here and delete it if found
5) Restart psd and see if it goes down cleanly

flannelhead · 2015-09-18 20:10:02

graysky wrote:

I don't understand the timing of what looks like mkinitcpio output with the same timestamp. DId you run an update while shutting down or...???

Good catch, that indeed seems suspicious. I didn't notice it myself before. However, it indeed seems to be normal behaviour and related to this sequence of events:

syys 18 22:53:55 stroemsoe mkinitcpio[8665]: ==> Starting build: none
syys 18 22:53:55 stroemsoe mkinitcpio[8665]: -> Running build hook: [sd-shutdown]
syys 18 22:53:55 stroemsoe mkinitcpio[8665]: ==> Build complete.

So mkinitcpio just builds an initramfs for systemd to be used for shutdown.

graysky wrote:

Try this:
1) Stop psd
2) As root, run `sed -i 's/ local-fs.target//' /usr/lib/systemd/user/psd.service`
3) Run as your user `systemctl --user daemon-reload`
4) Inspect your firefox and chrome profiles looking for a recursive symlink target as described here and delete it if found
5) Restart psd and see if it goes down cleanly

Thanks, I did those fixes. No recursive symlinks were present. When the system is running and psd is restarted, everything seems to go very cleanly. On shutdown, the same error appeared again. It seems to me that systemd just tries to unmount /run/user/1000 too early. Fortunately that doesn't seem to cause data losses as the tmpfs isn't unmounted due to the error, and psd has the full opportunity to sync the profiles back to disk.

Perhaps RequiresMountsFor could be used in the unit file to indicate that the mount is needed? The drop-in in my previous post only seemed to cause systemd parse errors (it didn't seem to understand $XDG_RUNTIME_DIR). However the following in $HOME/.config/systemd/user/psd.service.d/override.conf seems to do the trick:

[Unit]
RequiresMountsFor=/home/ /run/user/1000/

Tested by rebooting a few times. Now /run/user/1000 seems to be unmounted cleanly after psd has shut down.
Edit: this still doesn't seem to work??

Last edited by flannelhead (2015-09-18 21:46:32)

graysky · 2015-09-18 21:22:59

@flannelhead - I dunno how to get a service file to source env vars like $XDG_RUNTIME_DIR which is how it should be done I think...

flannelhead · 2015-09-18 21:29:13

graysky wrote:

@flannelhead - I dunno how to get a service file to source env vars like $XDG_RUNTIME_DIR which is how it should be done I think...

Yeah, I couldn't figure it out either. Given that the location of the tmpfs directory actually depends on the user's settings in psd.conf, that might be even harder to implement. Fortunately the unmount error doesn't seem to cause any data loss. I could add a section to the wiki page about how it can be manually circumvented like I did.

Edit: got the error once again, despite the latest fix.

Last edited by flannelhead (2015-09-18 21:47:19)

graysky · 2015-09-20 14:04:20

Just to update users, flannelhead has discovered some open issues against systemd that might explain his recent posts. We are troubleshooting.
References and related non-psd issues:
https://github.com/systemd/systemd/issues/867
https://bbs.archlinux.org/viewtopic.php?pid=1554049
https://bbs.archlinux.org/viewtopic.php?id=200469

hatuey · 2015-10-07 14:31:18

I've got the same problem (I think).

If I shutdown the system without stopping the user service first, psd doesn't sync back the profile to my home folder and I lose any change I made in the previous session.

If I run "systemctl --user stop psd.service" before shutting down, it works as intended.

I'm using psd with overlays enabled. For now, I've solved with a script that stops the service before running systemctl poweroff, but I don't like it.

dragonkore · 2015-10-14 19:00:14

hatuey wrote:

I've got the same problem (I think).
If I shutdown the system without stopping the user service first, psd doesn't sync back the profile to my home folder and I lose any change I made in the previous session.
If I run "systemctl --user stop psd.service" before shutting down, it works as intended.
I'm using psd with overlays enabled. For now, I've solved with a script that stops the service before running systemctl poweroff, but I don't like it.

This worked for me but I also don't like it. It causes a noticeable delay in shutdown while PSD is being stopped.

graysky · 2015-10-14 19:02:59

Several users have reported that systemd-227 fixes this bug (it is upstream, not with psd). Have either of you tried it?

k44 · 2015-10-15 20:20:04

Hey,

On a freshly installed Arch, I've got an issue with psd that I can't figure out. The systemd user service is not starting on his own. I don't have any messages in journalctl

[dalexandre@Ohara ~]$ systemctl --user status -l psd
● psd.service - Profile-sync-daemon
   Loaded: loaded (/usr/lib/systemd/user/psd.service; enabled; vendor preset: enabled)
   Active: inactive (dead)
     Docs: man:psd(1)
           man:profile-sync-daemon(1)
           https://wiki.archlinux.org/index.php/Profile-sync-daemon

It's not really an issue with psd because I can start it manually without any issues

[dalexandre@Ohara ~]$ systemctl --user start psd
[dalexandre@Ohara ~]$ systemctl --user status -l psd
● psd.service - Profile-sync-daemon
   Loaded: loaded (/usr/lib/systemd/user/psd.service; enabled; vendor preset: enabled)
   Active: active (exited) since jeu. 2015-10-15 22:13:22 CEST; 6s ago
     Docs: man:psd(1)
           man:profile-sync-daemon(1)
           https://wiki.archlinux.org/index.php/Profile-sync-daemon
  Process: 6465 ExecStart=/bin/true (code=exited, status=0/SUCCESS)
 Main PID: 6465 (code=exited, status=0/SUCCESS)
   CGroup: /user.slice/user-1000.slice/user@1000.service/psd.service

oct. 15 22:13:22 Ohara systemd[983]: Starting Profile-sync-daemon...
oct. 15 22:13:22 Ohara systemd[983]: Started Profile-sync-daemon.

graysky · 2015-10-16 23:04:14

Post the output of

find $HOME/.config/systemd/user -mindepth 1 -type d | xargs ls -gG --color

dragonkore · 2015-10-17 05:09:37

graysky wrote:

Several users have reported that systemd-227 fixes this bug (it is upstream, not with psd). Have either of you tried it?

Just updated, seems to be working fine now. Thanks!

Arch Linux

#626 2015-09-10 19:31:37

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#627 2015-09-10 19:55:26

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#628 2015-09-10 21:38:29

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#629 2015-09-11 18:49:30

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#630 2015-09-11 20:52:52

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#631 2015-09-11 21:07:29

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#632 2015-09-11 21:18:06

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#633 2015-09-11 21:28:59

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#634 2015-09-17 07:10:44

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#635 2015-09-17 08:20:48

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#636 2015-09-18 08:12:32

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#637 2015-09-18 09:14:56

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#638 2015-09-18 10:04:46

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#639 2015-09-18 18:51:35

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#640 2015-09-18 18:58:21

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#641 2015-09-18 20:10:02

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#642 2015-09-18 21:22:59

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#643 2015-09-18 21:29:13

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#644 2015-09-20 14:04:20

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#645 2015-10-07 14:31:18

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#646 2015-10-14 19:00:14

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#647 2015-10-14 19:02:59

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#648 2015-10-15 20:20:04

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#649 2015-10-16 23:04:14

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#650 2015-10-17 05:09:37

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

Board footer