profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

HansAnderson · 2016-05-12 21:16:44

Today after update i had the same issue as geo909
consider this https://bbs.archlinux.org/viewtopic.php?id=204965
I remove "default.target.wants" and "psd-resync.timer.d"
from .config/systemd/user and all work fine now (boot and psd "systemctl --user start psd).
But how to start psd auto now?
if i adding "default.target.wants/psd.service" in .config/systemd/user then have empty $XDG_RUNTIME_ID and

there's the delay problem and lots of stuff are messed up

graysky · 2016-05-12 22:11:14

What is the output of:

find $HOME/.config/systemd/user -mindepth 1 -type d | xargs ls -gG --color

HansAnderson · 2016-05-13 11:07:31

Same as

 [user@user ~]$ls -l

Last edited by HansAnderson (2016-05-13 11:11:55)

graysky · 2016-05-13 13:15:12

HansAnderson wrote:

Same as

 [user@user ~]$ls -l

...and that output would be...?

HansAnderson · 2016-05-14 00:09:12

Ok, you catch me.

$ find $HOME/.config/systemd/user -mindepth 1 -type d | xargs ls -gG --color
total 0
drwxr-xr-x 1    78 Apr 17 02:48 abs
drwxr-xr-x 1    42 May 22  2014 aur
drwxr-xr-x 1  1978 May 13 15:07 Desktop
drwxr-xr-x 1  7732 Feb 21  2009 dic
drwxr-xr-x 1    48 Oct 20  2014 Documents
drwxr-xr-x 1 27004 May 13 00:33 Downloads
drwxr-xr-x 1  3446 Apr 22 20:03 Music
drwxr-xr-x 1  1648 Feb  2 17:30 Pictures
drwxr-xr-x 1     0 Jan  4  2014 Public
drwxr-xr-x 1     0 Jan  4  2014 Templates
drwxr-xr-x 1    62 Jul 30  2015 tests
drwxr-xr-x 1   980 May 12 02:02 Videos
drwxrwxr-x 1   190 Mar 15 18:12 xerox-phaser-3040
drwxr-xr-x 1     10230 May 11 2008 03:03 XXX

graysky · 2016-05-14 01:07:24

HansAnderson wrote:

Ok, you catch me.

$ find $HOME/.config/systemd/user -mindepth 1 -type d | xargs ls -gG --color
total 0
drwxr-xr-x 1    78 Apr 17 02:48 abs
drwxr-xr-x 1    42 May 22  2014 aur
drwxr-xr-x 1  1978 May 13 15:07 Desktop
drwxr-xr-x 1  7732 Feb 21  2009 dic
drwxr-xr-x 1    48 Oct 20  2014 Documents
drwxr-xr-x 1 27004 May 13 00:33 Downloads
drwxr-xr-x 1  3446 Apr 22 20:03 Music
drwxr-xr-x 1  1648 Feb  2 17:30 Pictures
drwxr-xr-x 1     0 Jan  4  2014 Public
drwxr-xr-x 1     0 Jan  4  2014 Templates
drwxr-xr-x 1    62 Jul 30  2015 tests
drwxr-xr-x 1   980 May 12 02:02 Videos
drwxrwxr-x 1   190 Mar 15 18:12 xerox-phaser-3040
drwxr-xr-x 1     10230 May 11 2008 03:03 XXX

For psd to autostart properly, you should just have psd.service in there. No idea why in the world you would populate that dir with the contents you posted.

HansAnderson · 2016-05-14 15:01:23

Thank you for help
It was my mistake

graysky · 2016-05-14 15:38:27

You're welcome. You shouldn't need to manually put anything in ~/.config/systemd/user ... `systemctl --user action target` should do it all.

HansAnderson · 2016-05-17 20:46:57

Now again, all the time overlayfs size: 0
and grows rapidly profile size

 psd p
Profile-sync-daemon v6.21 on Arch Linux

 Systemd service is currently active.
 Systemd resync-timer is currently active.
 Overlayfs v23 is currently active.

Psd will manage the following per /home/user/.config/psd/.psd.conf:

 browser/psname:  chromium/chromium
 owner/group id:  suer/1002
 sync target:     /home/user/.config/chromium
 tmpfs dir:       /run/user/1002/user-chromium
 profile size:    463M
 overlayfs size:  0
 recovery dirs:   none

systemctl --user status psd
● psd.service - Profile-sync-daemon
   Loaded: loaded (/usr/lib/systemd/user/psd.service; enabled; vendor preset: enabled)
   Active: active (exited) since Tue 2016-05-17 23:38:13 MSK; 6min ago
     Docs: man:psd(1)
           man:profile-sync-daemon(1)
           https://wiki.archlinux.org/index.php/Profile-sync-daemon
  Process: 1150 ExecStart=/bin/true (code=exited, status=0/SUCCESS)
 Main PID: 1150 (code=exited, status=0/SUCCESS)
   CGroup: /user.slice/user-1002.slice/user@1002.service/psd.service

May 17 23:38:13 user systemd[1144]: Starting Profile-sync-daemon...
May 17 23:38:13 user systemd[1144]: Started Profile-sync-daemon.

Last edited by HansAnderson (2016-05-17 20:49:30)

graysky · 2016-05-17 21:01:03

Yes, initially the size is 0. As data are changed by the browser, the size grows..

 profile size:    86M
 overlayfs size:  0

 <<time passes as I browse>>

 profile size:    86M
 overlayfs size:  27M

 <<time passes as I browse>>

 profile size:    86M
 overlayfs size:  56M

 <<time passes as I browse>>

HansAnderson · 2016-05-17 23:28:54

it was before, the size grows
And now overlayfs size: 0

  psd p
Profile-sync-daemon v6.21 on Arch Linux

 Systemd service is currently active.
 Systemd resync-timer is currently active.
 Overlayfs v23 is currently active.

Psd will manage the following per /home/user/.config/psd/.psd.conf:

 browser/psname:  chromium/chromium
 owner/group id:  user/1002
 sync target:     /home/user/.config/chromium
 tmpfs dir:       /run/user/1002/user-chromium
 profile size:    466M
 overlayfs size:  0
 recovery dirs:   none

It was some sort of confusion with directories, after last update.

Profile-sync-daemon v6.21 on Arch Linux

 Systemd service is currently active.
 Systemd resync-timer is currently active.
 Overlayfs v23 is currently active.

Psd will manage the following per /home/user/.config/psd/.psd.conf:

 browser/psname:  chromium/chromium
 owner/group id:  user/1002
 sync target:     /home/user/.config/chromium
 tmpfs dir:       /run/user/1002/user-chromium
 profile size:    1.4G
 overlayfs size:  39M
 recovery dirs:   1 <- delete with the c option
  dir path/size:  /home/usr/.config/chromium-backup-crashrecovery-20160518_030002 (1.9G)

Last edited by HansAnderson (2016-05-18 23:12:29)

OdinEidolon · 2016-05-27 08:11:30

I use PSD with chromium, and I'm having some issues. My laptop is affected by a bumblebee issue which sometimes causes what looks like a kernel panic when shutting down, due to the fact that sometimes the nvidia module cannot be unloaded for some reason. When this happens, I lose my chromium profile completely (it says it's corrupted), as well as the most recent backup-crashrecovery.

How can this be fixed? A (terrible) workaround would be to always keep several different crashrecoveries, let's say one for each day, and rotate them. This way, even if I lose my profile, the worst thing that may happen is that I lose a single day of profile modifications.

graysky · 2016-05-27 12:41:47

@Odin - Without knowing more details as to how your system is panicing on shutdown, I don't know what can be done. Are you certain that the crashrecover dir is bad? That doesn't make sense since it a snapshot before the sync. Are you rotating the crash recovery back into place per the man page?

OdinEidolon · 2016-05-27 12:56:09

graysky wrote:

@Odin - Without knowing more details as to how your system is panicing on shutdown, I don't know what can be done. Are you certain that the crashrecover dir is bad? That doesn't make sense since it a snapshot before the sync. Are you rotating the crash recovery back into place per the man page?

Thanks.
Yep, I do rotate it as the wiki says. However even using the rotated crashrecovery, it says it is corrupted. I have to rollback the penultimate backup or a manual backup.

Unfortunately it's a random crash which is difficult to reproduce and when it does crash, I have no idea where I can find logs. If you have any suggestion about that, I'll try to track it down next time it happens.

openprivacy · 2016-06-05 19:31:09

PSD comes highly recommended and looks like a fantastic tool - thank you! - but it does not seem to be working for me as my google-chrome browser is now slow and laggy. PSD seems to run once on startup then die. I've been searching forums, etc. and have not found the answer - apologies if a known question/issue. I attempt to offer all asked in other posts below.

Running 4.5.4-1-ARCH updated weekly and here's my config:

» sed '/^#/d' ~/.config/psd/psd.conf

USE_OVERLAYFS="yes"

BROWSERS="firefox google-chrome"

psd parse looks good:

» psd p
Profile-sync-daemon v6.22 on Arch Linux

 Systemd service is currently active.
 Systemd resync-timer is currently active.
 Overlayfs v23 is currently active.

Psd will manage the following per /home/fen/.config/psd/.psd.conf:

[[ ... two firefox stanzas elided ... ]]

 browser/psname:  google-chrome/chrome
 owner/group id:  fen/1000
 sync target:     /home/fen/.config/google-chrome
 tmpfs dir:       /run/user/1000/fen-google-chrome
 profile size:    463M
 overlayfs size:  82M
 recovery dirs:   2 <- delete with the c option
  dir path/size:  /home/fen/.config/google-chrome-backup-crashrecovery-20160531_171130 (458M)
  dir path/size:  /home/fen/.config/google-chrome-backup-crashrecovery-20160604_124221 (460M)

...but the daemon appears to die (PID 12356 doesn't exist):

» systemctl --user status psd
● psd.service - Profile-sync-daemon
   Loaded: loaded (/usr/lib/systemd/user/psd.service; enabled; vendor preset: enabled)
   Active: active (exited) since Sun 2016-06-05 14:40:25 EDT; 10min ago
     Docs: man:psd(1)
           man:profile-sync-daemon(1)
           https://wiki.archlinux.org/index.php/Profile-sync-daemon
  Process: 12356 ExecStart=/bin/true (code=exited, status=0/SUCCESS)
 Main PID: 12356 (code=exited, status=0/SUCCESS)
   CGroup: /user.slice/user-1000.slice/user@1000.service/psd.service

Jun 05 14:40:25 truckin systemd[822]: Starting Profile-sync-daemon...
Jun 05 14:40:25 truckin systemd[822]: Started Profile-sync-daemon.

...and no timers are set:

» systemctl list-timers --all
NEXT                         LEFT     LAST                         PASSED  UNIT                         
Mon 2016-06-06 00:00:00 EDT  9h left  Sun 2016-06-05 00:00:07 EDT  14h ago logrotate.timer              
Mon 2016-06-06 00:00:00 EDT  9h left  Sun 2016-06-05 00:00:07 EDT  14h ago shadow.timer                 
Mon 2016-06-06 08:49:02 EDT  17h left Sun 2016-06-05 00:33:58 EDT  14h ago systemd-tmpfiles-clean.timer

The file `/run/user/1000/psd.pid` exists but is empty. I get the NOTIFY_SOCKET warning (have googled this and have not found the solution):

» /usr/bin/profile-sync-daemon resync
No status data could be sent: $NOTIFY_SOCKET was not set
firefox resync successful
firefox resync successful
google-chrome resync successful

And as it may be helpful, here's the status of the psd* services:

» systemctl --user list-unit-files | grep psd
psd-resync.service                  enabled 
psd.service                         enabled 
psd-resync.timer                    static

Again, I appreciate all the work you do and I hope that the answer to my question will be easy. Thanks!

graysky · 2016-06-05 19:40:44

@openp - As an aside, your 400-500 MB profile is kind big (shouldn't matter so long as you have enough RAM). Note that there is no PID since psd is just a glorified rsync script run by a systemd timer. Also you're checking system timers, not user timers. Try this:

systemctl --user list-timers --all

As to why you're "slow and lagged" I am not sure... how much memory do you have on the machine and how much is being allocated to your tmpfs? You can run the following to see:

df -h | grep fen

openprivacy · 2016-06-05 19:55:56

Wow! Quick reply - thanks! You're right, the timer looks good:

» systemctl --user list-timers --all
NEXT                         LEFT       LAST                         PASSED       UNIT             ACTIV
Sun 2016-06-05 16:41:02 EDT  56min left Sun 2016-06-05 15:41:02 EDT  3min 41s ago psd-resync.timer psd-r

1 timers listed.

I have mucho RAM (16G) with 20% (3.2G) in tmpfs on a Lenovo T530 w/ i7-3720QM CPU @ 2.60GHz

overlaid        3.2G  102M  3.1G   4% /run/user/1000/fen-google-chrome
overlaid        3.2G  102M  3.1G   4% /run/user/1000/fen-firefox-muc59vd2.default
overlaid        3.2G  102M  3.1G   4% /run/user/1000/fen-firefox-031jbrh4.dev-edition-default

Very noticeable delay when switching windows. Could it be due to this google-chrome-back-ovfs directory?:

» ls -ld ~/.config/google-*
lrwxrwxrwx  1 fen fen   32 Jun  5 14:40 /home/fen/.config/google-chrome -> /run/user/1000/fen-google-chrome
drwx------ 12 fen fen 4096 Jun  5 15:39 /home/fen/.config/google-chrome-back-ovfs
drwx------ 12 fen fen 4096 Jun  5 14:39 /home/fen/.config/google-chrome-backup
drwx------ 12 fen fen 4096 May 31 16:33 /home/fen/.config/google-chrome-backup-crashrecovery-20160531_171130
drwx------ 12 fen fen 4096 May 31 17:15 /home/fen/.config/google-chrome-backup-crashrecovery-20160604_124221
drwx------ 10 fen fen 4096 Apr  4 20:58 /home/fen/.config/google-chrome-beta
drwx------  2 fen fen 4096 Apr 18 12:45 /home/fen/.config/google-googletalkplugin

graysky · 2016-06-05 20:03:51

No, that is part of the overlayfs. As you try to troubleshoot, I recommend you enable one browser at a time.

1) Stop psd
2) Edit psd.conf to sync just google-chrome
3) Start psd

Still delayed?

openprivacy · 2016-06-05 20:18:23

Stopped psd (which also killed google-chrome browser).
Edited psd.conf to remove firefox.
Started psd.
Re-started google-chrome & waited for (three) windows to restore.

There's a 2-3 second delay from the time I click on a TAB to seeing that page render.

Opened Firefox for comparison - no delay at all when clicking between TABs.

openprivacy · 2016-06-05 20:42:40

If you can suggest ways for me to debug this, I'm open to suggestions. I checked basic r/w speed and my tmpfs appears to write twice as fast as my SSD but reads are ~20% slower:

Writing to/reading from tmpfs:

» dd if=/dev/zero of=/run/user/1000/zero bs=4k count=10000
40960000 bytes (41 MB, 39 MiB) copied, 0.0164638 s, 2.5 GB/s

» dd if=/run/user/1000/zero of=/dev/null bs=4k count=10000
40960000 bytes (41 MB, 39 MiB) copied, 0.0153783 s, 2.7 GB/s

Writing to/reading from SSD:

» dd if=/dev/zero of=/home/fen/zero bs=4k count=10000
40960000 bytes (41 MB, 39 MiB) copied, 0.0335325 s, 1.2 GB/s

» dd if=/home/fen/zero of=/dev/null bs=4k count=10000
40960000 bytes (41 MB, 39 MiB) copied, 0.0128328 s, 3.2 GB/s

graysky · 2016-06-05 22:19:51

Not sure why you're experiencing delays... are you certain that without psd running chrome is fine when switching tabs? If it is, try disabling the overlayfs mount and retesting it.

Last edited by graysky (2016-06-05 22:20:07)

OdinEidolon · 2016-06-06 07:37:21

openprivacy wrote:

If you can suggest ways for me to debug this, I'm open to suggestions. I checked basic r/w speed and my tmpfs appears to write twice as fast as my SSD but reads are ~20% slower:
Writing to/reading from SSD:
» dd if=/dev/zero of=/home/fen/zero bs=4k count=10000
40960000 bytes (41 MB, 39 MiB) copied, 0.0335325 s, 1.2 GB/s

» dd if=/home/fen/zero of=/dev/null bs=4k count=10000
40960000 bytes (41 MB, 39 MiB) copied, 0.0128328 s, 3.2 GB/s

Careful when using this method to benchmark stuff. You numbers are WAY out or your SSD is a RAID0 of modern NVMe drives, which I doubt it is on a t530.
https://wiki.archlinux.org/index.php/Be … ge_devices

Last edited by OdinEidolon (2016-06-06 07:37:39)

zaxdan69 · 2016-07-08 05:00:16

I recently installed psd and I have the same problem as @geo909.
When I enable the service I have a login delay about 30secs, I cannot shutdown/reboot/suspend without to prompt for authentication and the other symptoms mentioned above. I also mention that the system creates a ".psd.conf" file which is read only and doesn't use psd.conf. If I disable the service I have again a normal behaviour.

graysky · 2016-07-08 17:45:30

@zax - The .psd.conf construction is normal (see the man page).

1) Stop psd
2) Post the output of `psd p`

zaxdan69 · 2016-07-12 03:40:00

'psd p' output:

 browser/psname:  firefox/firefox
 owner/group id:  archuser/100
 sync target:     /home/archuser/.mozilla/firefox/g89ltmxv.default
 tmpfs dir:       /run/user/1000/archuser-firefox-g89ltmxv.default
 profile size:    17M
 recovery dirs:   13 <- delete with the c option
  dir path/size:  /home/archuser/.mozilla/firefox/g89ltmxv.default-backup-crashrecovery-20160707_143049 (17M)
  dir path/size:  /home/archuser/.mozilla/firefox/g89ltmxv.default-backup-crashrecovery-20160708_053324 (17M)
  dir path/size:  /home/archuser/.mozilla/firefox/g89ltmxv.default-backup-crashrecovery-20160708_053941 (17M)
  dir path/size:  /home/archuser/.mozilla/firefox/g89ltmxv.default-backup-crashrecovery-20160708_064338 (17M)
  dir path/size:  /home/archuser/.mozilla/firefox/g89ltmxv.default-backup-crashrecovery-20160708_064537 (17M)
  dir path/size:  /home/archuser/.mozilla/firefox/g89ltmxv.default-backup-crashrecovery-20160708_064927 (17M)
  dir path/size:  /home/archuser/.mozilla/firefox/g89ltmxv.default-backup-crashrecovery-20160708_065750 (17M)
  dir path/size:  /home/archuser/.mozilla/firefox/g89ltmxv.default-backup-crashrecovery-20160708_070608 (17M)
  dir path/size:  /home/archuser/.mozilla/firefox/g89ltmxv.default-backup-crashrecovery-20160708_074609 (17M)
  dir path/size:  /home/archuser/.mozilla/firefox/g89ltmxv.default-backup-crashrecovery-20160708_074729 (17M)
  dir path/size:  /home/archuser/.mozilla/firefox/g89ltmxv.default-backup-crashrecovery-20160708_074928 (17M)
  dir path/size:  /home/archuser/.mozilla/firefox/g89ltmxv.default-backup-crashrecovery-20160708_075136 (17M)
  dir path/size:  /home/archuser/.mozilla/firefox/g89ltmxv.default-backup-crashrecovery-20160708_083020 (17M)

 browser/psname:  google-chrome/chrome
 owner/group id:  archuser/100
 sync target:     /home/archuser/.config/google-chrome
 tmpfs dir:       /run/user/1000/archuser-google-chrome
 profile size:    232M
 recovery dirs:   13 <- delete with the c option
  dir path/size:  /home/archuser/.config/google-chrome-backup-crashrecovery-20160707_143049 (225M)
  dir path/size:  /home/archuser/.config/google-chrome-backup-crashrecovery-20160708_053323 (228M)
  dir path/size:  /home/archuser/.config/google-chrome-backup-crashrecovery-20160708_053940 (228M)
  dir path/size:  /home/archuser/.config/google-chrome-backup-crashrecovery-20160708_064338 (228M)
  dir path/size:  /home/archuser/.config/google-chrome-backup-crashrecovery-20160708_064536 (228M)
  dir path/size:  /home/archuser/.config/google-chrome-backup-crashrecovery-20160708_064926 (228M)
  dir path/size:  /home/archuser/.config/google-chrome-backup-crashrecovery-20160708_065749 (228M)
  dir path/size:  /home/archuser/.config/google-chrome-backup-crashrecovery-20160708_070607 (228M)
  dir path/size:  /home/archuser/.config/google-chrome-backup-crashrecovery-20160708_074541 (226M)
  dir path/size:  /home/archuser/.config/google-chrome-backup-crashrecovery-20160708_074732 (226M)
  dir path/size:  /home/archuser/.config/google-chrome-backup-crashrecovery-20160708_074931 (226M)
  dir path/size:  /home/archuser/.config/google-chrome-backup-crashrecovery-20160708_075136 (226M)
  dir path/size:  /home/archuser/.config/google-chrome-backup-crashrecovery-20160708_083020 (226M)

Right now I haven't enabled the service and I put "psd p" to autostart.

Arch Linux

#726 2016-05-12 21:16:44

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#727 2016-05-12 22:11:14

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#728 2016-05-13 11:07:31

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#729 2016-05-13 13:15:12

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#730 2016-05-14 00:09:12

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#731 2016-05-14 01:07:24

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#732 2016-05-14 15:01:23

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#733 2016-05-14 15:38:27

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#734 2016-05-17 20:46:57

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#735 2016-05-17 21:01:03

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#736 2016-05-17 23:28:54

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#737 2016-05-27 08:11:30

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#738 2016-05-27 12:41:47

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#739 2016-05-27 12:56:09

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#740 2016-06-05 19:31:09

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#741 2016-06-05 19:40:44

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#742 2016-06-05 19:55:56

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#743 2016-06-05 20:03:51

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#744 2016-06-05 20:18:23

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#745 2016-06-05 20:42:40

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#746 2016-06-05 22:19:51

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#747 2016-06-06 07:37:21

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#748 2016-07-08 05:00:16

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#749 2016-07-08 17:45:30

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#750 2016-07-12 03:40:00

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

Board footer