profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

graysky · 2013-03-18 20:14:56

nperry wrote:

I'm not to sure if this is because my /home partition is on a HDD and / is on an SSD.
When you say clearing cache on exit, is there a way to do this for chromium and Firefox, automated?

Like others said, depends on the size you're asking it to sync. Just out of curiosity, why not place /home on the SSD? Faster and more reliable.

For comparison:

% psd p
rofile-sync-daemon v5.31 on Arch Linux.

Daemon file /run/psd is present.
Service is currently active.

Psd will manage the following per /etc/psd.conf settings:

 browser/psname:  chromium/chromium
 owner/group:     graysky/users
 sync target:     /home/graysky/.config/chromium
 tmpfs dir:       /tmp/graysky-chromium
 profile size:    61M

% systemd-analyze blame | grep psd                                    
   473ms psd.service
    52ms psd-resync.service

nperry · 2013-03-19 08:53:21

/home is on a 1tb HDD with my swap and /var, my ssd is only 64gb on my main PC.

So I think my HDD is the problem, I may have to purchase a 256gb ssd for /home.

Thanks for all the replies

freestyler7 · 2013-03-21 06:40:39

I had to reset my computer (nothing but problems with the closed catalyst driver) and in the following boot, psd did not resync itself.
The unit was running but the profile didn't move to ram. A simple restart of the daemon did the trick.

I know resetting is not an expected behaviour so i'm only reporting this as a hint.

Last edited by freestyler7 (2013-03-21 06:41:12)

graysky · 2013-03-21 11:44:15

@freestyler7 - The reset shouldn't matter. Did systemctl throw any errors? Check your journal.

freestyler7 · 2013-03-21 12:02:31

Nope, no errors. Before restarting the unit i moved and edited the .pacnew conf an you can see (which didn't have any differences anyway).
.timer just delays the start of the daemon to not trash my hard disk on boot. (when the event timer is elapsed, systemd starts the daemon instantly on the following manually starts)
But anyway, it's no problem at all. This is the only time it happened and it was after a reset.

Mar 21 08:25:37 archlinux sudo[1097]: free : TTY=pts/0 ; PWD=/home/free ; USER=root ; COMMAND=/usr/bin/mv /etc/psd.conf.pacnew /etc/psd.conf
Mar 21 08:25:42 archlinux sudo[1107]: free : TTY=pts/0 ; PWD=/home/free ; USER=root ; COMMAND=/usr/bin/nano /etc/psd.conf
Mar 21 08:26:24 archlinux sudo[1152]: free : TTY=pts/0 ; PWD=/home/free ; USER=root ; COMMAND=/usr/bin/systemctl stop psd
Mar 21 08:26:34 archlinux sudo[1204]: free : TTY=pts/0 ; PWD=/home/free ; USER=root ; COMMAND=/usr/bin/systemctl start psd

Last edited by freestyler7 (2013-03-21 12:02:58)

graysky · 2013-03-21 13:14:05

@free - I can't explain why the service did not start without seeing any error log. Just as an fyi, you don't need to overwrite the conf file when presented with a pacnew file. I use vimdiff for merging files but there are other tools if you don't like vim. I think meld is a popular one. See this link for a quick howto use vimdiff if you wanna go that route:

http://amjith.blogspot.com/2008/08/quic … orial.html

med1972 · 2013-03-23 17:06:45

My laptop just crashed/huing on me. Power cycled and now firefox is giving me the "Your profile could not be found" business. psd is running and my profile directory is linked to /tmp but the target location is not there:

$ ls -l .mozilla/firefox
total 24
drwx------  4 citmkd users 4096 Jul 20  2012 Crash Reports
drwx------  8 citmkd users 4096 Mar 10 14:22 fbctf1xc.default
drwx------  8 citmkd users 4096 Mar 23 13:01 k9gz39mr.Safe Mode
lrwxrwxrwx  1 citmkd users   36 Mar 19 22:57 mwad0hks.default -> /tmp/citmkd-firefox-mwad0hks.default
drwx------ 16 citmkd users 4096 Mar 22 21:18 mwad0hks.default-backup
drwx------  6 citmkd users 4096 Mar 24  2012 nja4eria.default
-rw-------  1 citmkd users  168 Mar 23 12:33 profiles.ini
$
 $ ls -l /tmp
total 0
srwxr-xr-x 1 citmkd users  0 Mar 23 12:30 qtsingleapp-homeci-170d-2719
-rw-r--r-- 1 citmkd users  0 Mar 23 12:30 qtsingleapp-homeci-170d-2719-lockfile
drwxrwxrwt 2 root   root  40 Mar 23 12:30 systemd-private-fhzvWQ
drwxrwxrwt 2 root   root  40 Mar 23 12:29 systemd-private-SYYziI
drwxr-xr-x 2 citmkd users 60 Mar 23 12:33 vmrc-plugin
drwxr-xr-x 2 citmkd users 60 Mar 23 12:33 vmware-cs-plugin
$

Also, when I run "psd preview" is does not list my user despite being in the USERS variable in /etc/psd.conf and despite the fact that this was working fine prior to the crash.

$ profile-sync-daemon preview
Profile-sync-daemon v5.31 on Arch Linux.

Daemon file /run/psd is present.
Service is currently active.

Psd will manage the following per /etc/psd.conf settings:

$ grep citmkd /etc/psd.conf 
12:USERS="citmkd"
$

Restarting the psd service corrects this problem.

graysky · 2013-03-23 17:12:01

Do this:

cp -a ~/.mozilla/firefox/mwad0hks.default-backup ~
stop psd

The repost the output of:

ls -l ~/.mozilla/firefox

And of:

cat ~/.mozilla/firefox/profiles.ini

trusktr · 2013-03-29 06:24:36

Hi graysky, can you briefly describe what the resync option is for? I'm not sure what you mean by "Synchronize the tmpfs and media bound copy.".

And out of curiosity, why two systemd units instead of one? Perhaps explaining the resync option will bring this to light.

Feature request: allow for the USERS or BROWSERS variables in psd.conf to be empty (and for psd to do nothing in either case). It would be a valid use case for some people. Just an idea.

Great work!

graysky · 2013-03-29 08:56:31

Resync will update the media-bound copy. Say you add some bookmarks. The tmpfs copy knows about them but until it sync back to the disk, they are volatile. Resync does this.

To your feature request. What use case would want an empty USER? BROWSERS is allowed to be empty and is the default. See the man page.

Lockheed · 2013-04-08 11:43:16

For the third time I lost 40-page browser session in Opera after rebooting my system. I am beginning to thing using PSD is just too risky.

graysky · 2013-04-08 14:26:12

Need more details to figure out what is wrong. The project is pretty mature with thousands of users across multiple distros... I'm not saying that its perfect

You posted here making reference to external scripts you are running that also interact with your profile... Not sure what these are or if they could be causing problems...

Lockheed wrote:

2. I have external scripts that delete older cache files from opera profile. It is set to run every hour. If that occurs, won't the deleted cache files be restored back form memory upon daemon shutdown?

Do you experience problems with other browsers + psd?

Lockheed · 2013-04-08 14:58:37

The external scripts delete cache. They don't touch other folders.

I occasionally get this issue with Chrome as well, but because Opera is my main browser, it affects it the most. And 1 out of 10 times when I have to manually restore ~/opera-backup to ~/opera, the backup turns out to be purged.

Awebb · 2013-04-08 16:21:23

Tousands of users? Cool :-) Do you keep track of that in any way or did you just count the feedback?

graysky · 2013-04-08 19:19:09

@Lockheed - Hmm... now to figure out why this is happening on your system.

Plz post the output of:
`systemctl | grep psd`
`psd p`
`df -h /tmp`

@Awebb -
*For Arch, have a look at the repo-ck server stats... the back pages show unique downloads of the source. This only counts folks building from the AUR; people taking the source from github directly can't be counts as far as I know.
*Beyond that, I counted just over 1,600 downloads from my launchpad repo (Ubuntu/Debian/Mint).
*I can't count the SUSE, Gentoo, Exherbo, Chakra, and Fedora users since other people host their packages

Lockheed · 2013-04-08 23:21:38

psd.service                 loaded active exited    Profile-sync-daemon
psd-resync.timer            loaded active waiting   Timer for Profile-sync-daemon - 1Hour

$ psd p
Profile-sync-daemon v5.32 on Arch Linux.

Daemon file /run/psd is present.
Service is currently active.

Psd will manage the following per /etc/psd.conf settings:

 browser/psname:  opera/opera
 owner/group:     juha/users
 sync target:     /home/juha/.opera
 tmpfs dir:       /tmp/juha-opera
 profile size:    159M

 browser/psname:  google-chrome/chrome
 owner/group:     juha/users
 sync target:     /home/juha/.config/google-chrome
 tmpfs dir:       /tmp/juha-google-chrome
 profile size:    38M

$ df -h /tmp
Filesystem      Size  Used Avail Use% Mounted on
tmpfs           3.0G  254M  2.8G   9% /tmp

graysky · 2013-04-10 10:43:53

@LH - Output looks normal. Does the data loss occur on a system crash, suspend, hibernate? In other words, does it occur between resyncs? Resyncs are scheduled by default to occur once per hour. You can modify this interval to something more frequent if you wish. See: https://wiki.archlinux.org/index.php/Ps … ptional.29

Last edited by graysky (2013-04-10 10:46:34)

Lockheed · 2013-04-10 11:35:06

The last time it occurred was when I installed new kernel, rebooted and got stuck in command line, because nvidia module was not yet compiled. I compiled it, rebooted manually from and upon next boot the opera profile was gone.

So I assume psd has some occasional issues in syncing RAM to disk upon reboot.

graysky · 2013-04-10 13:29:36

@Lock - Hmm... if systemd stopped psd correctly, that shouldn't matter. I am not sure what could have caused that. Can you look in your systemd journal for psd entries with the date/time of this? Post the entire log to pastebin or the like if you want me to look at it.

FYI - There is no need to reboot in the scenario you describe: just build the module, install it, then modprobe it and restart your graphical greeter or start X manually.

Mr Green · 2013-04-11 09:38:56

Had to stop psd this morning as I was getting firefox profile missing error. Some output

 psd p
Profile-sync-daemon v5.32 on Arch Linux.

Daemon file /run/psd is not present.
Service is currently inactive.

Psd will manage the following per /etc/psd.conf settings:

 browser/psname:  firefox/firefox
 owner/group:     mrgreen/users
 sync target:     /home/mrgreen/.mozilla/firefox/9pcumxa2.default-backup
 tmpfs dir:       /tmp/mrgreen-firefox-9pcumxa2.default-backup
 profile size:    133M

switched profiles to backup and firefox works again, may have been a bad boot or mount (user error!)

Be nice to get psd working as it should again....

Last edited by Mr Green (2013-04-11 09:39:22)

graysky · 2013-04-11 21:12:17

@Mr.Green - I think I see the problem. Your sync target has the -backup suffix. This should not be. Since psd parses ~/.mozilla/firefox/profiles.ini to identify link targets, please make sure that the profile is correctly identified in there. It should look something like:

[General]
StartWithLastProfile=1

[Profile0]
Name=default
IsRelative=1
Path=9pcumxa2.default

It shouldn't contain a '-backup' appended on the Path. Does it?

If that looks good,

1) Stop psd.
2) `mv /home/mrgreen/.mozilla/firefox/9pcumxa2.default-backup /home/mrgreen/.mozilla/firefox/9pcumxa2.default-wtf`
3) `cp -a /home/mrgreen/.mozilla/firefox/9pcumxa2.default-wtf /home/mrgreen/.mozilla/firefox/9pcumxa2.default`
4)`psd p`

Does the output look right now? The sync target should be: /home/mrgreen/.mozilla/firefox/9pcumxa2.default and the tmpfs dir should be /home/mrgreen/.mozilla/firefox/9pcumxa2.default-backup

Last edited by graysky (2013-04-11 21:14:48)

Mr Green · 2013-04-12 05:39:55

I changed profile.ini to use default-backup simply to get firefox running again

Profile-sync-daemon v5.32 on Arch Linux.

Daemon file /run/psd is present.
Service is currently active.

Psd will manage the following per /etc/psd.conf settings:

 browser/psname:  firefox/firefox
 owner/group:     mrgreen/users
 sync target:     /home/mrgreen/.mozilla/firefox/9pcumxa2.default
 tmpfs dir:       /tmp/mrgreen-firefox-9pcumxa2.default
 profile size:    227M

Looks ok now thanks

I start firefox from autostart and sometimes its just too fast starting, then I get profile missing. Had to reset system the other day (down to my error) and then firefox would not start at all.

drwx------  5 mrgreen users 4096 Apr 12 06:30 .
drwx------  4 mrgreen users 4096 Aug 24  2012 ..
lrwxrwxrwx  1 mrgreen users   44 Apr 12 06:18 9pcumxa2.default -> /tmp/mrgreen-firefox-9pcumxa2.default-backup
drwx------  3 mrgreen users 4096 Apr 12 06:30 9pcumxa2.default-backup
drwxr-xr-x 18 mrgreen users 4096 Apr 12 06:18 9pcumxa2.default-backup-backup
lrwxrwxrwx  1 mrgreen users   44 Apr 12 06:18 9pcumxa2.default-backup-wtf -> /tmp/mrgreen-firefox-9pcumxa2.default-backup
drwxr-xr-x  3 mrgreen users 4096 Feb 27  2012 Crash Reports
-rw-r--r--  1 mrgreen users   94 Apr 12 06:30 profiles.ini

graysky · 2013-04-12 08:42:18

Yes, that makes sense. Wish I come up with a better way to handle the 'too fast' situation from within psd.

Mr Green · 2013-04-12 10:34:30

It is a combination of having an SSD and systemd, this thing boots so far now sometimes networking and psd just does not have time to catch up. Easy enough to add this too autostart

(sleep 10s && firefox) &

For goodness sake do not slow psd down

graysky · 2013-04-12 18:59:03

@Mr. Green - Have you checked into reducing that 227 MB profile? Less size = less to sync. Profile-cleaner does a nice job IMO.

Arch Linux

#326 2013-03-18 20:14:56

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#327 2013-03-19 08:53:21

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#328 2013-03-21 06:40:39

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#329 2013-03-21 11:44:15

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#330 2013-03-21 12:02:31

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#331 2013-03-21 13:14:05

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#332 2013-03-23 17:06:45

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#333 2013-03-23 17:12:01

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#334 2013-03-29 06:24:36

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#335 2013-03-29 08:56:31

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#336 2013-04-08 11:43:16

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#337 2013-04-08 14:26:12

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#338 2013-04-08 14:58:37

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#339 2013-04-08 16:21:23

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#340 2013-04-08 19:19:09

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#341 2013-04-08 23:21:38

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#342 2013-04-10 10:43:53

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#343 2013-04-10 11:35:06

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#344 2013-04-10 13:29:36

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#345 2013-04-11 09:38:56

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#346 2013-04-11 21:12:17

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#347 2013-04-12 05:39:55

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#348 2013-04-12 08:42:18

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#349 2013-04-12 10:34:30

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#350 2013-04-12 18:59:03

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

Board footer