profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

graysky · 2018-01-14 11:08:36

What color is the little circle in the first line? Green for me. You're starting a service that just creates the bind mounts. In any case, if `psd p` shows that you're active, you're good to go.

Mr Green · 2018-01-15 15:22:03

Ok my bad, I saw 'exited' and thought it had failed..

Output

psd p
Profile-sync-daemon v6.31 on Arch Linux

 Systemd service is currently active.
 Systemd resync-timer is currently active.
 Overlayfs technology is currently inactive.

Psd will manage the following per /home/mrgreen/.config/psd/.psd.conf:

 browser/psname:  firefox/firefox
 owner/group id:  mrgreen/100
 sync target:     /home/mrgreen/.mozilla/firefox/k4bhk10z.default
 tmpfs dir:       /run/user/1000/mrgreen-firefox-k4bhk10z.default
 profile size:    60M
 recovery dirs:   1 <- delete with the c option
  dir path/size:  /home/mrgreen/.mozilla/firefox/k4bhk10z.default-backup-crashrecovery-20171205_153530 (54M)

indianahorst · 2018-01-25 02:39:47

Hello everybody,

I have psd running, but I can't get OverlayFS working.

Module overlay is loaded:

$ lsmod | grep overlay
overlay                86016  3

Option OVERLAYFS is activated in config file:

$ cat ~/.config/psd/psd.conf | grep OVERLAYFS
USE_OVERLAYFS="yes"

My username is in the sudoers file,

$ sudo cat /etc/sudoers | grep psd
indianahorst ALL=(ALL) NOPASSWD: /usr/bin/psd-overlay-helper

...but OverlayFS is stil inactive:

$ psd preview
Profile-sync-daemon v6.31 on Arch Linux

 Systemd service is currently active.
 Systemd resync-timer is currently active.
 Overlayfs technology is currently inactive.

Psd will manage the following per ~/.config/psd/psd.conf:

 browser/psname:  firefox/firefox
 owner/group id:  indianahorst/100
 sync target:     /home/indianahorst/.mozilla/firefox/6z4dph9c.default
 tmpfs dir:       /run/user/1000/indianahorst-firefox-6z4dph9c.default
 profile size:    157M
 recovery dirs:   none

Any hints how I can activate it?

Last edited by indianahorst (2018-01-25 11:56:48)

graysky · 2018-01-26 00:09:21

@indiana - Did you happen to modify psd.conf while you had it running already? If so, changed will not be effective until you restart the service.

indianahorst · 2018-01-27 11:17:19

graysky wrote:

@indiana - Did you happen to modify psd.conf while you had it running already? If so, changed will not be effective until you restart the service.

Yes, I restarted the service multple times. It didn't helped. I also restarted the system several times - didn't help either.

indianahorst · 2018-01-28 13:26:25

Are there any other hints which I can try?

PBS · 2018-02-02 22:05:18

My earlier post seems to have been passed by, so I'll try to do a better job of explaining myself this time.

I would like to simply point out that psd's goal of "keeping browser profiles in memory and sync'ed" is exactly what the kernel is supposed to do anyway:

Both psd and the kernel ensure that modifications to files don't immediately hit the disk. psd keeps them in tmpfs, the kernel keeps them in disk cache.
Both psd and the kernel periodically sync the in-memory modifications to disk. (for the kernel, this is controlled by the 'commit=' mount parameter.)
Both psd and the kernel read files from disk on first access, then keep them in memory to speed up future accesses.

However, the exception is when either of two things cause a disk sync: i) the kernel performing a periodic commit, or ii) an application calling fsync.

Hence, the effect of psd is not really to "keep browser profiles in memory and sync'ed", which the kernel does anyway, but rather "inhibit mechanisms which cause a disk sync".

Now, a tool for blocking fsync already exists - it's called libeatmydata, and it has some advantages over psd: i) it's stupid simple, and ii) it can be deployed system-wide so the benefits aren't just limited to browsers.

Thus, to achieve the same advantages of psd another way, it's sufficient to turn on libeatmydata, and crank up the commit time of the filesystem so the kernel doesn't sync as frequently. My point was this obvious design point seems to have gone unnoticed.

Slithery · 2018-02-02 22:57:55

@PBS, you're missing a few important use cases.

The entire browser profile is loaded into RAM before being accessed, meaning that the browser will start quicker and disk access isn't required the first time that the cache/whatever is accessed.

Flushing happens much less regularly, meaning the drive doesn't spin up as often therefore saving power.

indianahorst · 2018-02-03 02:33:43

I'm not shure, but I think I found a bug:

 
$ ls -l /home/indianahorst/~
ls: cannot access '/home/indianahorst/~': No such file or directory

$ psd p
 First time running psd so please edit ~/.config/psd/psd.conf to your liking and run again.

$ ls -l /home/indianahorst/~
total 0

$ psd p
Profile-sync-daemon v6.31 on Arch Linux

 Systemd service is currently inactive.
 Systemd resync-timer is currently inactive.
 Overlayfs technology is currently inactive.

Psd will manage the following per ~/.config/psd/psd.conf:

 browser/psname:  firefox/firefox
 owner/group id:  indianahorst/100
 sync target:     /home/indianahorst/.mozilla/firefox/4j0rxjy2.testing
 tmpfs dir:       /run/user/1000/indianahorst-firefox-4j0rxjy2.testing
 profile size:    21M
 recovery dirs:   none

Why does psd create the directory "~" at the first call of "psd p"?
And with the second call of "psd p" I get the expected listing of my browser profiles and status of psd?

Last edited by indianahorst (2018-02-03 10:57:51)

graysky · 2018-02-03 09:24:28

~ is a shortcut for /home/indianahorst

indianahorst · 2018-02-03 10:57:23

graysky wrote:

~ is a shortcut for /home/indianahorst

No.
"psd p" creates a directory "~" IN my home directory /home/indianahorst.

progandy · 2018-02-03 11:14:07

This looks like your environment variables are set wrong. What is the output of

echo "$XDG_CONFIG_HOME"
echo "$HOME"

Both should be absolute paths without "~". XDG_CONFIG_HOME is allowed to be unset, as well.

Last edited by progandy (2018-02-03 12:31:31)

PBS · 2018-02-03 16:11:24

slithery wrote:

You're missing a few important use cases.
The entire browser profile is loaded into RAM before being accessed, meaning that the browser will start quicker and disk access isn't required the first time that the cache/whatever is accessed.

If you really want the entire profile to be preloaded at login, a general purpose tool which accomplishes this is ureadahead. Again, the benefits are not limited to browsers but apply to the whole system.

Otherwise, psd has an overlayfs mode whereby nothing is loaded into memory at startup. This also seems to be the recommended usage, as it does not slow down login and uses less memory. So my original point, that psd+overlayfs is effectively a complicated fsync blocker, still stands.

Flushing happens much less regularly, meaning the drive doesn't spin up as often therefore saving power.

Flushing can be made as infrequent as you like by modifying mount options. And desktop hard drives usually don't spin down for power management. As for laptops: I find that it's virtually impossible to keep the disk spun down for more than about 30 seconds anyway, as some process somewhere will always wake it up again. So unless you can guarantee that the drive will remain spun down for a long time, it's actually more efficient to just keep it spinning.

indianahorst · 2018-02-04 22:14:50

progandy wrote:

This looks like your environment variables are set wrong. What is the output of
echo "$XDG_CONFIG_HOME"
echo "$HOME"
Both should be absolute paths without "~". XDG_CONFIG_HOME is allowed to be unset, as well.

Thank you very much! Indeed it was $XDG_CONFIG_HOME, which was set wrong. Now I've unset it and psd works as desired.

OverlayFS is now also active.

Problem solved! Thanks again!

FarStar07 · 2018-02-28 12:24:29

Hello graysky,

Please, could you support Falkon, the successor to QupZilla? => https://davidrosca.blogspot.fr/2018/02/ … eased.html
The profile folder is ~/.config/falkon/

Thank you very much.

Last edited by FarStar07 (2018-02-28 13:34:38)

graysky · 2018-02-28 19:58:43

FarStar07 wrote:

Please, could you support Falkon, the successor to QupZilla? => https://davidrosca.blogspot.fr/2018/02/ … eased.html

v6.33

Last edited by graysky (2018-02-28 19:59:12)

FarStar07 · 2018-03-01 06:00:57

graysky wrote:

FarStar07 wrote:
Please, could you support Falkon, the successor to QupZilla? => https://davidrosca.blogspot.fr/2018/02/ … eased.html
v6.33

Thank you!

Mr Green · 2018-03-04 09:52:11

Cannot seem to get psd running. Ran psd set up ~/.config/psd/psd.conf ran psd again then

systemctl enable psd

Entered my user password then got

[mrgreen@arch psd]$ systemctl start psd
==== AUTHENTICATING FOR org.freedesktop.systemd1.manage-units ====
Authentication is required to start 'psd.service'.
Authenticating as: mrgreen
Password: 
==== AUTHENTICATION COMPLETE ====
Failed to start psd.service: Unit psd.service not found.

graysky · 2018-03-04 11:23:40

Mr Green wrote:

Cannot seem to get psd running. Ran psd set up ~/.config/psd/psd.conf ran psd again then

systemctl enable psd

Entered my user password then got

[mrgreen@arch psd]$ systemctl start psd
==== AUTHENTICATING FOR org.freedesktop.systemd1.manage-units ====
Authentication is required to start 'psd.service'.
Authenticating as: mrgreen
Password: 
==== AUTHENTICATION COMPLETE ====
Failed to start psd.service: Unit psd.service not found.

Psd isn't a system daemon, it's a user daemon:

man psd wrote:

START AND STOP PSD
With the release of the version 6.x series of psd, the only init system that is officially supported is systemd. Psd ships with a systemd user service to start or stop it (psd.service). Additionally, a provided resync-timer will run an hourly resync from tmpfs back to the disk. The resync-timer is started automatically with psd.service so there is no need to start the timer; only start psd.service.
$ systemctl --user [option] psd.service
Available options:
start
stop
enable
disable

Mr Green · 2018-03-04 17:39:57

psd.service                                                                           loaded active exited    Profile-sync-daemon

Hopefully it is working now ;-)

graysky · 2018-03-04 17:41:24

Mr Green wrote:

Hopefully it is working now ;-)

Just inspect `psd p` ... if your service is "active" you should be working.

Mr Green · 2018-03-04 17:45:37

Ah ha!

[mrgreen@arch tmp]$ psd p
Profile-sync-daemon v6.33 on Arch Linux

 Systemd service is currently active.
 Systemd resync-timer is currently active.
 Overlayfs technology is currently inactive.

Psd will manage the following per /home/mrgreen/.config/psd/.psd.conf:

 browser/psname:  firefox/firefox
 owner/group id:  mrgreen/100
 sync target:     /home/mrgreen/.mozilla/firefox/k4bhk10z.default
 tmpfs dir:       /run/user/1000/mrgreen-firefox-k4bhk10z.default
 profile size:    62M
 recovery dirs:   1 <- delete with the c option
  dir path/size:  /home/mrgreen/.mozilla/firefox/k4bhk10z.default-backup-crashrecovery-20171205_153530 (54M)

Was looking in /tmp doh!

emphire · 2018-04-14 00:14:47

Is it possible to add support for browser-based electron apps (like franz)?

graysky · 2018-04-14 10:28:48

No idea what that is or where it writes out data.

emphire · 2018-04-14 20:19:28

Electron is for making cross-platform 'apps' using an instance of chromium. Franz is one such app for assorted chat programs (messanger, whatsapp, skype etc). Franz stores its profile data under ~/.config/Franz in the same format as Chromium but with some extra files.

Arch Linux

#826 2018-01-14 11:08:36

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#827 2018-01-15 15:22:03

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#828 2018-01-25 02:39:47

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#829 2018-01-26 00:09:21

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#830 2018-01-27 11:17:19

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#831 2018-01-28 13:26:25

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#832 2018-02-02 22:05:18

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#833 2018-02-02 22:57:55

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#834 2018-02-03 02:33:43

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#835 2018-02-03 09:24:28

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#836 2018-02-03 10:57:23

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#837 2018-02-03 11:14:07

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#838 2018-02-03 16:11:24

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#839 2018-02-04 22:14:50

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#840 2018-02-28 12:24:29

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#841 2018-02-28 19:58:43

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#842 2018-03-01 06:00:57

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#843 2018-03-04 09:52:11

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#844 2018-03-04 11:23:40

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#845 2018-03-04 17:39:57

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#846 2018-03-04 17:41:24

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#847 2018-03-04 17:45:37

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#848 2018-04-14 00:14:47

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#849 2018-04-14 10:28:48

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#850 2018-04-14 20:19:28

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

Board footer