profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

graysky · 2018-04-14 20:33:14

OK... outside the scope of psd, sorry.

emphire · 2018-04-15 05:27:56

Fair enough. For anyone wanting to use it with Franz (at your own risk), just create a file at /usr/share/psd/browsers/franz with the following content:

DIRArr[0]="$XDG_CONFIG_HOME/Franz"
PSNAME="$browser"

Franz is just a modified version of Chromium so I don't think it should be too risky.

graysky · 2018-04-15 10:50:17

@emphire - I misunderstood... what you did is correct and it should work just fine.

qinohe · 2018-06-19 17:10:59

Hey graysky, would you add waterfox-bin to PSD, I have added the waterfox profile to PSD tested it and works.
It was fairly simple though, copied the firefox profile to waterfox-bin changed the paths and that was it.
Here you have my file, maybe you don't need it but anyways, here you go

if [[ -d $HOME/.waterfox ]]; then
    profileArr=( $(grep '[P,p]'ath= $HOME/.waterfox/profiles.ini |
    sed 's/[P,p]ath=//') )
    index=0
    PSNAME="$browser"
    for profileItem in ${profileArr[@]}; do
        if [[ $(echo $profileItem | cut -c1) = "/" ]]; then
            # path is not relative
            DIRArr[index]="$profileItem"
        else
            # we need to append the default path to give a
            # fully qualified path
            DIRArr[index]="$HOME/.waterfox/$profileItem"
        fi
        index=$index+1
    done
fi

check_suffix=1

Thanks mark

graysky · 2018-06-19 20:28:13

@qinohe - Just drop your profile in /usr/share/psd/browsers/ and you're all set.

qinohe · 2018-06-20 12:05:12

Did that in the first place, thought maybe you'd add it to the equation so it don't get overwritten on update (really not that important), but it's working so ,thanks.

edit: but they don't get overwritten, sorry for the noise.

Last edited by qinohe (2018-06-20 13:13:49)

Mr Green · 2019-05-01 15:06:58

How can you tell if psd is working? systemctl --user says its exited? config set too run Firefox... do you need any other output ...

graysky · 2019-05-01 18:49:08

% systemctl --user status psd
● psd.service - Profile-sync-daemon
   Loaded: loaded (/usr/lib/systemd/user/psd.service; enabled; vendor preset: enabled)
   Active: active (exited) since Wed 2019-05-01 13:40:21 EDT; 1h 8min ago
     Docs: man:psd(1)
           man:profile-sync-daemon(1)
           https://wiki.archlinux.org/index.php/Profile-sync-daemon
  Process: 4098 ExecStart=/bin/true (code=exited, status=0/SUCCESS)
 Main PID: 4098 (code=exited, status=0/SUCCESS)

May 01 13:40:21 ease systemd[4092]: Starting Profile-sync-daemon...
May 01 13:40:21 ease systemd[4092]: Started Profile-sync-daemon.

You can also query psd directly:

% psd p
Profile-sync-daemon v6.34 on Arch Linux

 Systemd service is currently active.
 Systemd resync-timer is currently active.
 Overlayfs v23 is currently active.

Psd will manage the following per /home/facade/.config/psd/.psd.conf:

 browser/psname:  chromium/chromium
 owner/group id:  facade/100
 sync target:     /home/facade/.config/chromium
 tmpfs dir:       /run/user/1000/facade-chromium
 profile size:    425M
 overlayfs size:  88M
 recovery dirs:   none

Mr Green · 2019-05-02 13:54:13

Got it, keep forgetting about --user so used to running systemctl....

OverlayFs Added ;-)

[ ~/.config/psd ] psd p
Profile-sync-daemon v6.34 on Arch Linux

 Systemd service is currently active.
 Systemd resync-timer is currently active.
 Overlayfs v23 is currently active.

Psd will manage the following per /home/mrgreen/.config/psd/psd.conf:

 browser/psname:  firefox/firefox
 owner/group id:  mrgreen/985
 sync target:     /home/mrgreen/.mozilla/firefox/1x2ynqkm.default
 tmpfs dir:       /run/user/1000/mrgreen-firefox-1x2ynqkm.default
 profile size:    48M
 overlayfs size:
 recovery dirs:   none

Thanks for that , off to adjust cache size now....

bilaliz · 2019-08-17 08:55:49

Thanks for all the hard work on this, so I am here and need a little help.
I installed this, then installed overlayfs and enabled it in the config for psd.
now when i restart my pi it does not load wicd, or dwagent (dwservice.net for remote access) nor does it launch chromium. I tried the --user-data-dir=/tmp/something and chromium launched without the profile.

What am i doing wrong and how do I fix it?

graysky · 2019-08-17 11:14:16

@bilaliz - What do you mean you installed overlayfs? This is a kernel built-in. Never tested psd on a RPi but I don't see how it would matter in principal. You do realize that memory on an RPi unless the 4G model is pretty limited and that you can easily approach 30-40% of the memory with a chromium profile, right?

bilaliz · 2019-08-17 18:30:00

thanks for the quick reply graysky, i'll readily admit I am a bit of a noob, trying to figure this all out. I am trying to make a kiosk running chromium in rpi and that's how I landed here.

when I say I installed overlayfs, I mean i ran this: https://gitlab.com/larsfp/rpi-readonly/ … r/setup.sh

without running the above psd ran just fine. WICD loaded and so did dwagent. I read through the script as best as I can but I still can't figure out where things went wrong.

My chromium profile is pretty small though, here is the output from psd p.

psd p
Profile-sync-daemon v6.31 on Raspbian GNU/Linux 10 (buster)

Systemd service is currently active.
Systemd resync-timer is currently active.
Overlayfs v23 is currently active.

Psd will manage the following per /home/pi/.config/psd/psd.conf:

browser/psname: chromium/chromium
owner/group id: pi/1000
sync target: /home/pi/.config/chromium
tmpfs dir: /run/user/1000/pi-chromium
profile size: 31M
overlayfs size: 0
recovery dirs: none

qinohe · 2020-01-08 15:54:28

@graysky, I'm using psd on my upboard 2G mem. and don't know if it's even useful to do so?

Now that's not the reason I'm here, I have updated Firefox twice, yesterday and today.
First time I had this problem using the search on this site https://wiki.opnsense.org/index.html, a reboot solved it.
Second time today, we have another update for FF and I have the same issue.

I have restarted psd, ran 'psd c' still the issue persists, so I would first ask you before I restart if psd(with overlays) could be the problem?
Maybe it's not and for some reason I need to restart the machine, but for now I have no idea why..

I use firejail and uMatrix, they have been turned off which didn't help BTW.

edit: the search just runs forever whitout giving results
similar sites like Nexcloud(RST) and PFsense just work..

Last edited by qinohe (2020-01-08 15:56:23)

graysky · 2020-01-08 16:10:34

@qinoh - Not sure what your problem is... I can search on the opnsense wiki with psd enabled. Not sure if/how firejail or umatrix affect anything. If you close the browser, make sure there aren't any silently running firefox processes (ps aux |grep firefox), then disable psd, does your problem go away?

Last edited by graysky (2020-01-08 16:11:28)

qinohe · 2020-01-08 16:35:01

Thanks @grasky, yes the poblem goes away however FF never really gets killed.
Running 'ps aux|grep firefox' with a closed FF keeps showing a running FF process, the issue however is gone.

guss77 · 2020-01-19 11:56:09

I have installed PSD on Ubuntu, and I'm pretty sure it isn't working as advertised. Might I have missed something?

$ psd p
Profile-sync-daemon v6.34 on Ubuntu 19.10

 Systemd service is currently active.
 Systemd resync-timer is currently active.
 Overlayfs v23 is currently active.

Psd will manage the following per /home/odeda//.config/psd/.psd.conf:

 browser/psname:  firefox/firefox
 owner/group id:  odeda/1000
 sync target:     /home/odeda//.mozilla/firefox/i1m74zv1.default
 tmpfs dir:       /run/user/1000/odeda-firefox-i1m74zv1.default
 profile size:    717M
 overlayfs size:  0
 recovery dirs:   none

So that looks promising, but

$ mount | grep mozilla/firefox
overlaid on /run/user/1000/odeda-firefox-i1m74zv1.default type overlay (rw,nosuid,nodev,relatime,lowerdir=/home/odeda//.mozilla/firefox/i1m74zv1.default-backup,upperdir=/run/user/1000/odeda-firefox-i1m74zv1.default-rw,workdir=/run/user/1000/.odeda-firefox-i1m74zv1.default)
$ lsof -p $(pidof -d$'\n' firefox | sort -n | head -n1) 2>/dev/null | awk '{print$9}' | egrep -o '.*i1m74zv1.default[^/]*' | sort -u
/home/odeda/.cache/mozilla/firefox/i1m74zv1.default
/home/odeda/.mozilla/firefox/i1m74zv1.default

PSD doesn't actually mount the profile - it mounts a backup directory to the tmpfs file system, but it doesn't mount the actual profile dir from the tmpfs file system. Firefox (as can be seen in the second command) still uses ~/.mozilla/firefox/i1m74zv1.default as its profile dir, which is the actual dir on the actual drive, and it doesn't look like PSD is doing anything for me.

My psd.conf is

USE_OVERLAYFS="yes"
BROWSERS="firefox"
USE_BACKUPS="no"

Am I missing some critical piece of the setup?

guss77 · 2020-01-19 12:53:45

guss77 wrote:

I have installed PSD on Ubuntu, and I'm pretty sure it isn't working as advertised. Might I have missed something?

Nevermind, after setting up chrome, I've noticed that it moved the original profile dir to "-backup" and created a link from the original dir name to the tmpfs mounted overlay. It didn't do that for firefox because:

1. my computer crashed
2. when restarted and I tied run Firefox it complained about a missing profile dir and wouldn't launch (PSD should have been running at this point, but the link wasn't created - maybe there was some failure).
3. I restored my profile dir from backup, then I had both the backup (where PSD is working from) and original profile.

After that, PSD restart would just mount the overlayfs without complaining but would not restore the link.

Deleting (all of) the backup dirs and restarting PSD solved the problem.

Mr Green · 2020-01-26 19:42:43

Yo graysky do you think psd will work with brave borwser given that its chrome based?

graysky · 2020-01-27 00:38:20

Should ... you just need to define the the two variables... see /usr/share/psd/browsers/chromium as an example. DIRArr[0] is the path to the profile and PSNAME is the name of the process of the running browser. Just copy it to that same location renamed.

Mr Green · 2020-01-27 14:40:17

Ok will do thank you very much ;-) x

graysky · 2020-05-01 19:10:09

Any users out there on laptops who suspend/sleep and use psd? A new feature will run a re-sync prior to the system going to sleep as extra protection in case the system doesn't wake up cleanly. I'd like some folks to test it beyond the group of 3 who have been so far before I make a formal release. I tagged it as v6.39b2. You can build it with thisPKGBUILD.

Note that you will need to diff your config against the new config so you can enable the feature which is disabled by default! The default config is of course: /usr/share/psd/psd.conf

...
# Uncomment and set to "yes" to resync on suspend to reduce potential data loss.
# Note that your system MUST have gdbus from glib2 installed to use this mode.
#
#USE_SUSPSYNC="yes"
...

Also remember that any changes to ~/.config/psd/psd.conf will not be active until you restart it AND remember that if you restart it with browsers running, they will be closed in doing so:

systemctl --user restart profile-sync-daemon

Please provide feedback on the github issue page or in this thread if you don't have a github account. Thanks.

Last edited by graysky (2020-05-01 19:17:22)

edo0 · 2020-08-14 19:51:19

graysky wrote:

Should ... you just need to define the the two variables... see /usr/share/psd/browsers/chromium as an example. DIRArr[0] is the path to the profile and PSNAME is the name of the process of the running browser. Just copy it to that same location renamed.

Hi! Brave user here as well. I have tried to figure it out from your sparse posts on Github, but unfortunately I was not really able to understand what I have to do with which folders in order to set up psd with Brave browser.
I am somewhat a noob with how browsers work, and I would be grateful if you could detail the steps required!
Thank you in advance!!

graysky · 2020-08-14 20:10:35

@edo0 - See this PR: https://github.com/graysky2/profile-syn … n/pull/175 or this https://github.com/graysky2/profile-syn … n/pull/240

OdinEidolon · 2020-08-24 12:10:06

I've been using PSD for a long time, and only now I notice that:

1) I only have one backup in ~/.config/chromium-backup-crashrecovery-20200119_170537, which is pretty old (PSD is configured to take up to three)
2) My ~/.config/chromium-backup/Default/Application Cache/Cache/ and ~/.config/chromium-backup-ovfs/Default/Application Cache/Cache/ folder take 6GB each.

From Chromium, I cleaned up all the cookies, site and app data: still 6GB.

Does anyone know why this happens?

PSD preview:

Profile-sync-daemon v6.40

 systemd service: active
 resync-timer:    active
 sync on sleep:   disabled
 use overlayfs:   enabled

Psd will manage the following per /home/adriano/.config/psd/.psd.conf:

 browser/psname:  chromium/chromium
 owner/group id:  adriano/1000
 sync target:     /home/adriano/.config/chromium
 tmpfs dir:       /run/user/1000/adriano-chromium
 profile size:    5.9G
 overlayfs size:  0
 recovery dirs:   1 <- delete with the c option
  dir path/size:  /home/adriano/.config/chromium-backup-crashrecovery-20200119_170537 (511M)

PSD config:

#
# $XDG_HOME_CONFIG/psd/psd.conf
#
# For documentation, refer to the psd man page or the wiki page
# https://wiki.archlinux.org/index.php/Profile-sync-daemon

## NOTE the following:
## To protect data from corruption, in the event that you do make an edit while
## psd is active, any changes made will be applied the next time you start psd.

# Uncomment and set to "yes" to use overlayfs instead of a full copy to reduce
# the memory costs and to improve sync/unsync operations. Note that your kernel
# MUST have this module available in order to use this mode
#
USE_OVERLAYFS="yes"

# List browsers separated by spaces to include in the sync. Useful if you do not
# wish to have all possible browser profiles sync'ed which is the default if
# this variable is left commented.
#
# Possible values:
#  chromium
#  chromium-dev
#  conkeror.mozdev.org
#  epiphany
#  firefox
#  firefox-trunk
#  google-chrome
#  google-chrome-beta
#  google-chrome-unstable
#  heftig-aurora
#  icecat
#  inox
#  luakit
#  midori
#  opera
#  opera-beta
#  opera-developer
#  opera-legacy
#  otter-browser
#  qupzilla
#  palemoon
#  rekonq
#  seamonkey
#  surf
#  vivaldi
#  vivaldi-snapshot
#
BROWSERS="chromium"

# Uncomment and set to "no" to completely disable the crash recovery feature.
#
# The default is to create crash recovery backups if the system is ungracefully
# powered-down due to a kernel panic, hitting the reset switch, battery going
# dead, etc. Some users keep very diligent backups and don't care to have this
# feature enabled.
#USE_BACKUPS="yes"

# Uncomment and set to an integer that is the maximum number of crash recovery
# snapshots to keep (the oldest ones are delete first).
#
# The default is to save the most recent 5 crash recovery snapshots.
BACKUP_LIMIT=3

PSD service status:

● psd.service - Profile-sync-daemon
     Loaded: loaded (/usr/lib/systemd/user/psd.service; enabled; vendor preset: enabled)
     Active: active (exited) since Mon 2020-08-24 13:37:22 CEST; 25min ago
       Docs: man:psd(1)
             man:profile-sync-daemon(1)
             https://wiki.archlinux.org/index.php/Profile-sync-daemon
    Process: 1299 ExecStart=/usr/bin/profile-sync-daemon startup (code=exited, status=0/SUCCESS)
   Main PID: 1299 (code=exited, status=0/SUCCESS)

Aug 24 13:37:22 XPS15af systemd[1279]: Starting Profile-sync-daemon...
Aug 24 13:37:22 XPS15af sudo[1326]: pam_systemd_home(sudo:account): Failed to query user record: Unit dbus-org.freedesktop.home1.service not found.
Aug 24 13:37:22 XPS15af profile-sync-daemon[1299]: psd startup check successful
Aug 24 13:37:22 XPS15af systemd[1279]: Finished Profile-sync-daemon.

graysky · 2020-08-24 14:16:37

This is not a psd thing, it's a chromium thing. I do not like chromium caching to my disk. See: https://wiki.archlinux.org/index.php/Ch … e_in_tmpfs

If you want to do this, recommend you:
1) stop psd
2) close chromium
3) backup your profile just in case you screw up something
4) Impliment the wiki's suggestion I linked
5) optionally manually delete the cached items outside of that scope
For example, I routinely delete:
~/.config/chromium/Service\ Worker/CacheStorage/*
~/.config/chromium/GPUCache/*

6) restart psd

Arch Linux

#851 2018-04-14 20:33:14

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#852 2018-04-15 05:27:56

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#853 2018-04-15 10:50:17

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#854 2018-06-19 17:10:59

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#855 2018-06-19 20:28:13

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#856 2018-06-20 12:05:12

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#857 2019-05-01 15:06:58

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#858 2019-05-01 18:49:08

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#859 2019-05-02 13:54:13

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#860 2019-08-17 08:55:49

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#861 2019-08-17 11:14:16

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#862 2019-08-17 18:30:00

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#863 2020-01-08 15:54:28

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#864 2020-01-08 16:10:34

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#865 2020-01-08 16:35:01

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#866 2020-01-19 11:56:09

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#867 2020-01-19 12:53:45

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#868 2020-01-26 19:42:43

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#869 2020-01-27 00:38:20

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#870 2020-01-27 14:40:17

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#871 2020-05-01 19:10:09

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#872 2020-08-14 19:51:19

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#873 2020-08-14 20:10:35

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#874 2020-08-24 12:10:06

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#875 2020-08-24 14:16:37

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

Board footer