profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

OdinEidolon · 2020-08-24 14:21:44

graysky wrote:

This is not a psd thing, it's a chromium thing. I do not like chromium caching to my disk. See: https://wiki.archlinux.org/index.php/Ch … e_in_tmpfs
If you want to do this, recommend you:
1) stop psd
2) close chromium
3) backup your profile just in case you screw up something
4) Impliment the wiki's suggestion I linked
5) optionally manually delete the cached items outside of that scope
For example, I routinely delete:
~/.config/chromium/Service\ Worker/CacheStorage/*
~/.config/chromium/GPUCache/*
6) restart psd

Well on the other hand I don't have 3GB of spare RAM to cache all the Application Cache (which is what is taking up most of the space)... though choice. Thanks anyway.

graysky · 2020-08-24 17:15:44

You don't need to relocate all application cache to tmpfs, you can do it just for your browser. See the link.

OdinEidolon · 2020-08-24 18:34:21

graysky wrote:

You don't need to relocate all application cache to tmpfs, you can do it just for your browser. See the link.

Thanks, I know that. However, my chromium Application Cache folder is 3GB (no idea why it is that large). I do not mean the cache for all system applications!

The one located in
~/.config/chromium/Default/Application Cache/Cache/

InvisibleRasta · 2021-01-28 09:25:39

been trying to get psd working but after installing and running psd and editing the config file... I launch the daemon with systemctl --user start psd and it fails:

Job for psd.service failed because the control process exited with error code.
See "systemctl --user status psd.service" and "journalctl --user -xe" for details.

systemctl --user status psd.service
● psd.service - Profile-sync-daemon
     Loaded: loaded (/usr/lib/systemd/user/psd.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Thu 2021-01-28 10:21:24 CET; 3min 31s ago
       Docs: man:psd(1)
             man:profile-sync-daemon(1)
             https://wiki.archlinux.org/index.php/Profile-sync-daemon
    Process: 6139 ExecStart=/usr/bin/profile-sync-daemon startup (code=exited, status=1/FAILURE)
   Main PID: 6139 (code=exited, status=1/FAILURE)

Jan 28 10:21:24 arch systemd[833]: Starting Profile-sync-daemon...
Jan 28 10:21:24 arch profile-sync-daemon[6139]: Refusing to start; firefox is running by invra!
Jan 28 10:21:24 arch systemd[833]: psd.service: Main process exited, code=exited, status=1/FAILURE
Jan 28 10:21:24 arch systemd[833]: psd.service: Failed with result 'exit-code'.
Jan 28 10:21:24 arch systemd[833]: Failed to start Profile-sync-daemon.

graysky · 2021-01-28 09:36:18

Do you have firefox running? Close it before starting psd.

InvisibleRasta wrote:

...
Jan 28 10:21:24 arch profile-sync-daemon[6139]: Refusing to start; firefox is running by invra!
...

cookiewookie · 2021-02-09 09:04:15

Thank you for this great contribution.

I wonder if I could misuse the tool to achieve a secondary sync via network.

My goal is to have several browsers ,*chrome,chromium, firefox, firefox-dev in sync on 2 machines ,*arch and macos, at home.

Would this be possible in theory?

The browser sync services do not suffice as these miss many settings especially regarding the browser profiles and extension profiles.

best

Last edited by cookiewookie (2021-02-09 09:05:18)

DAC324 · 2021-03-29 15:25:06

Thanks a lot from my side for this nice program.
Unfortunately, it seems to have some problems with the latest Chromium Developer versions. They spit out an error message saying "your profile may be damaged" and as a consequence, all your settings are lost.

UPDATE: Looks like, especially when setting up psd for more than one browser, you will have to increase the size of the tmpfs directory used.

Last edited by DAC324 (2021-03-29 15:58:46)

ziltoid · 2021-05-31 10:45:25

Hello and thanks for the program.

I recently installed it but I'm not sure if it's working as intended in my setup. Comparing the output of psd p found in Profile-sync-daemon#Preview_(parse)_mode and that obtained on my pc

Profile-sync-daemon v6.44

 systemd service: active
 resync-timer:    active
 sync on sleep:   disabled
 use overlayfs:   enabled

Psd will manage the following per /home/ziltoid/.config/psd/psd.conf:

 browser/psname:  firefox/firefox
 owner/group id:  ziltoid/1000
 sync target:     /home/ziltoid/.mozilla/firefox/lt3s81c4.default
 tmpfs dir:       /home/ziltoid/.config/ziltoid-firefox-lt3s81c4.default
 profile size:    4.0K
 overlayfs size:  
 recovery dirs:   none

 browser/psname:  firefox/firefox
 owner/group id:  ziltoid/1000
 sync target:     /home/ziltoid/.mozilla/firefox/64tjav0j.default-release
 tmpfs dir:       /home/ziltoid/.config/ziltoid-firefox-64tjav0j.default-release
 profile size:    211M
 overlayfs size:  
 recovery dirs:   none

I can see that in my case the overlayfs size is absent and the recovery dirs are not specified. Is it normal?

I have also relocated FF cache on RAM as wrote in https://wiki.archlinux.org/title/Firefo … disk_cache and the .default files are linked to the /run dir.

ls -Alh ~/.mozilla/firefox 
total 32K
lrwxrwxrwx  1 ziltoid ziltoid   55 mag 31 09:13  64tjav0j.default-release -> /run/user/1000/ziltoid-firefox-64tjav0j.default-release
drwx------ 16 ziltoid ziltoid 4.0K mag 31 11:34  64tjav0j.default-release-back-ovfs
drwx------ 16 ziltoid ziltoid 4.0K mag 31 00:01  64tjav0j.default-release-backup
drwx------  3 ziltoid ziltoid 4.0K mag  6 18:50 'Crash Reports'
-rw-r--r--  1 ziltoid ziltoid   62 feb  1 21:10  installs.ini
lrwxrwxrwx  1 ziltoid ziltoid   47 mag 31 09:13  lt3s81c4.default -> /run/user/1000/ziltoid-firefox-lt3s81c4.default
drwx------  2 ziltoid ziltoid 4.0K mag 31 09:13  lt3s81c4.default-back-ovfs
drwx------  2 ziltoid ziltoid 4.0K mag 30 21:11  lt3s81c4.default-backup
drwx------  2 ziltoid ziltoid 4.0K ott 15  2020 'Pending Pings'
-rw-r--r--  1 ziltoid ziltoid  259 feb  1 21:10  profiles.ini

Last edited by ziltoid (2021-05-31 10:57:06)

graysky · 2021-05-31 12:33:42

No, the size should be 0 if no data are written. Did you make changes to psd.conf while it was running? They will be applied the next time you start the daemon. Exit your browser, and restart the daemon.

ziltoid · 2021-05-31 14:18:40

Just to be sure I quit FF and stopped and restarted the service with

[ziltoid@arch ~]$ systemctl --user stop psd.service 
[ziltoid@arch ~]$ systemctl --user enable --now psd.service

but the output of psd p is the same as before.

jfernyhough · 2022-01-11 14:27:18

I might be hitting an issue with PSD and kernel 5.16 - with psd enabled and started, Firefox will complain "bookmarks and history won't work because a file is in use by another application".

I'll dig a little more and also make sure this isn't due to systemd 250.2.

Edit: working fine with linux-lqx 5.15.13, not working with linux-zen 5.16.0.

Edit 2: this might be affecting anything-sync-daemon too... wonder if it's related to overlayfs.

Edit 3: it is working with 5.16 on another system, so I don't know.

Edit 4: Could be related to ZFS and kernel 5.16, e.g. https://github.com/openzfs/zfs/issues/12926 . System with profile data on BTRFS is working fine.

Edit 5: It's ZFS bug, https://github.com/openzfs/zfs/issues/12971. Nothing to do with psd.

Last edited by jfernyhough (2022-01-17 01:36:21)

ron9 · 2022-02-03 12:22:57

Hello graysky,

Please, could you support Microsoft Edge dev. Microsoft Edge is based on Chromium.
The profile folder is ~/.config/microsoft-edge-dev/

Thank you very much.

graysky · 2022-02-03 12:29:24

ron9 wrote:

Hello graysky,
Please, could you support Microsoft Edge dev. Microsoft Edge is based on Chromium.
The profile folder is ~/.config/microsoft-edge-dev/
Thank you very much.

https://github.com/graysky2/profile-syn … issues/279

sazop · 2022-02-18 05:50:29

Hello.

How can i use PSD with custom Chromium Data Dir ?

For example,
chromium --app=https://tweetdeck.twitter.com/ --class=WebApp-Twitter3548 --user-data-dir=/home/alex/.local/share/ice/profiles/Twitter3548

sazop · 2022-03-24 22:05:54

Can u help me ?

Slithery · 2022-03-24 22:15:11

sazop wrote:

Can u help me ?

Don't do that...
https://wiki.archlinux.org/title/Genera … es#Bumping

jeois · 2022-04-12 19:52:04

sazop wrote:

chromium --app=https://tweetdeck.twitter.com/ --class=WebApp-Twitter3548 --user-data-dir=/home/alex/.local/share/ice/profiles/Twitter3548

I don't know how to do that with the psd scripts. However, if you look up anything-sync daemon (asd), you can configure asd to hold that user directory.
https://wiki.archlinux.org/title/Anything-sync-daemon

------------------------------------------------------------------

I'm having an issue where psd perhaps crashed when I rebooted and might have left a bad save. When this happens, Firefox leaves 2 files called .parentlock and a link called .lock. Usually, what I do is stop PSD and then go delete those lock files in every profile directory, and then I'm back in business.

This time, however, whenever I restart PSD, firefox will still think I'm "already running" the browser when I'm only starting the first instance. This error is typical of when the lock is placed on the profile. But there's no lock files present in those profile directories, and the browser launches fine if I stop psd. Moreover, using "firefox -p" and selecting my other empty profile doesn't work either.

Long story short, the browser launches if I stop psd, but it won't if psd is running... even though the 'preview' and 'status' aren't showing any errors anymore.

What is going on? Do I need to delete the crashrecovery or ovfs-backup directories too? I renamed the crashrecovery to default profile and restarted psd, but the result is the same. It loads an older version of my profile if I don't run psd, but the browser refused to launch when I do run psd.

jeois · 2022-04-12 21:45:41

[ 5349.876374] audit: type=1327 audit(1649799706.932:23820): proctitle="/usr/lib/firefox/firefox"
[ 5349.876376] audit: type=1400 audit(1649799706.932:23821): apparmor="DENIED" operation="open" profile="firefox" name="/proc/23594/comm" pid=23594 comm="firefox" requested_mask="r" denied_mask="r" fsuid=1000 ouid=1000

DMESG is showing denials by apparmor. Does anyone know how to edit the apparmor configs to allow profile-sync daemon to work again?

---
Edit: I reinstalled firefox-ubuntu-bin (firefox-appmenu 99.0-1 lacks a vaapi NVDEC fix for video hardware acceleration), and now it wants to work for no apparent reason. I'm pretty sure I already tried that earlier.

apparmor seems to be a pain and causes major breakages whenever psd or asd fails to unload. Sometimes it resolves itself without rhyme or reason. Other times you have to create a new browser profile or a new apparmor profile. It's very inconsistent. I hope there's a better way to manage it with these overlay sync scripts.

Last edited by jeois (2022-04-12 22:47:40)

geedorah · 2022-06-14 21:12:49

My Firefox profile seems to be getting wiped at boot. At one point psd was working fine, so I'm unsure when/where the issue was introduced. I've tried enabling/disabling overlayfs in my user's config file (/home/user/.config/psd/psd.conf) to no avail.

I see permission denied errors in my log files:

Jun 14 13:51:57 Arch-PC profile-sync-daemon[15646]: find: ‘/home/user/.local/share/containers/storage/overlay.bak/e2b475354c50ddf309212957890c2f7e07ac4b2f6ffa20d4a3ce16ac9f8b76af/work/work’: Permission denied
 414   │ Jun 14 13:51:57 Arch-PC profile-sync-daemon[15646]: find: ‘/home/user/.local/share/containers/storage/overlay.bak/d0a18c5bbf81381d95a0f8d13ad72b8dd3909f10dfeeee86ea734704add04db9/work/work’: Permission denied
 415   │ Jun 14 13:51:57 Arch-PC profile-sync-daemon[15646]: find: ‘/home/user/.local/share/containers/storage/overlay.bak/fc2c5f0e91bfe728a439807da35e5a17d6de0b199a8e8e6a53a92b093408671e/diff/var/cache/apt/archives/partial’: Permission denied
 416   │ Jun 14 13:51:57 Arch-PC profile-sync-daemon[15643]: find: ‘/home/user/.local/share/containers/storage/overlay.bak/e2b475354c50ddf309212957890c2f7e07ac4b2f6ffa20d4a3ce16ac9f8b76af/work/work’: Permission denied

Those errors continue for about 30 lines.

Checked the Arch Wiki page and confirmed that my sudoers is correct:

user ALL=(ALL) NOPASSWD: /usr/bin/psd-overlay-helper

Any clues to what I might be missing here? Thanks for the tool, it's great.

zaxdan69 · 2022-09-29 08:34:33

Hello,
I just tried firefox-profile-daemon and I'm not happy about it. I use it with firefox and I followed wiki to configure and enabled it. I didn't notice any speed changes at startup of firefox or loading pages plus when I reboot it needs 2-3 minutes to can run firefox. Until then I cannot run it and I get the error that firefox is already running...
Is there anything more I can do? And if not is needed any special actions to do except to uninstall it so to don't loose my profile?
This is my psd.conf:

#
# $XDG_CONFIG_HOME/psd/psd.conf
#
# For documentation, refer man 1 psd or to the wiki page
# https://wiki.archlinux.org/index.php/Profile-sync-daemon

## NOTE the following:
## To protect data from corruption, in the event that you do make an edit while
## psd is active, any changes made will be applied the next time you start psd.

# Uncomment and set to "yes" to use overlayfs instead of a full copy to reduce
# the memory costs and to improve sync/unsync operations. Note that your kernel
# MUST have this module available in order to use this mode.
#
USE_OVERLAYFS="yes"

# Uncomment and set to "yes" to resync on suspend to reduce potential data loss.
# Note that your system MUST have gdbus from glib2 installed to use this mode.
#
USE_SUSPSYNC="yes"

# List any browsers in the array below to have managed by psd. Useful if you do
# not wish to have all possible browser profiles managed which is the default if
# this array is left commented.
#
# Possible values:
#  chromium
#  chromium-dev
#  conkeror.mozdev.org
#  epiphany
#  falkon
#  firefox
#  firefox-trunk
#  google-chrome
#  google-chrome-beta
#  google-chrome-unstable
#  heftig-aurora
#  icecat
#  inox
#  luakit
#  midori
#  opera
#  opera-beta
#  opera-developer
#  opera-legacy
#  otter-browser
#  qupzilla
#  qutebrowser
#  palemoon
#  rekonq
#  seamonkey
#  surf
#  vivaldi
#  vivaldi-snapshot
#
BROWSERS=(firefox)

# Uncomment and set to "no" to completely disable the crash recovery feature.
#
# The default is to create crash recovery backups if the system is ungracefully
# powered-down due to a kernel panic, hitting the reset switch, battery going
# dead, etc. Some users keep very diligent backups and don't care to have this
# feature enabled.
USE_BACKUPS="yes"

# Uncomment and set to an integer that is the maximum number of crash recovery
# snapshots to keep (the oldest ones are deleted first).
#
# The default is to save the most recent 5 crash recovery snapshots.
BACKUP_LIMIT=3

graysky · 2022-09-29 10:26:27

Shutdown times should not be affected. Post the output of: psd p

To uninstall, simply stop psd (systemctl --user psd stop) and disable it. Your profile should be as it was.

zaxdan69 · 2022-09-29 12:32:17

output of psd p:

Profile-sync-daemon v6.48

 systemd service: active
 resync-timer:    active
 sync on sleep:   enabled
 use overlayfs:   enabled

Psd will manage the following per /home/archzaxus/.config/psd/psd.conf:

 browser/psname:  firefox/firefox
 owner/group id:  archuser/1000
 sync target:     /home/archuser/.mozilla/firefox/eus5u3ci.default-release
 tmpfs dir:       /run/user/1000/psd/archuser-firefox-eus5u3ci.default-release
 profile size:    725M
 overlayfs size:
 backup limit:    3
 recovery dirs:   1 <- delete with the c option
  dir path/size:  /home/archuser/.mozilla/firefox/eus5u3ci.default-release-backup-crashrecovery-20220929_111827 (706M)

I don't have problem with shutdown(firefox or machine). The main problem is that after a reboot when I login to desktop and run firefox I get the error that is already running and I cannot run it for 2-3 minutes. After that it starts and I don't have any problem to close it and restart it(at least for the short period is used it).
EDIT:
Maybe I need to use it a bit more. I just did another reboot now and this time I didn't have that problem. So maybe it happens because was the first time I use firefox after enabled/start the service. Also this time loading was a bit faster, so I will keep it.I will report back if the problem appears again.
Thanks for the quick reply though.

Last edited by zaxdan69 (2022-09-29 12:40:48)

Stebs · 2023-01-22 15:45:54

I have a problem with the "BACKUP_LIMIT=" setting which is not respected even when set. In fact, psd seems to produce an unlimited number of backups (not only 5), even when left at default (unset).

Setting in ~/.config/psd/psd.conf:

USE_OVERLAYFS="yes"
USE_SUSPSYNC="yes"
#BROWSERS=()
USE_BACKUPS="yes"
BACKUP_LIMIT=2

After restarting the psd.service (as user), this is in .psd.conf:

USE_OVERLAYFS="yes"
USE_SUSPSYNC="yes"
BROWSERS="opera-developer chromium-dev qupzilla inox chromium epiphany heftig-aurora otter-browser icecat falkon google-chrome midori opera opera-next vivaldi-snapshot rekonq conkeror.mozdev.org qutebrowser surf firefox-trunk firefox google-chrome-unstable opera-legacy seamonkey vivaldi google-chrome-beta luakit palemoon opera-beta"
USE_BACKUPS="yes"

So the BACKUP_LIMIT variable is getting omitted, I have to regularly delete a lot of backups with "psd c". (Laptop hangs from time to time).

glitsj16 · 2023-01-22 18:36:56

Stebs wrote:

I have a problem with the "BACKUP_LIMIT=" setting which is not respected even when set. In fact, psd seems to produce an unlimited number of backups (not only 5), even when left at default (unset).
[...]
So the BACKUP_LIMIT variable is getting omitted, I have to regularly delete a lot of backups with "psd c". (Laptop hangs from time to time).

I think this is caused by a missing variable in the do_sync() function. Here's a patch you can try: https://gist.github.com/glitsj16/dae447 … 1f68a7e0d5.

Stebs · 2023-01-23 11:42:35

glitsj16 wrote:

I think this is caused by a missing variable in the do_sync() function. Here's a patch you can try: https://gist.github.com/glitsj16/dae447 … 1f68a7e0d5.

Patch seems to work, at least BACKUP_LIMIT="2" is now correctly in .psd.conf and the limit is also shown with "psd parse".
BTW, I think the line in the default psd.conf:
#BACKUP_LIMIT=5 should be changed to:
#BACKUP_LIMIT="5"

Thank you.
Edit:
Also just now filed a bug about this issue: https://github.com/graysky2/profile-syn … issues/350

Last edited by Stebs (2023-01-24 16:48:27)

Arch Linux

#876 2020-08-24 14:21:44

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#877 2020-08-24 17:15:44

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#878 2020-08-24 18:34:21

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#879 2021-01-28 09:25:39

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#880 2021-01-28 09:36:18

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#881 2021-02-09 09:04:15

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#882 2021-03-29 15:25:06

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#883 2021-05-31 10:45:25

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#884 2021-05-31 12:33:42

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#885 2021-05-31 14:18:40

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#886 2022-01-11 14:27:18

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#887 2022-02-03 12:22:57

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#888 2022-02-03 12:29:24

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#889 2022-02-18 05:50:29

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#890 2022-03-24 22:05:54

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#891 2022-03-24 22:15:11

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#892 2022-04-12 19:52:04

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#893 2022-04-12 21:45:41

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#894 2022-06-14 21:12:49

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#895 2022-09-29 08:34:33

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#896 2022-09-29 10:26:27

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#897 2022-09-29 12:32:17

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#898 2023-01-22 15:45:54

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#899 2023-01-22 18:36:56

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#900 2023-01-23 11:42:35

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

Board footer