profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

ggg377 · 2015-05-27 12:32:26

I have run into some trouble with psd 5.73 and chromium+firefox. I have set up /etc/psd.conf correctly with username and overlayfs preference specified (based on a previously working setup) and profile-sync-daemon parse returns that systemd service, resync service and overlayfs are active at all times. Sync target is: /home/user/.config/chromium and tmpfs dir is: /tmp/user-chromium. However the sync target folder does not get symlinked and the /tmp/user-chromium folder does not exist despite the chromium profile being 43mb. I'm fairly certain nothing is being written into RAM at any time. I haven't noticed any other weird behavior or errors. The exact same thing happens with Firefox.

Edit: just figured out it's probably my encrypted home causing this. Starting to look for a fix, hopefully there is one.

Edit2: Went with full disk encryption instead of home partition encryption and this solved my troubles

Last edited by ggg377 (2015-05-28 18:32:32)

utopyand · 2015-06-02 23:28:42

Had to disable psd, with that enabled i couldn't update some Firefox addons, they were giving an error in browser console saying something about getTrashDir or some hash wrong.
Thought about a bad ram problem, but a single run of Memtest86+ showed no errors.

Last edited by utopyand (2015-06-02 23:30:39)

graysky · 2015-06-02 23:34:35

Haven't the seen that before... Which extension?

Last edited by graysky (2015-06-02 23:34:56)

utopyand · 2015-06-03 00:47:53

Various extensions including restartless and not ones, like ABP and others.
Don't have any log anymore, resolved with a systemd stop psd (profile folder disappeared, but turned back after a F5 in the file manager, that's normal cause psd is copying back profile folder but i should do backups before trying something...)

I found something in dmesg
overlayfs: workdir/#dcf42200 already exists
overlayfs: workdir/#dcf42900 already exists

At least 30 times in log, that getTrashDir in Firefox browser's console... Makes me think an incompatibility between Firefox and overlayfs. Maybe Firefox trying to delete the xpi file.

USE_OVERLAYFS="yes" in /etc/psd.conf
MODULES="overlay" in /etc/mkinitcpio.conf

Last edited by utopyand (2015-06-03 00:49:24)

darzki · 2015-06-04 17:36:06

utopyand wrote:

Had to disable psd, with that enabled i couldn't update some Firefox addons, they were giving an error in browser console saying something about getTrashDir or some hash wrong..

Same here. For me Imagus and Ghostery stopped working (could not open preferences of each extension). I tried to reinstall them, but it was not possible to delete particular extension using extension manager - after browser restart they were displayed as disabled not deleted. I don't know when exactly it happened, as Firefox is not my main browser, but Opera and Vivaldi works fine (with Imagus and Ghostery).

From Firefox output:

addons.xpi      ERROR   Failed to remove empty directory /home/darzki/.mozilla/firefox/2jrz62ls.default/extensions/trash/{3d7eb24f-2740-49df-8937-200b1cc08f8a}/defaults/preferences: [Exception... "Component returned failure code: 0x80520006 (NS_ERROR_FILE_TARGET_DOES_NOT_EXIST)

Removing Firefox from psd solved the problem.

graysky · 2015-06-04 18:47:54

Odd. I use ghostery on chromium + psd and haven't experienced a single issue using overlay mode but I don't see how that would matter.

bendavis78 · 2015-06-10 17:43:30

Is there a similar script to PSD that will mount custom directories? I want to mount my keyring dir in memory, as gnome-keyring-daemon currently has a bug that, when paired with Chrome, causes a LOT of disk i/o on startup.

graysky · 2015-06-10 18:56:05

You can check out anything-sync-daemon.

valo · 2015-06-23 13:17:10

Thanks a lot for your work, I saw the discussion and well, I got some problem with the extensions too

graysky wrote:

Haven't the seen that before... Which extension?

In particular was firefox and cookie monster The plugin was installed but was not enabled so I restarted firefox many times but still it wasn't getting enabled 'till I manually stopped psd and started firefox. After that the plugin was fine and I was able to restart psd. Now everything is working good with psd.
Mount says

overlaid on /tmp/svalo-firefox-xyz.default type overlay (rw,relatime,lowerdir=/home/svalo/.mozilla/firefox/xyz.default-backup,upperdir=/tmp/svalo-firefox-xyz.default-rw,workdir=/tmp/.svalo-firefox-xyz.default)

Hope it helps

killozap · 2015-07-15 15:10:34

I got (severe) problems when overlay is enabled---only if my laptop is booted without the the power cord plugged in.

1) every keyboard input or command (terminal, firefox, editor) causes messages like

login[391]: FAILED LOGIN 1 FROM tty1 FOR ls, Authentication failure
login[391]: pam_tally(login:auth): pam_get_uid; no such user
login[391]: pam_unix(login:auth): check pass; user unknown

(here I typed "ls" in the terminal)
This works like a keylogger that prints every command to the journal -- including my root password in plaintext when entered!

2) The system doesn't unlock but a black screen is shown, leaving restart as only option.

I use current arch with xfce and lightdm.

When USE_OVERLAYFS="no", none of this happens. The option seems to interfere with lightdm. When disabled, before login, lightdm-greeter creates two sessions for user lightdm (destroys one), when enabled, just one is started and -- if at startup only the battery was connected -- these problems occur...
(Haven't been using arch for a long time and still learning, so I can't make that much out of this... sorry -- or maybe I'm trying something stupid? :X)

Last edited by killozap (2015-07-15 15:22:03)

graysky · 2015-07-15 19:47:06

@kill - I cannot reproduce on my system. Launching firefox from a term with psd + overlayfs results in only one warning which is also present without psd running:

% firefox

(process:4164): GLib-CRITICAL **: g_slice_set_config: assertion 'sys_page_size == 0' failed

regulament · 2015-07-25 12:51:44

Hi, any chance you could add support for qutebrowser? As far as I can see it uses ~/.cache/qutebrowser for cache.

killozap · 2015-07-27 16:20:55

graysky: just for info -- in the meantime I reinstalled arch and, with more or less the same setup, everything works fine. some weird clash of modules or configuration it seems...

ansulev · 2015-08-12 12:21:05

@Graysky, Big thanks for all your contrbutions and for profile-sync-daemon, too. Do you plan to add support for ligtherfox? "A light Firefox edition. Many components are slimmed down. So it can start faster and use less memory." In my system it uses almost 1/3 RAM, compared with vanila FFx and is much faster. Stores the profile in:
~/.light/light/xxxxxxxx.default. SF Page: http://sourceforge.net/projects/lightfirefox/ , AUR: (a bit outdated, hope the manatiner will update it soon, but manually edit PKGBUILD fetchs the latest version): https://aur4.archlinux.org/packages/lightfirefox/
Thanks!

Last edited by ansulev (2015-08-12 12:22:09)

jugs · 2015-08-27 02:01:00

I have the same issue as darzki I think, I can't update my addons in firefox. I get the following error:

1440640719418   addons.xpi      ERROR   Failed to remove empty directory /home/user/.mozilla/firefox/fxzxp5er.default-1437352251159/extensions/trash/support@lastpass.com/platform/Linux_x86_64-gcc3
/components: [Exception... "Component returned failure code: 0x80520006 (NS_ERROR_FILE_TARGET_DOES_NOT_EXIST) [nsIFile.remove]"  nsresult: "0x80520006 (NS_ERROR_FILE_TARGET_DOES_NOT_EXIST)"  locat
ion: "JS frame :: resource://gre/modules/addons/XPIProvider.jsm :: recursiveRemove :: line 1524"  data: no] Stack trace: recursiveRemove()@resource://gre/modules/addons/XPIProvider.jsm:1524 < forE
ach()@self-hosted:210 < recursiveRemove()@resource://gre/modules/addons/XPIProvider.jsm:1521 < forEach()@self-hosted:210 < recursiveRemove()@resource://gre/modules/addons/XPIProvider.jsm:1521 < fo
rEach()@self-hosted:210 < recursiveRemove()@resource://gre/modules/addons/XPIProvider.jsm:1521 < forEach()@self-hosted:210 < recursiveRemove()@resource://gre/modules/addons/XPIProvider.jsm:1521 < 
DirInstallLocation_getTrashDir()@resource://gre/modules/addons/XPIProvider.jsm:7521 < DirInstallLocation_installAddon()@resource://gre/modules/addons/XPIProvider.jsm:7543 < XPI_processPendingFileC
hanges/<()@resource://gre/modules/addons/XPIProvider.jsm:2829 < forEach()@self-hosted:210 < XPI_processPendingFileChanges()@resource://gre/modules/addons/XPIProvider.jsm:2658 < XPI_checkForChanges
()@resource://gre/modules/addons/XPIProvider.jsm:3713 < XPI_startup()@resource://gre/modules/addons/XPIProvider.jsm:2288 < callProvider()@resource://gre/modules/AddonManager.jsm:221 < _startProvid
er()@resource://gre/modules/AddonManager.jsm:703 < AMI_startup()@resource://gre/modules/AddonManager.jsm:871 < AMP_startup()@resource://gre/modules/AddonManager.jsm:2511 < AMC_observe()@resource:/
/gre/components/addonManager.js:55 < <file:unknown>
1440640719419   addons.xpi      ERROR   Failed to install staged add-on vimperator@mozdev.org in app-profile: [Exception... "Component returned failure code: 0x80520006 (NS_ERROR_FILE_TARGET_DOES_
NOT_EXIST) [nsIFile.remove]"  nsresult: "0x80520006 (NS_ERROR_FILE_TARGET_DOES_NOT_EXIST)"  location: "JS frame :: resource://gre/modules/addons/XPIProvider.jsm :: recursiveRemove :: line 1524"  d
ata: no] Stack trace: recursiveRemove()@resource://gre/modules/addons/XPIProvider.jsm:1524 < forEach()@self-hosted:210 < recursiveRemove()@resource://gre/modules/addons/XPIProvider.jsm:1521 < forE
ach()@self-hosted:210 < recursiveRemove()@resource://gre/modules/addons/XPIProvider.jsm:1521 < forEach()@self-hosted:210 < recursiveRemove()@resource://gre/modules/addons/XPIProvider.jsm:1521 < fo
rEach()@self-hosted:210 < recursiveRemove()@resource://gre/modules/addons/XPIProvider.jsm:1521 < DirInstallLocation_getTrashDir()@resource://gre/modules/addons/XPIProvider.jsm:7521 < DirInstallLoc
ation_installAddon()@resource://gre/modules/addons/XPIProvider.jsm:7543 < XPI_processPendingFileChanges/<()@resource://gre/modules/addons/XPIProvider.jsm:2829 < forEach()@self-hosted:210 < XPI_pro
cessPendingFileChanges()@resource://gre/modules/addons/XPIProvider.jsm:2658 < XPI_checkForChanges()@resource://gre/modules/addons/XPIProvider.jsm:3713 < XPI_startup()@resource://gre/modules/addons
/XPIProvider.jsm:2288 < callProvider()@resource://gre/modules/AddonManager.jsm:221 < _startProvider()@resource://gre/modules/AddonManager.jsm:703 < AMI_startup()@resource://gre/modules/AddonManage
r.jsm:871 < AMP_startup()@resource://gre/modules/AddonManager.jsm:2511 < AMC_observe()@resource://gre/components/addonManager.js:55 < <file:unknown>
1440640719472   addons.xpi      ERROR   Failed to remove empty directory /home/user/.mozilla/firefox/fxzxp5er.default-1437352251159/extensions/trash/support@lastpass.com/platform/Linux_x86_64-gcc3
/components: [Exception... "Component returned failure code: 0x80520006 (NS_ERROR_FILE_TARGET_DOES_NOT_EXIST) [nsIFile.remove]"  nsresult: "0x80520006 (NS_ERROR_FILE_TARGET_DOES_NOT_EXIST)"  locat
ion: "JS frame :: resource://gre/modules/addons/XPIProvider.jsm :: recursiveRemove :: line 1524"  data: no] Stack trace: recursiveRemove()@resource://gre/modules/addons/XPIProvider.jsm:1524 < forE
ach()@self-hosted:210 < recursiveRemove()@resource://gre/modules/addons/XPIProvider.jsm:1521 < forEach()@self-hosted:210 < recursiveRemove()@resource://gre/modules/addons/XPIProvider.jsm:1521 < fo
rEach()@self-hosted:210 < recursiveRemove()@resource://gre/modules/addons/XPIProvider.jsm:1521 < forEach()@self-hosted:210 < recursiveRemove()@resource://gre/modules/addons/XPIProvider.jsm:1521 < 
DirInstallLocation_getTrashDir()@resource://gre/modules/addons/XPIProvider.jsm:7521 < DirInstallLocation_installAddon()@resource://gre/modules/addons/XPIProvider.jsm:7543 < XPI_processPendingFileC
hanges/<()@resource://gre/modules/addons/XPIProvider.jsm:2829 < forEach()@self-hosted:210 < XPI_processPendingFileChanges()@resource://gre/modules/addons/XPIProvider.jsm:2658 < XPI_checkForChanges
()@resource://gre/modules/addons/XPIProvider.jsm:3713 < XPI_startup()@resource://gre/modules/addons/XPIProvider.jsm:2288 < callProvider()@resource://gre/modules/AddonManager.jsm:221 < _startProvid
er()@resource://gre/modules/AddonManager.jsm:703 < AMI_startup()@resource://gre/modules/AddonManager.jsm:871 < AMP_startup()@resource://gre/modules/AddonManager.jsm:2511 < AMC_observe()@resource:/
/gre/components/addonManager.js:55 < <file:unknown>
1440640719473   addons.xpi      ERROR   Failed to install staged add-on support@lastpass.com in app-profile: [Exception... "Component returned failure code: 0x80520006 (NS_ERROR_FILE_TARGET_DOES_N
OT_EXIST) [nsIFile.remove]"  nsresult: "0x80520006 (NS_ERROR_FILE_TARGET_DOES_NOT_EXIST)"  location: "JS frame :: resource://gre/modules/addons/XPIProvider.jsm :: recursiveRemove :: line 1524"  da
ta: no] Stack trace: recursiveRemove()@resource://gre/modules/addons/XPIProvider.jsm:1524 < forEach()@self-hosted:210 < recursiveRemove()@resource://gre/modules/addons/XPIProvider.jsm:1521 < forEa
ch()@self-hosted:210 < recursiveRemove()@resource://gre/modules/addons/XPIProvider.jsm:1521 < forEach()@self-hosted:210 < recursiveRemove()@resource://gre/modules/addons/XPIProvider.jsm:1521 < for
Each()@self-hosted:210 < recursiveRemove()@resource://gre/modules/addons/XPIProvider.jsm:1521 < DirInstallLocation_getTrashDir()@resource://gre/modules/addons/XPIProvider.jsm:7521 < DirInstallLoca
tion_installAddon()@resource://gre/modules/addons/XPIProvider.jsm:7543 < XPI_processPendingFileChanges/<()@resource://gre/modules/addons/XPIProvider.jsm:2829 < forEach()@self-hosted:210 < XPI_proc
essPendingFileChanges()@resource://gre/modules/addons/XPIProvider.jsm:2658 < XPI_checkForChanges()@resource://gre/modules/addons/XPIProvider.jsm:3713 < XPI_startup()@resource://gre/modules/addons/
XPIProvider.jsm:2288 < callProvider()@resource://gre/modules/AddonManager.jsm:221 < _startProvider()@resource://gre/modules/AddonManager.jsm:703 < AMI_startup()@resource://gre/modules/AddonManager
.jsm:871 < AMP_startup()@resource://gre/modules/AddonManager.jsm:2511 < AMC_observe()@resource://gre/components/addonManager.js:55 < <file:unknown>

Last edited by jugs (2015-08-27 02:02:38)

graysky · 2015-08-27 19:14:54

Are you using overlayfs? If so, can you try without it and see if you experience the inability to update the addons?

graysky · 2015-08-30 17:47:32

Open invite: please test out PSD 6.00pre.

AUR package: https://aur.archlinux.org/packages/prof … aemon-git/

What's new:
* In this version, psd will be called as a systemd user service rather than a system service.
* This is much more simple and means that
1) There is no more need for /etc/psd.conf and the USERS array therein.
2) Different users can have their own config files that THEY own (~/.psd/psd.conf).
3) Encrypted $HOME should be supported under this model.

NOTE - I have been running it on my system for a few hours; it is not complete at this time. I need to remove some legacy code (support for other init systems), update the man page, etc. The core functionality is unchanged and should be equally robust.

Instruction:
1) Stop psd v5.7x and close your browsers.
2) Build the package linked above and install it.
3) Run psd to create your ~/.psd/psd.conf and then edit it as you normally would.
4) Check it in parse mode `psd p` and if happy with the output, run it via systemd usermode: `systemctl --user start psd` (and optionally enable it).

Note that if you're using overlayfs mode, your user needs to have sudo right to /usr/bin/mount and /usr/bin/umount or else psd will refuse to run in overlayfs mode.

Last edited by graysky (2015-08-30 17:47:53)

graysky · 2015-09-03 19:27:27

No takers? I have been running it for 5 days now without a hiccup. I'd like to get some feedback prior to me going live with an official release.

When you do install it, please report on the update process. It should automatically stop psd (v5.75) for you.

Last edited by graysky (2015-09-04 08:43:30)

mentat · 2015-09-04 08:02:17

Hi graysky,
Thanks for your work.
Love psd but sorry I'm a lazy waiter of the next official stable v6 before tweak all my conf & sys

nasedo · 2015-09-04 17:24:28

I gave it a shot. Seems to work as good as before.

One request, please support XDG directory specification and make psd.conf reside in $XDG_CONFIG_HOME/psd/psd.conf (or just ~/.config/psd/psd.conf). Also, pid files could go into $XDG_RUNTIME_DIR, but that's less important.

Edit: Since the first time I stopped psd.service manually and you asked to report on the update process, I just retried installing profile-sync-daemon, starting it and then installing profile-sync-daemon-git. The system service got automatically stopped like it should. It stayed enabled though, but I guess that's out of scope.

Last edited by nasedo (2015-09-04 17:45:23)

graysky · 2015-09-04 18:22:15

@nasedo - I had it setup to use $XDG_HOME_CONFIG initially but wanted it to reside in ~/.psd since that is less characters for me to type; I will look into $XDG_RUNTIME_DIR. To your edit, yes, as with any Arch package, the PKGBUILD should not modify the init system -- that is up to the user. I did add some code to the post install script that will at least remind the user to remove the symlink upon updating:

...
[[ -h /etc/systemd/system/multi-user.target.wants/psd.service ]] &&
    echo '--> Please remove the now outdated system-level service file: /etc/systemd/system/multi-user.target.wants/psd.service'
...

Last edited by graysky (2015-09-04 18:43:10)

nasedo · 2015-09-04 18:45:46

Please make the config dir somewhat configurable if you find the time.

graysky · 2015-09-06 18:56:38

OK, version 6.00 has gone gold. The pre_install scriptlet should tell you what you need to know.

skuggo · 2015-09-06 19:41:22

After upgrading to 6.0 and trying to systemctl --user start psd.service I get an error message, after some investigation I found the problem. There is no psd.service file whatsoever to be started after the upgrade. Any clues?

Best Regards

graysky · 2015-09-06 20:16:43

@skuggo - The service is provided by the package:

% pacman -Ql profile-sync-daemon | grep service
profile-sync-daemon /usr/lib/systemd/user/psd.service

Post the output of:

psd p

Then of:

systemctl --user start psd.service
systemctl --user status psd.service

Note that all commands should be run as your user (not as root or prefixed with sudo).

Last edited by graysky (2015-09-06 20:31:55)

Arch Linux

#576 2015-05-27 12:32:26

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#577 2015-06-02 23:28:42

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#578 2015-06-02 23:34:35

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#579 2015-06-03 00:47:53

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#580 2015-06-04 17:36:06

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#581 2015-06-04 18:47:54

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#582 2015-06-10 17:43:30

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#583 2015-06-10 18:56:05

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#584 2015-06-23 13:17:10

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#585 2015-07-15 15:10:34

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#586 2015-07-15 19:47:06

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#587 2015-07-25 12:51:44

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#588 2015-07-27 16:20:55

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#589 2015-08-12 12:21:05

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#590 2015-08-27 02:01:00

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#591 2015-08-27 19:14:54

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#592 2015-08-30 17:47:32

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#593 2015-09-03 19:27:27

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#594 2015-09-04 08:02:17

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#595 2015-09-04 17:24:28

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#596 2015-09-04 18:22:15

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#597 2015-09-04 18:45:46

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#598 2015-09-06 18:56:38

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#599 2015-09-06 19:41:22

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#600 2015-09-06 20:16:43

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

Board footer