You are not logged in.

#301 2013-01-01 17:34:00

GNA
Member
Registered: 2011-02-15
Posts: 57

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

graysky wrote:

Overly simplified example using chromium:

DIR=$HOME/.config/chromium
BACKUP=$DIR-backup

1) Make empty tmpfs container if not there:
 e.g. /tmp/$USER-chromium

2a) Move profile to profile-backup if not there:
 e.g. $DIR --> $BACKUP
2b) Create link to tmpfs
 e.g. /tmp/$USER-chromium --> $DIR

3) Sync the HDD backup to  tmpfs
 e.g, $BACKUP/ /tmp/$USER-chromium/

I believe you are proposing an order of: 1, 3, 2a, 2b.  The code would need to adjusted to sync $DIR not $BACKUP for the initial sync.  So if rsync is active when the browser is started, the browser would still read $DIR.  What I don't know is what happens if files are modified during the simultaneous browser usage/rsync step...?

What I am proposing is to change the order like this:
1, 2a, 3, 2b

So after 2a until reaching 2b, firefox won't find the profile folder, as it was renamed to profile-backup. So it will display something like: "Profile not found". So while the critical part is running, there just wont be any profile from the perspective of firefox. So if you mistakeably try to start firefox too early, it will end with you having to click away an annoying message but not breaking anything. This could be made even better by delaying firefox until this process has finished.

Last edited by GNA (2013-01-01 17:35:19)

Offline

#302 2013-01-02 07:40:44

dodo3773
Member
Registered: 2011-03-17
Posts: 801

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

Thanks for another great project graysky. I seem to be using more and more things from you. The large benefit of psd for me is that I constantly empty my cache. Now with this it went from a few seconds to less than one. I too am experiencing slow boot times though. My "systemd-analyze blame" puts psd.service at 22209ms. I would rather wait the extra time and continue to use psd though. It's worth it. Just wanted to let you know / verify with my experience what freestyler7 is saying.

Offline

#303 2013-01-12 10:55:18

GNA
Member
Registered: 2011-02-15
Posts: 57

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

Ok, here is my patch:

move

ln -s "$VOLATILE/$user-$browser$suffix" "$DIR"                                          
chown -h $user:$group "$DIR" 

after this line:

rsync -aog --delay-updates "$BACKUP/" "$DIR/"

and change that lane to

 rsync -aog --delay-updates "$BACKUP/" "$VOLATILE/$user-$browser$suffix/"  

Done.

What does it do? See post #301.

For future reference, does anyone know if it is possible to commit diff files for aur packages?

Offline

#304 2013-01-19 13:50:15

barraponto
Member
From: Campinas, Brazil
Registered: 2009-10-15
Posts: 29
Website

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

Would we benefit from running psd as a user daemon? I saw instructions for MPD at https://wiki.archlinux.org/index.php/MP … th_systemd and thought we could shave some boot time by going that way.

Offline

#305 2013-01-19 13:51:19

graysky
Wiki Maintainer
From: :wq
Registered: 2008-12-01
Posts: 10,595
Website

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

barraponto wrote:

Would we benefit from running psd as a user daemon? I saw instructions for MPD at https://wiki.archlinux.org/index.php/MP … th_systemd and thought we could shave some boot time by going that way.

Bad idea.

1) No benifits.
2) Hardcoded to run as root so it can maintain ownership of sync targets.


CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#306 2013-01-24 16:26:44

fradeve
Member
Registered: 2013-01-24
Posts: 9

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

Hi guys,
this is my first month with Arch. I noticed that Firefox was slightly slower on the same machine in comparison with other distros, so I decided to try PSD. This is my conf:

/etc/fstab

tmpfs       /tmp    tmpfs           nodev,nosuid                    0  0

/etc/psd.conf

# List users separated by spaces whose browser(s) profile(s) will get symlinked
# and sync'ed to tmpfs
# Do NOT list a user twice!
#
# Example
# USERS="facade happy"
USERS="fradeve"

# List browsers separated by spaces to include in the sync. Useful if you do not
# wish to have all possible browser profiles sync'ed
#
# Possible values:
#               chromium
#               conkeror.mozdev.org
#               firefox
#               google-chrome
#               heftig-aurora
#               midori
#               opera
#               opera-next
#               qupzilla
#
# Uncomment and select which browsers to manage if you wish
# Otherwise all available/supported browsers will be managed
BROWSERS="firefox midori"

# Define where browser profiles will reside in tmpfs
# This location must be mounted to tmpfs and MUST have permissions of 777
# Use NO trailing backslash!
# If running a standard ARCH install the following default is fine
VOLATILE="/tmp"

profile-sync-daemon parse

Profile-sync-daemon v5.10

Psd will manage the following per /etc/psd.conf settings:

 browser/psname:  firefox/firefox
 owner/group:     fradeve/users
 sync target:     /home/fradeve/.mozilla/firefox/f5cnrqwq.default-1357907171863
 tmpfs dir:       /tmp/fradeve-firefox-f5cnrqwq.default-1357907171863
 profile size:    59M

 browser/psname:  midori/midori
 owner/group:     fradeve/users
 sync target:     /home/fradeve/.config/midori
 tmpfs dir:       /tmp/fradeve-midori
 profile size:    1,4M

So, I enabled PSD with

sudo systemctl enable psd.service

and rebooted.

When loggin in XFCE4, when launching FF, it complains about the missing user profile, and FF refuses to launch.

Any hint?

Thank you

Last edited by fradeve (2013-01-24 18:19:44)

Offline

#307 2013-01-24 17:16:35

aesiris
Member
Registered: 2012-02-25
Posts: 97

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

fradeve wrote:

Hi guys,
this is my first month with Arch. I noticed that Firefox was slightly slower on the same machine in comparison with other distros, so I decided to try PSD. This is my conf:

/etc/fstab
[...]

By default systemd mounts /tmp as a tmpfs, you do not need this line in /etc/fstab; but I don't think this is a problem.

[...]
Every login on XFCE4, when launching FF, it complains about the missing user profile, and FF refuses to launch.

Any hint?

Thank you

Did you run firefox before psd.service was finished?
Maybe you will need to recover your profile from "/home/fradeve/.mozilla/firefox/f5cnrqwq.default-1357907171863-backup"

Please provide also the output of these commands:

systemctl status psd.service
journalctl _SYSTEMD_UNIT=psd.service
ls -l /home/fradeve/.mozilla/firefox/

Offline

#308 2013-01-24 18:38:00

fradeve
Member
Registered: 2013-01-24
Posts: 9

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

Hmmm it seems that things are working now. This is very strange because for 2 days FF gave me almost randomly the above mentioned error. The only thing I've done is

sudo systemctl disable psd.service

reboot.

sudo systemctl enable psd.service

reboot.

Btw:
systemctl status psd.service

psd.service - Profile-sync-daemon
          Loaded: loaded (/usr/lib/systemd/system/psd.service; enabled)
          Active: active (exited) since gio 2013-01-24 20:28:17 CET
         Process: 255 ExecStart=/usr/bin/profile-sync-daemon sync (code=exited, status=0/SUCCESS)

journalctl _SYSTEMD_UNIT=psd.service
empty

ls -l /home/fradeve/.mozilla/firefox/

drwx------ 17 fradeve users 12288 24 gen 19.33 f5cnrqwq.default-1357907171863
-rw-r--r--  1 fradeve users   132 11 gen 13.26 profiles.ini

Since I start FF about a couple of minutes after logging in, I'm pretty sure that psd.service has exited at that time.
Is it possible that, for some reasons, psd.service takes longer to end sometimes after logging in? It seems the only possibile scenario to me.

Offline

#309 2013-01-24 20:49:50

graysky
Wiki Maintainer
From: :wq
Registered: 2008-12-01
Posts: 10,595
Website

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

fradeve wrote:

Since I start FF about a couple of minutes after logging in, I'm pretty sure that psd.service has exited at that time.
Is it possible that, for some reasons, psd.service takes longer to end sometimes after logging in? It seems the only possibile scenario to me.

Yes, it is possible.  Since systemd parralellizes everything, many services start at boot and all want disk IO to complete.  It could be that you launched the browser before psd finished syncing.  Post the output of `systemd-analyze blame` to give a rough idea.  My system uses a pretty fast SSD so it is not an issue for me:

% systemd-analyze blame
  3693ms netctl@static.service
   346ms psd.service
   109ms ufw.service
    83ms systemd-udev-trigger.service
    71ms mnt-data.mount
    57ms systemd-logind.service
    56ms lm_sensors.service
    46ms systemd-modules-load.service
    43ms tmp.mount
    43ms systemd-remount-fs.service
    43ms systemd-sysctl.service
    40ms gpm.service
    40ms sys-kernel-debug.mount
    40ms dev-mqueue.mount
    39ms systemd-vconsole-setup.service
    39ms cpupower.service
    36ms dev-hugepages.mount
    36ms systemd-tmpfiles-setup.service
    33ms systemd-udevd.service
    26ms ntpd.service
    26ms polkit.service
    23ms systemd-user-sessions.service
    21ms rtkit-daemon.service
    20ms boot.mount
    14ms udisks2.service
    10ms home.mount
     7ms upower.service
     6ms var.mount
     6ms scratch.mount
     0ms sys-fs-fuse-connections.mount
     0ms home-facade-.cache.mount

CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#310 2013-01-24 21:13:09

fradeve
Member
Registered: 2013-01-24
Posts: 9

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

Probably is it.

I think it could be useful -- since it is a pretty interesting command to describe how things works -- adding systemd-analyze blame reference to the PSD wiki page, as a troubleshooting tip, to avoid boring users on the forum smile

This is my boot chart:

   903ms wicd.service
   540ms psd.service
   341ms gdm.service
   339ms systemd-logind.service
   186ms systemd-vconsole-setup.service
   146ms systemd-udev-trigger.service
   110ms systemd-user-sessions.service
    86ms accounts-daemon.service
    82ms systemd-remount-fs.service
    72ms colord.service
    60ms polkit.service
    55ms udisks2.service
    53ms systemd-udevd.service
    50ms sys-kernel-debug.mount
    40ms dev-mqueue.mount
    39ms systemd-tmpfiles-setup.service
    34ms dev-hugepages.mount
    33ms systemd-sysctl.service
    23ms tmp.mount
    17ms upower.service
    16ms udisks.service
    15ms ntpd.service
     8ms home.mount
     7ms rtkit-daemon.service
     6ms proc-sys-fs-binfmt_misc.mount
     2ms sys-fs-fuse-connections.mount

Thank you guys!

Offline

#311 2013-01-24 21:18:21

graysky
Wiki Maintainer
From: :wq
Registered: 2008-12-01
Posts: 10,595
Website

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

540 ms is pretty quick... I doubt that you are starting firefox too fast unless you are logging in and instantaneously launching the browser (autostart)?


CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#312 2013-01-24 21:35:39

graysky
Wiki Maintainer
From: :wq
Registered: 2008-12-01
Posts: 10,595
Website

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

GNA wrote:

Ok, here is my patch:
...

Seems an okay fix at first pass.  Others care to test the patch proposed by GNA... preferably those with firefox?  Working fine with chromium.

--- /usr/bin/profile-sync-daemon  2012-12-20 08:53:11.000000000 -0500
+++ /usr/bin/profile-sync-daemon  2013-01-24 16:32:46.668852037 -0500
@@ -2,7 +2,7 @@
 # By graysky <graysky AT archlinux DOT us>
 # Inspired by some code originally  written by Colin Verot
 export BLD="\e[01m" RED="\e[01;31m" GRN="\e[01;32m" YLW="\e[01;33m" NRM="\e[00m"
-VERS="5.10"
+VERS="5.11"

 if [[ ! -f /etc/psd.conf ]]; then
  echo "Cannot find /etc/psd.conf so bailing.  Reinstall package to use Profile-sync-daemon." # nothing to do if there is no conf file
@@ -255,8 +255,6 @@
          # backup target and link to tmpfs container
          if [[ $(readlink "$DIR") != "$VOLATILE/$user-$browser$suffix" ]]; then
            mv "$DIR" "$BACKUP"
-           ln -s "$VOLATILE/$user-$browser$suffix" "$DIR"
-           chown -h $user:$group "$DIR"
          fi

          # sync the tmpfs targets to the disc
@@ -265,7 +263,9 @@
          else
            # initial sync
            # keep user from launching browser while rsync is active
-           rsync -aog --delay-updates "$BACKUP/" "$DIR/"
+           rsync -aog --delay-updates "$BACKUP/" "$VOLATILE/$user-$browser$suffix/"
+           ln -s "$VOLATILE/$user-$browser$suffix" "$DIR"
+           chown -h $user:$group "$DIR"
            touch "$DIR/.flagged"
          fi
        fi

CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#313 2013-01-26 10:28:15

fradeve
Member
Registered: 2013-01-24
Posts: 9

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

graysky wrote:

540 ms is pretty quick... I doubt that you are starting firefox too fast unless you are logging in and instantaneously launching the browser (autostart)?

Sure, I'm not so fast smile

Anyway, after a couple of reboots, I noticed the PSD strange behaviour: it seems to start correctly or give error almost randomly; so it can happen that FF works as expected and the PSD daemon was running, or the contrary. I omitted to say that I'm using an encrypted home partition, configured as reported in https://wiki.archlinux.org/index.php/EC … tfs_module, using eCryptfs, PAM, GDM and Xfce4. So, I suppose that the problem is that PSD daemon cannot read browser data until PAM mounts eCryptfs private dir, and the concurrency between the two processes seems to generate random results; sometime it works, sometimes it doesn't.

So, I tried an alternative solution:

sudo systemctl disable psd.service

added the following line in /etc/sudoers

fradeve ALL=(ALL:ALL) NOPASSWD:/usr/bin/systemctl

added the following in $HOME/.config/autostart/

[Desktop Entry]
Encoding=UTF-8
Version=0.9.4
Type=Application
Name=psd.service
Comment=
Exec=sudo /usr/bin/systemctl start psd.service
OnlyShowIn=XFCE;
StartupNotify=false
Terminal=false
Hidden=false

I know, it's awful but -- hey -- it works.

Any comment? Is it too dangerous to give a user sudo permissions without password request on systemctl?

Offline

#314 2013-01-26 11:16:25

graysky
Wiki Maintainer
From: :wq
Registered: 2008-12-01
Posts: 10,595
Website

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

An encrypted $HOME would muck things up and require psd to start after it was in place as you describe.  Your solution works and I don't think it's dangerous.  How many users have this in their /etc/sudoers

%wheel ALL=(ALL) NOPASSWD: ALL

A more universal solution would be to make psd wait for wherever mount your encrypted $HOME.  It's not clear to me what that is and how to do it in the psd.service file... suggestions are welcome.

Last edited by graysky (2013-01-26 11:18:04)


CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#315 2013-02-27 12:56:22

Lockheed
Member
Registered: 2010-03-16
Posts: 1,512

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

I have some questions about the inner workings of this tool:

1. Does it load profile dirs into RAM even if the browser is not running, thus permanently occupying portion of memory?

2. I have external scripts that delete older cache files from opera profile. It is set to run every hour. If that occurs, won't the deleted cache files be restored back form memory upon daemon shutdown?

Offline

#316 2013-02-27 17:53:05

aesiris
Member
Registered: 2012-02-25
Posts: 97

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

Lockheed wrote:

I have some questions about the inner workings of this tool:

1. Does it load profile dirs into RAM even if the browser is not running, thus permanently occupying portion of memory?

Yes.
But if you are on RAM pressure it will probably be swapped out.

Lockheed wrote:

2. I have external scripts that delete older cache files from opera profile. It is set to run every hour. If that occurs, won't the deleted cache files be restored back form memory upon daemon shutdown?

No.
When the service is started your profile directory is a link to a directory in RAM; only a backup is kept on disk and it will be restored in case of ungraceful stop (for example power failure, crash, etc.).
You should just check that psd.service is completely started before running your script.

Offline

#317 2013-02-27 19:44:55

graysky
Wiki Maintainer
From: :wq
Registered: 2008-12-01
Posts: 10,595
Website

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

arseris is right about the first question.  Depending on your script, the answer to the 2nd question may also be yes.  In other words, psd symlinks your profile dir.  If your script is just pointing to the profile dir (the symlink) to delete stuff, it should happily do so blissfully unaware that it is operating on a symlink, rather than on a 'live' directory.  There should be no need to stop psd before you run your script is what I'm saying.


CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#318 2013-02-27 20:08:29

Lockheed
Member
Registered: 2010-03-16
Posts: 1,512

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

Ok, guys. That helped alot. Thank you.

Offline

#319 2013-03-17 16:34:34

FarStar07
Member
Registered: 2012-04-02
Posts: 48

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

Hello graysky,

For your information, there is a change in the profile path of Qupzilla:

QupZilla 1.4.x
------------------
Very important change is that config directory is now in ~/.config/qupzilla by default. It will, however, use the old ~/.qupzilla if it exists (and ~/.config/qupzilla does not).
While it is (and will always be) backwards compatible, I'd recommend you to manually move ~/.qupzilla into ~/.config/qupzilla.

Source

Offline

#320 2013-03-17 18:25:58

graysky
Wiki Maintainer
From: :wq
Registered: 2008-12-01
Posts: 10,595
Website

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

Thanks, FarStar.  v5.31 will handle this.

EDIT: Just pushed to the AUR.

Last edited by graysky (2013-03-17 23:35:31)


CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#321 2013-03-18 05:25:58

FarStar07
Member
Registered: 2012-04-02
Posts: 48

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

Thanks wink

Offline

#322 2013-03-18 10:25:05

nperry
Member
Registered: 2010-05-16
Posts: 86

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

Is is normal for psd and psd-rsync to take such a long time at boot?

─╼ sudo systemd-analyze blame
  7297ms psd.service
  1131ms psd-resync.service
   238ms systemd-fsck@dev-disk-by\x2duuid-431d523f\x2d52ec\x2d44a7\x2dae2d\x2d38c389cb80a5.service
   237ms teamviewerd.service
   215ms systemd-modules-load.service
   214ms var.mount
   210ms systemd-vconsole-setup.service
   198ms systemd-udevd.service
   165ms systemd-sysctl.service
   157ms systemd-udev-trigger.service
   146ms dhcpcd.service
   143ms systemd-remount-fs.service
   127ms systemd-fsck@dev-disk-by\x2duuid-7be0c8b4\x2d64f9\x2d405e\x2db0f3\x2d7e2c544bad29.service
   110ms home.mount
   103ms systemd-random-seed-load.service
   102ms systemd-fsck@dev-disk-by\x2duuid-a9de3235\x2d64de\x2d4f3d\x2db4f2\x2d1a9c01b5c272.service
   100ms alsa-restore.service
    80ms systemd-tmpfiles-setup.service
    74ms systemd-logind.service
    60ms dev-mqueue.mount
    49ms systemd-user-sessions.service
    44ms boot.mount
    37ms dev-disk-by\x2duuid-c617c491\x2deefd\x2d4b9d\x2daf5d\x2d4b6df908aadd.swap
    34ms dev-hugepages.mount
    28ms systemd-journal-flush.service
    27ms sys-kernel-debug.mount
    26ms systemd-tmpfiles-clean.service
    24ms tmp.mount
     3ms home-neil-.cache.mount

Offline

#323 2013-03-18 12:10:32

FarStar07
Member
Registered: 2012-04-02
Posts: 48

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

nperry wrote:

Is is normal for psd and psd-rsync to take such a long time at boot?

It depends on the size of the browser user profile(s).
You can clear the browser cache on exit to decrease this boot time.
You can also use graysky's profile-cleaner script to reduce the size of the profile(s).
For me, for Firefox + Opera + Qupzilla =>  1482ms psd.service

Last edited by FarStar07 (2013-03-18 12:21:30)

Offline

#324 2013-03-18 14:04:08

nperry
Member
Registered: 2010-05-16
Posts: 86

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

I've just used the profile cleaner and I've now reduced it, but the load time still seems to be a bit high.

─╼ profile-sync-daemon parse
Profile-sync-daemon v5.30 on Arch Linux.

Daemon file /run/psd is present.
Service is currently active.

Psd will manage the following per /etc/psd.conf settings:

browser/psname:  chromium/chromium
owner/group:     neil/users
sync target:     /home/neil/.config/chromium
tmpfs dir:       /tmp/neil-chromium
profile size:    90M

browser/psname:  firefox/firefox
owner/group:     neil/users
sync target:     /home/neil/.mozilla/firefox/a62kelqm.default
tmpfs dir:       /tmp/neil-firefox-a62kelqm.default
profile size:    32M

I'm not to sure if this is because my /home partition is on a HDD and / is on an SSD.

When you say clearing cache on exit, is there a way to do this for chromium and Firefox, automated?

Offline

#325 2013-03-18 14:41:07

FarStar07
Member
Registered: 2012-04-02
Posts: 48

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

My Home partition is on an SSD. For me, after cleaning:

$ profile-sync-daemon parse

Profile-sync-daemon v5.31 on Chakra Linux (2013.01.23 - Benz).

Daemon file /run/psd is present.                                                                                                                                                         
Service is currently active.

Psd will manage the following per /etc/psd.conf settings:

browser/psname:  qupzilla/qupzilla
owner/group:     farstar/users
sync target:     /home/farstar/.config/qupzilla
tmpfs dir:       /tmp/farstar-qupzilla
profile size:    1,2M

browser/psname:  firefox/firefox
owner/group:     farstar/users
sync target:     /home/farstar/.mozilla/firefox/dgdg4wji.default
tmpfs dir:       /tmp/farstar-firefox-dgdg4wji.default
profile size:    15M

browser/psname:  opera/opera
owner/group:     farstar/users
sync target:     /home/farstar/.opera
tmpfs dir:       /tmp/farstar-opera
profile size:    16M

$ sudo systemd-analyze blame
  999ms psd.service

Clearing cache on exit:

- For Firefox: Preferences > Privacy > History
Choose "Use custom settings for history"
Check "clear history when Firefox closes"
In Settings, check "cache".

-  For Chrome, I don't know because I don't use it.

Last edited by FarStar07 (2013-03-18 15:09:28)

Offline

Board footer

Powered by FluxBB