profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

graysky · 2012-12-06 00:49:00

@yminus - I know that Julian modified the code to work with the gentoo init system. I have it setup here on Arch to work only when called by root. It is preferable to allow the init system to sync or unsync. What happens if you run it as roots. Also, v5.08 is current.

OhneHerren · 2012-12-07 21:55:53

I updated Chromium to Version 23.0.1271.95 (169798), then psd stopped working. I updated psd and the same problem is exhibiting. Psd is syncing an empty "andrew-chromium" folder to /tmp. It seems to be choosing an unusual folder to sync also. I removed my profile and started with a fresh one (waiting for Chromium sync to finish and rebooted before I enabled psd) but it did not help. Here is the output of psd parse:

~  » profile-sync-daemon parse         
Profile-sync-daemon v5.08

Psd will manage the following per /etc/psd.conf settings:

 browser/psname:  chromium/chromium
 owner/group:     andrew/users
 sync target:     /home/andrew/.config/chromium
 tmpfs dir:       /tmp/andrew-chromium
 profile size:    133M

~  » ls ~/.config/chromium                
andrew-chromium                 Safe Browsing Cookies
Certificate Revocation Lists    Safe Browsing Cookies-journal
chromium                        Safe Browsing Csd Whitelist
chromium-backup                 Safe Browsing Download
Default                         Safe Browsing Download Whitelist
Dictionaries                    Service State
First Run                       SingletonCookie
Local State                     SingletonLock
Safe Browsing Bloom             SingletonSocket
Safe Browsing Bloom Prefix Set  Temp

~  » ls ~/.config/chromium/chromium/
andrew-chromium               Safe Browsing Cookies
Certificate Revocation Lists  Safe Browsing Cookies-journal
chromium-backup               Safe Browsing Csd Whitelist_new
Default                       Safe Browsing Download_new
Dictionaries                  Safe Browsing Download Whitelist_new
First Run                     SingletonCookie
Local State                   SingletonLock
Safe Browsing Bloom_new       SingletonSocket

I have no idea what the above "chromium/chromium" folder is but it wasn't there in the last Chromium version. Like I said this is a fresh profile.

OhneHerren · 2012-12-07 23:28:30

So I deleted the chromium .config folder for a third time and tried again. This time there was no phantom chromium/chromium folder and psd is working as intended. I notice that user DragonLich had a similar problem here. Perhaps this is a bug with chromium.

graysky · 2012-12-08 00:13:57

Glad you sorted it out; enjoy psd.

alain · 2012-12-09 11:56:01

Hello Graysky,

Just to say you that all is working for me.

Thanks Again.
Best regards.

Alain

graysky · 2012-12-09 12:09:36

Glad to hear it! Thanks for helping me make psd a more featureful and robust product.

alain · 2012-12-09 12:15:19

You're welcome

pingpong · 2012-12-09 19:56:37

Unfortunately, profile-cleaner is broken for firefox for me:

~ [i]> profile-cleaner f
profile-cleaner v1.94

 Cleaning profile for firefox
cat: /tmp/work2do.firefox: No such file or directory

grep: /tmp/accounting.firefox: No such file or directory
 Profile(s) for firefox reduced by  Mbytes.

looking for the fix. Thanks

graysky · 2012-12-09 20:34:18

@pingpong - plz post to the profile-cleaner thread. This one is for psd. When you do, please show me the contents of ~/.mozilla/firefox/profiles.ini

Last edited by graysky (2012-12-09 20:42:10)

freestyler7 · 2012-12-09 22:13:16

Psd.service took around 20+ secs according to systemd-blame.
Manually starting the service while system is idle: 3 secs.

I tried the timer option and it dropped to around 8 secs.
In case anyone don't know how to set a timer for a unit/service, here are some instructions.

Create "psd.timer" file under "/usr/lib/systemd/system/"

# nano /usr/lib/systemd/system/psd.timer

Add the following contents to it:

[Timer]
OnBootSec=70

[Install]
WantedBy=multi-user.target

We have to specify which .service to start when our criteria meets. (70 secs after bootup)
By default the .timer will search for the corresponding .service file with the same name. So no need to change anything else in our file, psd.service will be selected in our case.
Disable psd.service & enable our new psd.timer

# systemctl disable psd.service
# systemctl enable psd.timer

You have to find the best timer value for your system.
Ideally, the psd.service should kick in the moment your system goes idle after booting up. (after dropping to the deskop, the cpu might still be used till all the desktop enviroment stuff are complete)

Last edited by freestyler7 (2012-12-09 22:17:35)

graysky · 2012-12-09 22:35:00

Thanks for sharing those instructions. The only 'danger' with it is that if your browser is open, psd will refuse to start. I dunno if the timer will attempt to restart or not. Never used one.

GNA · 2012-12-11 13:26:16

I must admit, I did not read the whole 12 pages.
It works well, if started manually.
If I enable psd in systemd and restart, my whole profile is reset to default settings. Everything is gone except addons. Luckily I followed your advice and backed everything up beforehand.
Is there anyone else with this problem? How can I fix this?

Last edited by GNA (2012-12-11 13:33:27)

graysky · 2012-12-11 20:18:00

@GNA -
1) Enable psd.service
2) Reboot (why? because I suspect your problem is that you hdd is slowly starting psd along with other startup scripts and that you are using the browser before it finishes syncing but let's test).
3) Post the output of `systemd-analyze blame | grep psd`

GNA · 2012-12-13 08:02:22

ok, this is strange...
It killed my profile 4 times. Then it stopped doing so. I have no idea why o.O .
Should it begin anew, I will post again.
Thank you .

GNA · 2012-12-19 07:24:33

Could you prevent firefox from starting while you sync the profiles? When running systems with low overhead (I just use i3, no desktop environment) I risk to load incomplete profiles. If I do, I have to copy my backup again.

graysky · 2012-12-19 09:56:21

I don't know how to exercise control over another app like that. I do have psd refuse to start. Itself if any managed browser is running, but I don't know how I would have it control what the user does while the initial sync is in progress. Suggestions are welcome.

aesiris · 2012-12-19 10:32:33

One option is to revoke permissions for the user before rsyncing, than give them back when finished.
i.e change

rsync blah

to

chmod -wrx /tmp/...
rsync blah
chmod +wrx /tmp/...

For firefox it works just fine, it complains with: "Firefox is already running, but is not responding. To open a new window, you must first close the existing Firefox process, or restart your system."

Don't know how it would affect other browsers

Last edited by aesiris (2012-12-19 10:33:13)

graysky · 2012-12-19 19:38:25

I like it. I've tested on chromium and it works... plz take a look at the github repo. Needs more with other browsers before going live.

aesiris · 2012-12-20 13:01:38

Mm, I was completely wrong

rsync sets permissions too early here, I can access the profile before it is complete.

Even the "chmod +wrx" part is wrong: it opens it for everybody, nono! better leave rsync manage the right ones or use mode 700 to be safe.

My next idea (I hope it is better than the provious one) is to rsync to a different location under /tmp and then mv it at the end; in the meanwhile the symlink is broken and the browser can not read the profile.

graysky · 2012-12-20 14:29:07

v5.10-1 reverts the behavior.

GNA · 2012-12-26 12:20:06

this problem should be solved by shoving some lines around:
first do the backup
then rsync to ram
then symlink

Just move

ln -s "$VOLATILE/$user-$browser$suffix" "$DIR"

after the initial sync.

This could be improved by delaying firefox until the link is created, but I don't know if packages are allowed to mess around with other packages like that.

Last edited by GNA (2012-12-26 12:27:51)

Doctor Colossus · 2012-12-27 06:52:53

An idea for a kludge
(or maybe something similar could even be integrated into profile-sync-daemon):

Users, instead directly running firefox, could run a script which checks whether or not rsync's busy
and if it is waits for it to finish before executing firefox.

Last edited by Doctor Colossus (2012-12-27 06:53:45)

graysky · 2012-12-27 19:44:48

Overly simplified example using chromium:

DIR=$HOME/.config/chromium
BACKUP=$DIR-backup

1) Make empty tmpfs container if not there:
 e.g. /tmp/$USER-chromium

2a) Move profile to profile-backup if not there:
 e.g. $DIR --> $BACKUP
2b) Create link to tmpfs
 e.g. /tmp/$USER-chromium --> $DIR

3) Sync the HDD backup to  tmpfs
 e.g, $BACKUP/ /tmp/$USER-chromium/

I believe you are proposing an order of: 1, 3, 2a, 2b. The code would need to adjusted to sync $DIR not $BACKUP for the initial sync. So if rsync is active when the browser is started, the browser would still read $DIR. What I don't know is what happens if files are modified during the simultaneous browser usage/rsync step...?

aesiris · 2012-12-28 08:05:31

I have been using this [github] without problems.
It creates an empty and not readable profile dir, then performs rync to a tmp location and at the and put it in the right place.

I will be away for about a week: happy new year!

graysky · 2012-12-28 11:09:08

Maybe it's because I didn't get much sleep, but I you still have step 2a after step 1...

Arch Linux

#276 2012-12-06 00:49:00

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#277 2012-12-07 21:55:53

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#278 2012-12-07 23:28:30

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#279 2012-12-08 00:13:57

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#280 2012-12-09 11:56:01

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#281 2012-12-09 12:09:36

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#282 2012-12-09 12:15:19

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#283 2012-12-09 19:56:37

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#284 2012-12-09 20:34:18

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#285 2012-12-09 22:13:16

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#286 2012-12-09 22:35:00

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#287 2012-12-11 13:26:16

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#288 2012-12-11 20:18:00

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#289 2012-12-13 08:02:22

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#290 2012-12-19 07:24:33

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#291 2012-12-19 09:56:21

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#292 2012-12-19 10:32:33

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#293 2012-12-19 19:38:25

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#294 2012-12-20 13:01:38

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#295 2012-12-20 14:29:07

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#296 2012-12-26 12:20:06

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#297 2012-12-27 06:52:53

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#298 2012-12-27 19:44:48

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#299 2012-12-28 08:05:31

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#300 2012-12-28 11:09:08

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

Board footer