profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

graysky · 2012-03-05 23:50:24

30 sec? How many profiles are you syncing and how large are they? Guess you can background the daemon but you'll have the same problem of having to wait for it to sync before launching your browser.

thetrivialstuff · 2012-03-06 01:09:48

qnd wrote:

Actually, foregrounding the daemon adds a rather hefty 30 seconds to boot. Is there something that could be done about it, or will it have to be an unfortunate side effect of enjoying the benefits of this daemon, problem-free?

psd could be modified to foreground-copy only the important stuff (settings files, bookmarks, etc.), then background itself and lazily copy the cache. I don't know how well chrome would tolerate being started with its cache directory only partly intact (since others have mentioned it being really sensitive about having it modified), but Firefox and Opera certainly don't care.

To make sure the browser didn't give up on the entire cache as corrupt and nuke it, you could start that lazy copy in order of most recent mtime first -- that would ensure that the cache index files probably make it across before the user has a chance to log in and start the browser. Then the next-most-recently modified files would likely belong to whatever tabs were open, so those should be in place by the time the browser finishes loading, and who cares if the rest takes another minute or two...

Or, as a much lower-tech solution, you could hibernate instead of shutting the machine down, thus avoiding booting at all

~Felix.

qnd · 2012-03-06 05:56:32

graysky wrote:

30 sec? How many profiles are you syncing and how large are they? Guess you can background the daemon but you'll have the same problem of having to wait for it to sync before launching your browser.

Gah, turns out it was an user error. That question about how many profiles and how large they were got me to checking it out (I have only one browser and one profile), and my profile had grown to the size of 344.9 MiB.

Hmm, how could I make it smaller... A-ha! "Clear recent history" (with time range set to "everything" and selecting browsing history, cookies, cache, active logins, site preferences, so everything except saved sessions created by an addon) and the size goes down to 85.1 MiB and the daemon takes now only 2 seconds to start. Doesn't seem entirely proportional to the profile size, so maybe there was some funny business going on with something taking long to sync.

Well, at least others can learn from my mistakes. Remember to foreground your daemon and clear your browser junk.

EDIT: Also it might be worth considering to put a manual limit on your browser cache size to keep the boot time reasonable.

Last edited by qnd (2012-03-06 10:05:25)

graysky · 2012-03-06 10:15:42

Also, clean your Firefox dbs... http://mozillalinks.org/2009/07/vacuum- … erformance

boswbr25 · 2012-03-13 01:09:45

Hi,

I'm not sure if this is the best place to post this but I am trying to start using psd and I having a problem trying to change the cache directory. Let me know if this is not in realm of the profile-sync-daemon (I wasn't sure how related the two are) and I will post my own topic.

Anyways, I was going through the Wiki and trying the step on Chromium Tweaks to move the cache directory and I get the following error:

 [boswbr25@boswbr25-MX46 ~]$ chromium --disk-cache-dir=/home/boswbr25/.config/chromium/cache
[1364:1371:1067464391:ERROR:object_proxy.cc(239)] Failed to call method: org.freedesktop.DBus.Error.Spawn.ChildExited: Launch helper exited with unknown return code 1
Created new window in existing browser session.

First off, I was wondering if anyone knew how to get this to work, and also if I can't get this to work, would there be any point of using psd? I'd like to keep the browser in RAM bc I have more of that than I do HD speed and CPU. I know it works a lot smoother that way from running Puppy Linux. I just wasn't sure if it will still work if I have to keep my cache on physical disk.

Sorry if this is a dumb question. Thanks in advance for any help

~Boz~

Edit: I should mention that I first tried using the exact code mentioned in the wiki

 chromium --disk-cache-dir=/tmp/cache

both with and w/o chromium running first, and when that didn't work, I found this forum post and tried to do the things mentioned in it which is where the output above comes from.

Edit: I just noticed that it is not only when I try to change the cache directory that this happens. I tried to edit the scroll speed of mouse wheel with

 chromium --scroll-pixels=200

and got the same error.

Last edited by boswbr25 (2012-03-13 02:16:16)

graysky · 2012-03-13 13:31:16

@boswbr25 - What happens if you stop psd to minimize potential variables and define your --disk-cache-dir=/path/to/somewhere/OUTSIDE_of_your_profile

This is the keyboard shortcut I use with no problems:

chromium --enable-seccomp-sandbox --scroll-pixels=230 --memory-model=low --disk-cache-dir=/tmp/cache --user-agent="Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/17.0.963.79 Safari/535.7"

boswbr25 · 2012-03-13 17:27:48

Thanks for the support.

I will try this again when I get home from work but I am pretty sure I will get the same error because when I first my question posted on here, I was changing the cache before I had even started the psd daemon. I wasn't sure if it would do me any good to run psd without having the cache directory moved.

Also, after I did start running (or attempting to run the psd daemon), it wasn't showing up in my processes. I added psd into the DAEMONS section of my rc.conf and when I booted, the process wasn't showing up. I then tried to do a rc.d start psd but it still wasn't showing up. I didn't really investigate this too much since I couldn't figure out the cache problem either and it was getting late last night.

Let me know if there's anything off hand you can think of that I might be doing wrong. I know there is not much information to go on. I will provide some more details from my system when I am home.

~Boz~

graysky · 2012-03-13 20:12:15

Right... psd does not continually run like sshd or ntpd. It is literally a cron job triggering a single (simple) script. The daemon is somewhat misleading in this regard. To see that the daemon is active, simply:

$ ls -l /run/daemons
total 0
-rw-r--r-- 1 root root 0 Mar 13 14:51 arch32
-rw-r--r-- 1 root root 0 Mar 13 08:01 crond
-rw-r--r-- 1 root root 0 Mar 13 08:01 dbus
-rw-r--r-- 1 root root 0 Mar 13 14:51 distccd
-rw-r--r-- 1 root root 0 Mar 13 08:01 fancontrol
-rw-r--r-- 1 root root 0 Mar 13 08:01 gpm
-rw-r--r-- 1 root root 0 Mar 13 08:01 netfs
-rw-r--r-- 1 root root 0 Mar 13 08:01 network
-rw-r--r-- 1 root root 0 Mar 13 08:01 ntpd
-rw-r--r-- 1 root root 0 Mar 13 08:01 psd
-rw-r--r-- 1 root root 0 Mar 13 08:01 samba
-rw-r--r-- 1 root root 0 Mar 13 08:01 sensors
-rw-r--r-- 1 root root 0 Mar 13 08:01 syslog-ng

You will see it there.

boswbr25 · 2012-03-14 02:20:45

So I think it is working right even with the Error message. I changed the cache directory and checked it and the files are getting placed in there. I was assuming the command wasn't working since I was getting an Error message but everything appears to be functioning as it is supposed to.

I just changed the command for the application in my menu so the cache is changed and I don't have to see the error.

Thanks for you help. Sorry that I didn't investigate it enough before asking.

If anyone else gets that error and figures out what it means let me know.

~Boz~

duquesnc · 2012-03-14 21:58:48

It works really well and requires almost no configuration. Thank you, this is a great package!

AugustePop · 2012-03-17 21:11:21

would we get better performance to use bind mount instead of symlinks?

as we are already running it as a daemon, bind mount should not have privilege problems.

graysky · 2012-03-17 21:32:22

@aug-I don't think it would make a difference.

AugustePop · 2012-03-22 04:46:56

graysky wrote:

@aug-I don't think it would make a difference.

web browsers are somewhat disk heavy applications. every time i open/close a page, re-size the window, some sort of data may be written into the config directories. if the application keeps a bunch of file descriptors of its configuration files, and use them though out, symlink and mounting would have little difference. but if the application tries to open the file every time it has something to read or write, symlink may produce one extra disk read.

furthermore, we can not be guaranteed that the supported browsers would always be compliant to out setup. maybe some time in the future, some bug introduced in some browser may break our setup by somehow using de-referenced symlinks for other paths. we can not be sure it would never happen. however, using a mount point to the configuration directory, it would be totally transparent to the browser. we can avoid introducing possible bugs that is activated only in our setup. if there's a bug in browser, it should happen to everyone.

and at last, i'd like to mention that my bind mount thinking is not good. directly mounting a tmpfs would be more efficient. however, bind mounting can allow the use case when a user want to move profile into another physical disk, and preserve his/her memory for other usage. directly mounting a tmpfs would not allow this kind of usage.

duquesnc · 2012-03-22 08:47:13

@aug I would not worry about that: open() resolves symlinks transparently. I would say the only reason for this to stop working would be that browsers explicitly try to detect that their config dir has been symlinked to another place and fail on this detection. This would be dumb and probably a good reason to drop the browser in question.

Nearthesoul · 2012-03-25 11:43:45

Almost every time i reboot my system i lost my chromium profile data

graysky · 2012-03-25 12:28:42

Nearthesoul wrote:

Almost every time i reboot my system i lost my chromium profile data

How are you starting psd?

Last edited by graysky (2012-03-25 12:40:17)

Nearthesoul · 2012-03-25 12:53:11

I follow step-by-step the wiki https://wiki.archlinux.org/index.php/Pr … ync-daemon:
- setting my USERS in /etc/psd.conf (with VOLATILE="/tmp")
- adding psd to DAEMONS in /etc/rc.conf
- addind in /etc/fstab this mount point "tmpfs /tmp tmpfs rw,mode=777 0 0"

The command "ls -l /" give me:
drwxrwxrwt 19 root root 500 25 mar 14.43 tmp

Almost every time i lost my profile. This is a nightmare Thanks in advance for any help

Last edited by Nearthesoul (2012-03-25 12:53:36)

broken pipe · 2012-03-25 15:07:23

do you use any software like bleachbit which flushes /tmp and remove your profile?

tmpfs /tmp tmpfs nodev,nosuid,noatime 0 0

FarStar07 · 2012-04-02 11:28:14

Hello!
Thanks for your script. It seems useful.
Please, can you tell me if opera-next is supported?
Thank you.

Last edited by FarStar07 (2012-04-02 11:31:28)

graysky · 2012-04-02 11:35:12

@FarStar - I can add support if it's not there currently. Does opera-next store the user profile dirs under ~/.opera-next or ~/.opera or ...?

FarStar07 · 2012-04-02 11:42:58

graysky wrote:

@FarStar - I can add support if it's not there currently. Does opera-next store the user profile dirs under ~/.opera-next or ~/.opera or ...?

It's ~/.opera-next
Thanks a lot!

graysky · 2012-04-02 13:30:05

FarStar07 wrote:

graysky wrote:
@FarStar - I can add support if it's not there currently. Does opera-next store the user profile dirs under ~/.opera-next or ~/.opera or ...?
It's ~/.opera-next
Thanks a lot!

Just pushed v3.1 - please try it and let me know.

FarStar07 · 2012-04-02 14:00:43

graysky wrote:

FarStar07 wrote:
graysky wrote:
@FarStar - I can add support if it's not there currently. Does opera-next store the user profile dirs under ~/.opera-next or ~/.opera or ...?
It's ~/.opera-next
Thanks a lot!
Just pushed v3.1 - please try it and let me know.

I try it, but I obtain this error: "The requested URL returned error: 404"

graysky · 2012-04-02 14:12:47

Try now???

FarStar07 · 2012-04-02 14:22:35

graysky wrote:

Try now???

It works great! Thank you very much!

Arch Linux

#76 2012-03-05 23:50:24

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#77 2012-03-06 01:09:48

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#78 2012-03-06 05:56:32

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#79 2012-03-06 10:15:42

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#80 2012-03-13 01:09:45

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#81 2012-03-13 13:31:16

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#82 2012-03-13 17:27:48

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#83 2012-03-13 20:12:15

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#84 2012-03-14 02:20:45

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#85 2012-03-14 21:58:48

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#86 2012-03-17 21:11:21

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#87 2012-03-17 21:32:22

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#88 2012-03-22 04:46:56

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#89 2012-03-22 08:47:13

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#90 2012-03-25 11:43:45

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#91 2012-03-25 12:28:42

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#92 2012-03-25 12:53:11

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#93 2012-03-25 15:07:23

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#94 2012-04-02 11:28:14

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#95 2012-04-02 11:35:12

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#96 2012-04-02 11:42:58

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#97 2012-04-02 13:30:05

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#98 2012-04-02 14:00:43

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#99 2012-04-02 14:12:47

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#100 2012-04-02 14:22:35

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

Board footer