profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

graysky · 2012-09-24 23:26:25

@samuvuo - Do you have sudo installed? (Shouldn't need it since that var will return null).

Last edited by graysky (2012-09-24 23:45:10)

samuvuo · 2012-09-24 23:41:12

I do. Have had it since Arch was installed. But now that you mention it... will check sudoers and aliases, maybe something is not vanilla.

graysky · 2012-09-24 23:45:14

Help me troubleshoot here...

1) Stop psd
2) Upgrade to 3.14-1
3) Post the output of :

$ sed '/#/d' /etc/psd.conf

4) Post the output of:

$ profile-sync-daemon debug

Last edited by graysky (2012-09-24 23:54:04)

samuvuo · 2012-09-25 00:28:55

I have:

1) Removed alias sudo='sudo ' [1]
2) Stopped psd
3) Removed psd
4) Installed psd again (and checked it's 3.14-1)

Outputs:

$ sed '/#/d' /etc/psd.conf

USERS="samuvuo"

BROWSERS="chromium"

VOLATILE="/tmp"

$ profile-sync-daemon debug

user: samuvuo
group: root samuvuo
homedir: /root /home/samuvuo
browser chromium

$ sudo profile-sync-daemon debug

user: samuvuo
group: samuvuo
homedir: /home/samuvuo
browser: chromium

Now if I start psd (sudo rc.d start psd) it creates pid files in /run and also /run/daemons - but does not create directories to /tmp nor copy anything anywhere.

BTW: /tmp is mounted as tmpfs and it's:

Access: (1777/drwxrwxrwt)  Uid: (    0/    root)   Gid: (    0/    root)

[1] Had used this old "space trick" because aliases were not working when using sudo. "If the last character of the alias value is a space or tab character, then the next command word following the alias is also checked for alias expansion." See http://www.gnu.org/software/bash/manual … ml#Aliases

graysky · 2012-09-25 00:37:07

Very odd! What is the output of:

$ echo $SUDO_USER

samuvuo · 2012-09-25 01:02:31

graysky wrote:

Very odd! What is the output of:
$ echo $SUDO_USER

It gives me an empty line.

Weird indeed. To test it further, I made a script:

#!/bin/bash
user=samuvuo
logname=$(logname)

homedir="$(getent passwd $SUDO_USER | grep $user | cut -d: -f6)"
newhomedir="$(getent passwd $logname | grep $user | cut -d: -f6)"

whoami
echo $SUDO_USER
echo $homedir
echo $newhomedir

Running it without and with sudo:

$ sh sudotest.sh
samuvuo

/root /home/samuvuo
/home/samuvuo

$sudo sh sudotest.sh
samuvuo

/root /home/samuvuo
/home/samuvuo

No output from $SUDO_USER. I will do some more poking around. Something in my sudo settings is seriously messed up, it seems, since no one else is reporting profile-sync-daemon not working after the latest update.

We can continue this in email, if you want to keep the thread cleaner.

Edit: Ok, the "homedir: /root /home/samuvuo" mystery got solved. root's entry in /etc/passwd had "samuvuo" on that line. Have removed it, will re-install psd and test further.

Edit 2: And that fixed it. My sudotest.sh works now as well. The culprit was "samuvuo" in root's /etc/passwd file comment field. How did it get there? I must have used it as "real name" when installing.

Was:
root:x:0:0:samuvuo:/root:/bin/bash

Changed to:
root:x:0:0:root:/root:/bin/bash

Last edited by samuvuo (2012-09-25 01:26:06)

graysky · 2012-09-25 01:22:42

Actually, there was no reason to have the variable in the code at all. Please try version 3.15-1 which I believe will fix all problems.

...why did your root entry in /etc/passwd contain your user as well?

samuvuo · 2012-09-25 01:30:00

graysky wrote:

...why did your root entry in /etc/passwd contain your user as well?

Must have used it as a comment ("real name"), when installing Arch. Cannot really remember. But as I said, it's working now. Will try 3.15-1 next.

graysky · 2012-09-25 01:33:12

Cool, it should be a dead bug now. Thanks for helping to identify it.

kozaki · 2012-10-04 12:27:43

Running psd on both a desktop and a netbook that have wholy fatty Firefox profiles loaded with their respectful lot of extensions (One for webDev, another for languages etc), love the responsiveness it added to them. But before configuring the browsers, the boot-time was quite impacted, and it eventually made me worrying for the old HDDs / next SSD

I'm thinking about to edit the wiki entry (please see question bellow), based on these references:

As it might be handy for us psd users to get reference for configuring their browser right into the wiki entry, e.g. bellow with Firefox.
Do you know of settings that would complete the following?

1) Control your Firefox profile's various caches
Where to check usage: about:cache
Where to configure: about:config

- browser.cache.disk.enable;true
- browser.cache.disk.capacity;25600
- browser.cache.disk.smart_size{.first_run,_cached_value} # false as it superseeds cache.disk.capacity
- browser.cache.offline.enable;true #is only used for/if web applications are installed

2) Turn off anti-phishing
Effect: this single setting achieves minus 5 to >60 MB per profile
Before you start: Backup the profile then remove ~/.mozilla/firefox/<profile>/urlclassifier3.sqlite

- browser.safebrowsing.enabled;false
#True : Compare visited URLs against a blacklist (that is stored in urlclassifier3.sqlite) or submit URLs to a third party to determine whether a site is legitimate. (Default)
#False : Disable "Safe Browsing".
- browser.safebrowsing.malware.enabled;false
- browser.safebrowsing.remoteLookups;true
#Note : Whatever you put here won't change anything if "safe Browsing" is disabled.

3) Defragment the profile's other SQLite databases
Effect: here it decreased each of my profiles' sizes by a nice ~15 MB (whether empty or heavily loaded profile)
A good way to do so is to use Graysky's profile-cleaner

Results:
While keeping the whole lots of extensions and the Cache feature where I had use for it, configuring Firefox allowed me to much decrease the data to be moved to /tmpfs:

Netbook (1024 MB RAM), 2 users with one browser profile each :
Before configuring Firefox

97      /tmp/USER1-mozilla
103     /tmp/USER2-mozill
200	/tmp/*mozilla

After:

42      /tmp/USER1-mozilla
56      /tmp/USER2-mozilla
98	/tmp/*mozilla

Desktop (2 GB RAM), 2 users, 4 browser profiles:
Default:

 129    /tmp/USER2-mozilla
 260    /tmp/USER1-mozilla
 389    /tmp/*mozilla

After:

 24     /tmp/USER2-mozilla
 149    /tmp/USER1-mozilla (3 profiles, biggest use 50 MB for Cache, 40MB for the ~20 extensions)
 169    /tmp/*mozilla

I'll add a reference to psd's wiki entry into https://wiki.archlinux.org/index.php/Firefox_Tweaks
As for psd's wiki entry, should I add with the detailed config steps, or rather with just the links to specific/already available documentation?

Last edited by kozaki (2012-10-04 12:32:36)

domac · 2012-10-04 15:44:03

Just some request… starting the psd.service takes the most time of all services started…

$domac.vex systemd-analyze blame
 11918ms psd.service
[...]

Maybe this could be speed up?
Besides, works well for me! Great tool and pretty handy.
Cheers

EDIT: Just found the answer… sorry for posting (answer).

Last edited by domac (2012-10-04 17:35:03)

graysky · 2012-10-04 18:57:48

kozaki wrote:

As for psd's wiki entry, should I add with the detailed config steps, or rather with just the links to specific/already available documentation?

Great results. Glad you have sped up your system. I wouldn't post the instructions on the psd wiki page itself. I like your idea to place the links. Go for it.

capoeira · 2012-10-13 21:51:29

can following be included someway in the package, if that does any sense?
https://bbs.archlinux.org/viewtopic.php … 8#p1109638

Last edited by capoeira (2012-10-13 21:54:34)

graysky · 2012-10-13 21:56:57

capoeira wrote:

can following be included someway in the package, if that does any sense?
https://bbs.archlinux.org/viewtopic.php … 8#p1109638

The timer option seems to be easy enough to implement but the actual wait time would seem to be system-specific, no? The guy in the thread your referenced used 25 sec but what if you system only need 10 or whatever. I'm not sure how to handle that. Thoughts are welcome.

capoeira · 2012-10-13 22:19:45

graysky wrote:

capoeira wrote:
can following be included someway in the package, if that does any sense?
https://bbs.archlinux.org/viewtopic.php … 8#p1109638
The timer option seems to be easy enough to implement but the actual wait time would seem to be system-specific, no? The guy in the thread your referenced used 25 sec but what if you system only need 10 or whatever. I'm not sure how to handle that. Thoughts are welcome.

perhaps with "OnBootSec="?

if that means after boot is finished.

OnBootSec= defines a timer relative to when the machine was booted up

I'll try that out

EDIT: forget it. doesn't make sense. after boot, loading of DE is going on

Last edited by capoeira (2012-10-13 22:24:31)

avrelaun · 2012-10-28 09:07:19

Hey thank you and keep up the good work graysky !

I've been using psd and profile cleaner for a few weeks now, very nice and fast. Although since I use systemd and launch firefox at startup with i3, it starts before the syncing is finished, and creates a new profile. I need to figure out how to delay the firefox startup to after the sync is complete.

Also, I made a pull request yesterday to add aurora support, please review it.

graysky · 2012-10-28 09:47:08

@avrelaun - Thanks for the kind words. Let's keep the discussion of the merge requests on github; I agree that something needs to be done about the systemd stuff. Perhaps falconindy can suggest (I emailed him asking for his input).

EDIT: Dave or other systemd experts reading this, since all these browsers require X to run... what about:

Before=graphical.target

Last edited by graysky (2012-10-28 09:53:19)

Awebb · 2012-10-28 20:29:40

It depends on how you start x. I use a getty@tty6.service -> autologin@.service to start x via tty6 automatically, like it is described on the wiki and at least one blog post on the planet. In this case, graphical.target would not be the wisest choice, as it "Wants=display-manager.service". You'd exclude non-DM users.

I wanted to write something about graphical.target wanting a DM and excluding non-DM users, but it seems like it is started on my machine anyway. This is confusing.

graysky · 2012-10-28 21:52:21

EDIT: Nevermind. Need to come up with a better readme.install to control this.

Last edited by graysky (2012-10-28 22:14:02)

avrelaun · 2012-10-28 22:17:20

Small bug during install of 4.01-1:

...
(1/1) checking available disk space [########################################] 100%
/tmp/alpm_XZ6cHP/.INSTALL: line 16: [: BROWSERS="mozilla: binary operator expected
(1/1) upgrading profile-sync-daemon [########################################] 100%
...

EDIT: Apparently the bug is bigger than expected. It did not properly remove .mozilla/ from the ram to my ~ during the switch, and after a reboot the link to /tmp was still present. I had to move back the .mozilla-backup.

Last edited by avrelaun (2012-10-28 22:36:48)

graysky · 2012-10-28 23:19:05

Yeah, this is why I pulled it. I have to give the install file more thought. Basically, everyone will need to stop the daemon, update the conf and restart the daemon. I would love to automate it, but I think the best I can do is to stop it if it is running and echo the instructions. I'm up to my neck in shit right now and can't work on it until tomorrow.

Last edited by graysky (2012-10-28 23:19:35)

graysky · 2012-10-29 17:20:36

OK, 4.0.2-1 is up on the AUR and should work for all. Plz report issues as usual.

Awebb · 2012-11-04 09:34:43

I just updated from 3.16-1 to 4.03-1. I lost my chromium profile in the process. It somehow remembered my gmail account I was logged in with, but everything else is "lost". At least it's chromium, so I didn't lose anything but local plugin configs.

This is what I did: I closed the browser… stopped psd… grabbed the tarball… makepkg -s… pacman -U… delete the old psd.conf, rename psd.conf.pacnew and fill in my info… systemctl --system daemon-reload… systemctl start psd… profile dead.

graysky · 2012-11-04 11:31:41

Sounds like you've done everything right. No idea what went wrong. In reality, after you stop psd, there should be no danger of anything related to the package since the profile is rotated back to its virgin state on your disk. I've tested the upgrade process with the psd.install scriptlette in many different scenarios and never got data loss. Wish I knew what to tell you.

Awebb · 2012-11-04 11:46:03

That's what I thought. Something else must have gone wrong, maybe during stopping psd or even before. But there was another weird thing: After starting the new version for the first time, there were four folders, /tmp/$USER-{chromium,opera,firefox,google-chrome}, although I only have chromium installed and configured in the psd.conf. After stopping and starting the service again, there was only -chromium, as I am used to.

Last edited by Awebb (2012-11-04 11:56:32)

Arch Linux

#176 2012-09-24 23:26:25

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#177 2012-09-24 23:41:12

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#178 2012-09-24 23:45:14

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#179 2012-09-25 00:28:55

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#180 2012-09-25 00:37:07

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#181 2012-09-25 01:02:31

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#182 2012-09-25 01:22:42

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#183 2012-09-25 01:30:00

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#184 2012-09-25 01:33:12

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#185 2012-10-04 12:27:43

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#186 2012-10-04 15:44:03

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#187 2012-10-04 18:57:48

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#188 2012-10-13 21:51:29

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#189 2012-10-13 21:56:57

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#190 2012-10-13 22:19:45

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#191 2012-10-28 09:07:19

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#192 2012-10-28 09:47:08

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#193 2012-10-28 20:29:40

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#194 2012-10-28 21:52:21

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#195 2012-10-28 22:17:20

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#196 2012-10-28 23:19:05

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#197 2012-10-29 17:20:36

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#198 2012-11-04 09:34:43

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#199 2012-11-04 11:31:41

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#200 2012-11-04 11:46:03

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

Board footer