You are not logged in.

#201 2012-11-04 11:46:25

capoeira
Member
From: Vila Velha - Brasil
Registered: 2010-05-25
Posts: 470

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

Awebb wrote:

I just updated from 3.16-1 to 4.03-1. I lost my chromium profile in the process. It somehow remembered my gmail account I was logged in with, but everything else is "lost". At least it's chromium, so I didn't lose anything but local plugin configs.

This is what I did: I closed the browser… stopped psd… grabbed the tarball… makepkg -s… pacman -U… delete the old psd.conf, rename psd.conf.pacnew and fill in my info… systemctl --system daemon-reload… systemctl start psd… profile dead.

look if you have "chromium-backup" folder in ~/.config
that saved me before
just replace the folder

Offline

#202 2012-11-04 11:57:13

Awebb
Member
Registered: 2010-05-06
Posts: 6,268

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

Unfortunately, there is no -backup folder. Looks like psd did it's job :-)

also updated the package on my netbook and everything went as expected. I then played a bit with the daemon. It looks like stopping the service while chromium is open has a certain chance of scrambleing the profile, depending on what chromium is doing at that time. I could do some diff'ing, but I have the feeling it is connected to the sync option with the gmail profile, because all the files were still there, chromium only seemed assumed, that it was installed on a new machine, because one of the files it uses to sync with the account was in an inconsistent state. That might explain why all the passwords in the password store are still there, as I unchecked them in the sync option.

Yay, cloud computing is soooo exciting…

Last edited by Awebb (2012-11-04 11:57:43)

Offline

#203 2012-11-04 13:35:55

graysky
Wiki Maintainer
From: :wq
Registered: 2008-12-01
Posts: 10,592
Website

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

Awebb wrote:

... After starting the new version for the first time, there were four folders, /tmp/$USER-{chromium,opera,firefox,google-chrome}, although I only have chromium installed and configured in the psd.conf. After stopping and starting the service again, there was only -chromium, as I am used to.

That would happen if:
1) You have all four browsers installed on your machine AND you have the BROWSERS array in /etc/psd.conf commented.
2) You have all four browsers installed on your machine AND you manually defined all of them in the BROWSERS array in /etc/psd.conf commented.


Awebb wrote:

...I then played a bit with the daemon. It looks like stopping the service while chromium is open has a certain chance of scrambleing the profile, depending on what chromium is doing at that time.

Interesting.  I am going to mess around with mine right now -- after it is backed-up of course smile

EDIT: Without getting too verbose, the version 3 --> version 4 update is a tough one in that I cannot cause pacman to abort the installation if your certain conditions on your system exist.  This should only be a problem going to v3.x to v4.x; subsequent version bumps do not care if, for example, browsers are open or closed.

Best practice for the users of version 3.x is to exit your browsers before you attempt to install version 4.x.

Last edited by graysky (2012-11-04 14:11:40)


CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#204 2012-11-04 18:35:54

graysky
Wiki Maintainer
From: :wq
Registered: 2008-12-01
Posts: 10,592
Website

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

Your experience, Awebb, caused me to think about how to avoid these situations.  I came up with adding a check to the unsync function that sends SIGTERM to the browser if running thus quitting it.  This way, browser profile integrity will be preserved if a user stops the deamon while it is running.  The downside is that the browser will be quit, but the upside is that user profiles will be intact!

I will work on this later today; plz post your ideas for alternatives to this thread.

I'm thinking pkill -SIGTERM but if that fails, do we need a stronger kill command?

Last edited by graysky (2012-11-04 20:08:20)


CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#205 2012-11-04 20:36:13

Awebb
Member
Registered: 2010-05-06
Posts: 6,268

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

You could start by keeping the browser profile in /tmp instead of killing it instantly, maybe with a -bak suffix. This way the we could undo the damage by closing the browser, restarting the service and renaming the folder in /tmp.

Maybe you could sanitize psd's behavior. There must be a way to check whether a file is open, being written to, whatever. I have incron in mind right now, maybe the underlying mechanism, I don't have much expertise in that field.

About killing the browser:
1. I am not sure I'd be comfortable with this. I sometimes have work done in a browser window, open connections, open downloads, I would not want that to be lost.
2. At the moment, the only reasons for a cleanup-and-exit of psd are a) the user stopping the service or b) updating psd via AUR. Updating via AUR is still quite a manual intervention - I am excluding people relying on yaourt's updater here, because they should know better - so updating psd should obviously be a special moment for a browser user, so closing the browser first does not sound like witchcraft.

Offline

#206 2012-11-04 20:58:52

graysky
Wiki Maintainer
From: :wq
Registered: 2008-12-01
Posts: 10,592
Website

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

@Awebb - Correct, only two conditions that would require this:

1) User stopping the service.
2) Updating v3.x --> v4.x which should be a one-time thing.

To point #1, I can envision several scenarios where data could be lost: all are reboots/shutdowns.

...what if a reboot/shutdown is triggered and the user left the browsers open? For example using `sudo shutdown -h x` which I do pretty frequently.
...what if it is a multi-user box and user1 is logged in with browser open and idle and user2 logs into the box and reboots it?  This too has happened to me when the wife leaves her shit open (browser included logged into facebook.com usually) and I sit down, and decide to reboot after rebuilding a kernel smile
...what if the user leaves a browser open on a laptop and the pm-daemon shuts down the box do due to critical battery levels being reached?

In all the of cases I outlined above, the init system would be running the unsync function.  Adding a 'kill' function therein would safe guard the browser profiles in these events so I do think some sort of sanity check/sync back to disk before the daemon goes down is value-added.

I like the idea of placing a backup somewhere, but in the event of reboots/shutdowns, the backup would be lost since it's tmpfs.

Last edited by graysky (2012-11-04 21:01:09)


CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#207 2012-11-05 03:39:33

barraponto
Member
From: Campinas, Brazil
Registered: 2009-10-15
Posts: 29
Website

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

Regarding the issue of running the browser before files are cached, what if PSD rsyncd the folders first, symlinked later? That way it'd never run with a crippled profile.

Last edited by barraponto (2012-11-05 03:39:52)

Offline

#208 2012-11-05 08:03:40

Awebb
Member
Registered: 2010-05-06
Posts: 6,268

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

Multi-user setups usually use a DM. If I understand systemd correctly, shutting down means working backwards through the targets. That means the browser will be dead or in a dying state by the time psd starts shutting down.

EDIT: On a multiuser system, I'd generally make it a habit of closing other user's sessions before rebooting, since psd isn't the only cause of possible inconsistence in such cases.

Last edited by Awebb (2012-11-05 08:05:09)

Offline

#209 2012-11-05 17:35:10

barraponto
Member
From: Campinas, Brazil
Registered: 2009-10-15
Posts: 29
Website

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

I'm running psd 4.10-2 and it's syncing both firefox and chromium, despite setting BROWSERS to firefox on /etc/psd.conf
This is the content of my config file: https://gist.github.com/03114a3ccb5744c8caa1

Offline

#210 2012-11-05 19:57:17

graysky
Wiki Maintainer
From: :wq
Registered: 2008-12-01
Posts: 10,592
Website

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

barraponto wrote:

I'm running psd 4.10-2 and it's syncing both firefox and chromium, despite setting BROWSERS to firefox on /etc/psd.conf
This is the content of my config file: https://gist.github.com/03114a3ccb5744c8caa1

@barraponto - Please post the output of:

% profile-sync-daemon debug

Last edited by graysky (2012-11-05 19:57:50)


CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#211 2012-11-08 01:08:49

soupcan
Member
From: ?
Registered: 2008-10-25
Posts: 268

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

I'm having a similar problem to samuvuo. I have no output from echo $SUDO_USER - I've reinstalled sudo and checked my configuration, I can't see anything wrong. My user is not present in the root lies as in samuvo's case. Added my user to sudoers via this line: USER_NAME   ALL=(ALL) ALL. psd is starting on boot but it seems that my profile is not immediately synced. I have to login to chromium and then close and open it a few times before my settings are synced.
output of profile-sync-debug:

% profile-sync-daemon debug                                                                                 ~
::DEBUG MODE::
profile-sync-daemon version: 4.11

settings in /etc/psd.conf will make profile-sync-daemon manage the following browers/user combos when invoked:

browser to manage: chromium
owner/group:       killian:power
homedir:           /home/killian
backup target:     /home/killian/.config/chromium-backup

browser to manage: firefox
owner/group:       killian:power
homedir:           /home/killian
backup target:     /home/killian/.mozilla/firefox-backup

Hoping I can figure out what's going on, I'm going to test a few things. Hopefully I can figure out what's wrong with my sudo config. I'll provide any other outputs/info if needed.
EDIT: Things seem to be working right after a reboot now... I'll keep checking.

Last edited by soupcan (2012-11-08 01:28:15)

Offline

#212 2012-11-08 09:52:47

graysky
Wiki Maintainer
From: :wq
Registered: 2008-12-01
Posts: 10,592
Website

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

From your output, psd sees that you have a Firefox profile and a chromium profile.  Psd does not use sudo at all so the status of that on your system does not matter.  Are you starting it via the included service file?


CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#213 2012-11-08 17:25:04

bassu
Member
Registered: 2011-12-21
Posts: 90

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

Graysky: You are fuckin AWESOME!!! Love you!!!
First, the ck repo and now the psd -- you must be god-sent big_smile

Last edited by bassu (2012-11-08 17:25:32)


The greatest threat to knowledge is not ignorance - it is the illusion of knowledge!

Offline

#214 2012-11-08 23:59:06

soupcan
Member
From: ?
Registered: 2008-10-25
Posts: 268

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

graysky wrote:

From your output, psd sees that you have a Firefox profile and a chromium profile.  Psd does not use sudo at all so the status of that on your system does not matter.  Are you starting it via the included service file?

Yes, starting it on boot with systemctl enable psd.service.

Offline

#215 2012-11-11 13:43:06

Awebb
Member
Registered: 2010-05-06
Posts: 6,268

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

I have now found why chromium thinks the profile is gone. After the last reboot, there was a /tmp/myuser-chromium/chromium-backup with all the data, but no data in /tmp/myuser-chromium. Closing the browser, deleting all other files and then moving "chromium-backup/* .." solved it. There must be something going wrong sometimes either on syncing to the hdd before the reboot or on copying the data to /tmp on boot. I will inspect more.

Offline

#216 2012-11-11 14:08:11

graysky
Wiki Maintainer
From: :wq
Registered: 2008-12-01
Posts: 10,592
Website

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

@Awebb - Odd... this is exactly what the check function does hmm

EDIT: I think I made the crash recovery more robust.  Please try v4.13.
https://aur.archlinux.org/packages/profile-sync-daemon

Last edited by graysky (2012-11-11 14:32:52)


CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#217 2012-11-13 05:16:44

pingpong
Member
From: Los Angeles
Registered: 2010-12-30
Posts: 104

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

Hum, are there tweaks that can make psd.service boot up faster?  Here is my systemd-analyze blame output:
While 2.1s is not bad but it's still almost 7 times slower than the 2nd slowest service which is wicd.

Psd is very nice so I plan to keep it. And any suggestions to make it load faster are welcome, thanks in advance.

  2133ms psd.service
   323ms wicd.service
   233ms systemd-logind.service
   173ms systemd-udev-trigger.service
   166ms systemd-user-sessions.service
   155ms systemd-vconsole-setup.service
   155ms systemd-modules-load.service
   154ms sys-kernel-debug.mount
   147ms dev-mqueue.mount
   127ms dev-hugepages.mount
   124ms systemd-sysctl.service
    94ms systemd-remount-fs.service
    44ms tmp.mount
    39ms colord.service
    37ms systemd-udevd.service
    20ms systemd-tmpfiles-setup.service
    13ms home.mount
    10ms boot.mount

Offline

#218 2012-11-13 05:22:44

graysky
Wiki Maintainer
From: :wq
Registered: 2008-12-01
Posts: 10,592
Website

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

@pingpong - You can do several things:

1) Reduce your browser profile's footprint.
2) Reduce the number of browser profiles you want psd to manage (see /etc/psd.conf).
3) Upgrade to an SSD (if you haven't already).

#1 can be accomplished via several methods including removing/limiting cache, history, etc.  You can also use profile-cleaner for browser profile maintenance.

Last edited by graysky (2012-11-13 05:22:56)


CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#219 2012-11-13 05:29:52

pingpong
Member
From: Los Angeles
Registered: 2010-12-30
Posts: 104

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

@graysky

Thanks for the quick reply! For 2) I have only one browser (firefox) and one profile being synced, so not much I can do there. And the system is on an SSD indeed.
I will look into 1) and report back if I can achieve something significantly faster.

Offline

#220 2012-11-13 12:43:36

graysky
Wiki Maintainer
From: :wq
Registered: 2008-12-01
Posts: 10,592
Website

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

@pingpong - How large is the firefox directory currently? 

du -Lhs $HOME/.mozilla/firefox

Run profile-cleaner and repeat that command to see the difference.

As a point of comparison for you, my chromium profile is 98M and syncing takes <700 ms.

% systemd-analyze blame | grep psd
   695ms psd.service
% du -Lhs $XDG_CONFIG_HOME/chromium
98M	/home/facade/.config/chromium

Last edited by graysky (2014-11-03 08:40:33)


CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#221 2012-11-13 14:56:39

pingpong
Member
From: Los Angeles
Registered: 2010-12-30
Posts: 104

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

@graysky

Thanks for the tips. Well here is what my output of

du -Lhs $HOME/.mozilla/firefox
~ [i]> du -Lhs $HOME/.mozilla/firefox/
du: cannot access ‘/home/pong/.mozilla/firefox/5ydjr43n.default/lock’
90M	/home/pong/.mozilla/firefox/

So around 90M and that's after running profile-cleaner.
Unfortunately, in my case, after all these psd.service still takes 2163ms to start.

By the way, I don't understand why there is a link

lrwxrwxrwx  1 pong users       15 Nov 13 06:44 lock -> 127.0.0.1:+2090

in my profile though, could it be the problem? Firefox regenerate the link after it's been removed. Any idea?

Offline

#222 2012-11-13 15:33:49

graysky
Wiki Maintainer
From: :wq
Registered: 2008-12-01
Posts: 10,592
Website

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

I believe that you have to close firefox before you run the du scan.  Chromium has a similar lock when running.

Did you run the cleaner?
How faster is your SSD (SATA2 or SATA3)?
Mine is a SATA3 for reference.

Last edited by graysky (2012-11-13 16:18:11)


CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#223 2012-11-13 15:37:14

pingpong
Member
From: Los Angeles
Registered: 2010-12-30
Posts: 104

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

@graysky

Oh, I just realize that having that lock link is normal when firefox is running. It goes away when I quit it.

However, in the output systemd-analyed plot, it shows that psd.service has a red bar (i.e. activating) for the whole 2s. While others services has their bars part red and part pink.
I don't know if this helps in diagnosis you start 3 times faster than I.

Offline

#224 2012-11-13 16:17:58

graysky
Wiki Maintainer
From: :wq
Registered: 2008-12-01
Posts: 10,592
Website

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

pingpong wrote:

...in the output systemd-analyed plot, it shows that psd.service has a red bar (i.e. activating) for the whole 2s. While others services has their bars part red and part pink. I don't know if this helps in diagnosis you start 3 times faster than I.

No idea... I assume that 'activating' in this context means that you're waiting for the rsync to finish.  I am thinking that you simply have a slower machine/SSD than I do.

My CPU is an i7-3770k @ 4.5 GHz.
My chipset is a Z77.
My SSD is OCZ-VERTEX 4 on the Z77 SATA3 controller.  See its benchmark results on the wiki page: https://wiki.archlinux.org/index.php/SS … Controller

Several questions for you:

1)

graysky wrote:

How fast is your SSD (SATA2 or SATA3)?  Mine is a SATA3 for reference.

2) How brand/model is your SSD?
3) What do your benchmark numbers look like in endpoints shown on the wiki page?

Last edited by graysky (2012-11-13 16:26:29)


CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#225 2012-11-13 16:54:01

pingpong
Member
From: Los Angeles
Registered: 2010-12-30
Posts: 104

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

It may well be the case. You have an awesome system smile

Mine is a Toshiba R835-P56. The cpu is i5 2450 @2.3 GHz.
Has to look to find out the chipset. And it's on Vertex 3. (SATA 3)

I like it, it's more than enough for what I need it for.

Hum... it there is way to let me have the login prompt before psd finish syncing the profile?

Without it my boot time is just < 3s.

Offline

Board footer

Powered by FluxBB