profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

graysky · 2013-09-11 16:57:22

masnormen wrote:

Sep 11 18:22:58 archlinux profile-sync-daemon[1185]: /home/masnormen/.config/chromium does not exist or is a broken symlink! Is /home or /tmp unmounted?
Sep 11 18:22:58 archlinux profile-sync-daemon[1185]: /home/masnormen/.conkeror.mozdev.org does not exist or is a broken symlink! Is /home or /tmp unmounted?
Sep 11 18:22:58 archlinux profile-sync-daemon[1185]: mv: cannot move ‘/home/masnormen/.config/google-chrome’ to ‘/home/masnormen/.config/google-chrome-backup/google-chrome’: Directory not empty
Sep 11 18:22:58 archlinux profile-sync-daemon[1185]: cannot delete non-empty directory: google-chrome
Sep 11 18:22:58 archlinux profile-sync-daemon[1185]: /home/masnormen/.config/midori does not exist or is a broken symlink! Is /home or /tmp unmounted?
Sep 11 18:22:58 archlinux profile-sync-daemon[1185]: /home/masnormen/.opera does not exist or is a broken symlink! Is /home or /tmp unmounted?
Sep 11 18:22:58 archlinux profile-sync-daemon[1185]: /home/masnormen/.opera-next does not exist or is a broken symlink! Is /home or /tmp unmounted?
Sep 11 18:22:58 archlinux profile-sync-daemon[1185]: /home/masnormen/.local/share/luakit does not exist or is a broken symlink! Is /home or /tmp unmounted?
Sep 11 18:22:58 archlinux systemd[1]: Started Timed Profile-resync.

Is the desired behavior for psd to manage both firefox and google-chrome or just one of them? On the surface, the errors you're seeing are resulting from psd not being able to see your /home when it [psd] is getting called to start by systemd... potentially due to lvm? I do not use lvm and do not have a deep understanding of how it is initialized.

1) Post your /etc/fstab for me?

masnormen wrote:

My last chrome profile size is about 80-90 MB. Did my Chrome profile lost?
Anyway, in my last reboot, systemd-analyze said that psd takes 1 minute to start. Now, it's normal, and Chrome profile still can't get sync'ed. WTF?

If there was some disconnect, your profile should be stored under ~/.config/google-chrome-backup.

Let's figure this out:
1) Manually stop psd.
2) Do you see the directory I mentioned above?
3) Is there a broken symlink to 'google-chrome -> /tmp/mansormen-google-chrome'? If so you may remove the symlink and rename the backup to the native name 'google-chrome'

masnormen · 2013-09-12 06:12:58

graysky wrote:

Is the desired behavior for psd to manage both firefox and google-chrome or just one of them? On the surface, the errors you're seeing are resulting from psd not being able to see your /home when it [psd] is getting called to start by systemd... potentially due to lvm? I do not use lvm and do not have a deep understanding of how it is initialized.

Yes. And, I don't use LVM. It's too complicated...

my /etc/fstab:

/dev/sda2               /               ext4            rw,noatime,data=ordered           0               1
/dev/sda1               /boot           ext4            rw,noatime                             0               2
/dev/sda3               /home           ext4            rw,noatime,data=ordered       0               2
tmpfs                   /tmp            tmpfs           defaults,nodev,nosuid,auto      0               0
#/dev/sda4              none            swap            sw                                      0               0

graysky wrote:

If there was some disconnect, your profile should be stored under ~/.config/google-chrome-backup.
Let's figure this out:
1) Manually stop psd.
2) Do you see the directory I mentioned above?
3) Is there a broken symlink to 'google-chrome -> /tmp/mansormen-google-chrome'? If so you may remove the symlink and rename the backup to the native name 'google-chrome'

1. No problem
2. Yes, and... The backup dir size is just about 50 MB, same as my "new" Chrome profile (the one after my old profile is gone)
3. I see no symlink. And I renamed the backup dir to "google-chrome". It's just the backup of the "new" profile. See above.

I think the old profile is gone because of data loss, I can't find my old profile anywhere

graysky · 2013-09-12 08:55:33

@masmoren - I would agree with your assessment. Do you still get the odd messages in the systemctl status regarding nonexistent profile dirs? Do both browsers get synced now?

masnormen · 2013-09-12 11:12:38

graysky wrote:

@masmoren - I would agree with your assessment. Do you still get the odd messages in the systemctl status regarding nonexistent profile dirs? Do both browsers get synced now?

Yes, both runs well. And the systemctl says nothing, everything runs well. Psd runs normally now. Probably I have to complain "that" to my electricity provider, because.... well, you can guess....

And btw, why people can't spell or even type my nickname correctly? xD

graysky · 2013-09-12 19:10:27

masnormen wrote:

And btw, why people can't spell or even type my nickname correctly? xD

You try typing a smart phone

graysky · 2013-09-12 19:40:13

I do not believe psd is to blame for your data loss as it does a check to see if the symlink is present on the filesystem whether or not it is valid and acts accordingly. In other words, your machine hard crashes... the symlink to the browser profile is still on the filesystem upon a reboot, but it is pointing to an invalid target since your tmpfs got cleared. Currently, psd detects this scenario and moves the "last good" backup into place before it setups for another tmpfs linking run. I just modified psd to first save a backup of your profile in the "last good" state before it then rotates it back to the native state for linking. This should serve as a sanity check that psd did no harm. I will try it out before releasing v5.40 which will have this feature.

EDIT: Seems to work well and should give users piece of mind. I also fixed the false positive error message bug I noticed in your output. Enjoy v5.40-1.

Last edited by graysky (2013-09-12 20:47:51)

masnormen · 2013-09-13 11:39:03

graysky wrote:

I do not believe psd is to blame for your data loss as it does a check to see if the symlink is present on the filesystem whether or not it is valid and acts accordingly. In other words, your machine hard crashes... the symlink to the browser profile is still on the filesystem upon a reboot, but it is pointing to an invalid target since your tmpfs got cleared. Currently, psd detects this scenario and moves the "last good" backup into place before it setups for another tmpfs linking run. I just modified psd to first save a backup of your profile in the "last good" state before it then rotates it back to the native state for linking. This should serve as a sanity check that psd did no harm. I will try it out before releasing v5.40 which will have this feature.
EDIT: Seems to work well and should give users piece of mind. I also fixed the false positive error message bug I noticed in your output. Enjoy v5.40-1.

Yeah, I also think so.

Lone_Wolf · 2013-09-19 12:01:47

I am trying this out running openrc - apg pacakges, and encountered some issues :

- apg packages use /etc/openrc/init.d
I solved this by adding a ? to the OPENRC_INITDIR directive in your makefile, so it can be overridden .

- the psd service file for openrc uses !sbin/runscript (probably the gentoo default) and fails.
I manually edited /etc/openrc/init.d/psd to !usr/bin/runscript and it now works.

To build psd for apg openrc I ofcourse need to adapt the PKGBUILD, so could use sed commands for these changes.
Or do you see a possibility to deal with these issues in the source ?

graysky · 2013-09-19 18:54:08

@LW - Sorry, but I don't want to make it more complex... the Makefile already handles several different init systems (3 actually); since you are a very niche Arch user (i.e. psd + openrc), I don't think me adding and maintaining a fourth option would be to my liking

orschiro · 2013-09-20 05:25:41

Is there any way to improve the startup times of both, asd and psd?

After launching `startx`, my Gnome system takes almost a minute to show the Desktop. And I think asd and psd are the cause of it:

[orschiro@thinkpad ~]$ sudo systemd-analyze blame
[sudo] password for orschiro: 
         27.973s psd.service
         16.269s asd.service
          6.348s NetworkManager.service
          4.535s tlp.service
          2.467s upower.service
          2.463s thermald.service
          2.460s systemd-logind.service
          2.233s systemd-vconsole-setup.service
          1.721s systemd-fsck@dev-disk-by\x2duuid-a390d1a9\x2dfb13\x2d4895\x2d8297\x2d03dba4cd6162.service
          1.668s systemd-modules-load.service

graysky · 2013-09-20 09:59:59

@orschiro - Nothing can be done in the code. These are physical limitations of your system.

1) Reduce the amount of data in each respective sync (profile-cleaner may help for browsers)
2) Upgrade to a faster disk (SSD)

Lone_Wolf · 2013-09-20 16:02:09

graysky wrote:

@LW - Sorry, but I don't want to make it more complex... the Makefile already handles several different init systems (3 actually); since you are a very niche Arch user (i.e. psd + openrc), I don't think me adding and maintaining a fourth option would be to my liking

No problem, Graysky.
Maybe a note on the psd wiki page would be helpful ?

It was a nice surprise to see a Makefile catering with 3 different init systems and still be easily understandable.

While firefox was already fast on my system with PSD it feels more responsive.
You've done a great job.

Last edited by Lone_Wolf (2013-09-20 16:04:52)

orschiro · 2013-09-23 05:50:41

@graysky

Thanks. So it is clearly a physical limitation of my old hard disk...

Anyway, do you know why I get this error message?

Sep 23 07:36:25 thinkpad profile-sync-daemon[18956]: mv: cannot move ‘/home/orschiro/.config/chromium’ to ‘/home/orschiro/.config/chromium-backup/chromium’: 
Sep 23 07:36:26 thinkpad profile-sync-daemon[18956]: cannot delete non-empty directory: chromium

graysky · 2013-09-23 06:31:04

@orschiro - Stop psd. Post the output of `ls -l /home/orschiro/.config/`

orschiro · 2013-09-23 07:39:19

Amongst other folders:

...
drwx------ 6 orschiro users  4096 Sep 23 09:36 chromium-backup
drwx------ 5 orschiro users  4096 Sep 23 09:38 chromium

graysky · 2013-09-23 07:44:58

@orschiro - Are you sure psd is stopped? You should not have a 'chromium-backup' when it is stopped. If you are sure it's stopped... do this:

`mv ~/.config/chromium-backup ~/.config/chromium-backup-what`

Now restart psd and psd-resync.

orschiro · 2013-09-23 08:50:21

@graysky

That seemed to work. Thanks.

Any idea why this happened?

graysky · 2013-09-23 08:59:07

@orschiro - Not sure... somehow you had two physical dirs rather than one dir and one symlink.

orschiro · 2013-10-12 05:56:39

What can be causing the issue that psd somtimes cannot store / restore my Chromium successfully?

After closing Chromium and rebooting I sometimes end up with a completely new session, having lost all my settings and tabs.

This is the log output:

Oct 12 07:19:14 thinkpad systemd[1]: Started Profile-sync-daemon.
Oct 12 07:19:14 thinkpad systemd[1]: Starting Timed Profile-resync...
Oct 12 07:19:14 thinkpad profile-sync-daemon[482]: [131B blob data]

graysky · 2013-10-12 12:23:28

@orschiro - Never saw that blob data line before. Do you have any "last good" backups present? See the FAQ section in the psd manpage or on the wiki page.

orschiro · 2013-10-12 15:57:47

@graysky

No "last good" backup was created. All I have is just /chromium and /chromium-backup. But the latter does not contain anything of my last used profile data.

orschiro · 2013-10-13 07:15:53

In relation to my problem I have another question. What happens actually if I close my browser and immediately shutdown my system? All services will then be terminated. Can this cause data loss since psd does not have sufficient time to sync the files before shutdown?

If this is the case, can the system be configure to wait with shutdown until psd has synced all files back?

graysky · 2013-10-13 12:15:53

@orschiro -
*Since you cannot find any lastgood folders, that means you have not experienced any crashes/unclean shutdowns of psd.
*Systemd waits for psd to finish unsyncing before it shuts the system off or restarts it.

orschiro · 2013-10-14 07:16:05

*Systemd waits for psd to finish unsyncing before it shuts the system off or restarts it.

Good to here that. I was wondering because quite often I get some of the following message at shutdown:

Cannot finalize remaining files [...]

Anyway, I will investigate the issue further.

orschiro · 2013-11-06 10:08:11

@graysky

Is this message on startup something I have to worry about?

home-orschiro-.cache.mount: Directory /home/orschiro/.cache to mount over is not empty, mounting anyway.
Nov 06 10:43:21 thinkpad systemd[1]: Mounted /home/orschiro/.cache.

psd takes care of my ~/.cache to sync my Chromium profile. I set up the following tmpfs in /etc/fstab as suggested in the Wiki:

~ $ cat /etc/fstab 
# 
# /etc/fstab: static file system information
#
# <file system>	<dir>	<type>	<options>	<dump>	<pass>
tmpfs	/home/orschiro/.cache	tmpfs	noatime,nodev,nosuid,size=2048M	0	0

UUID=469ca819-9b4d-48d1-b9f9-7377f47ff722 / ext4 defaults,noatime 0 1
UUID=732ac739-0110-4554-b86d-4d7413e89995 none swap defaults 0 0
UUID=a390d1a9-fb13-4895-8297-03dba4cd6162 /home ext4 defaults,noatime 0 2

Arch Linux

#426 2013-09-11 16:57:22

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#427 2013-09-12 06:12:58

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#428 2013-09-12 08:55:33

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#429 2013-09-12 11:12:38

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#430 2013-09-12 19:10:27

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#431 2013-09-12 19:40:13

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#432 2013-09-13 11:39:03

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#433 2013-09-19 12:01:47

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#434 2013-09-19 18:54:08

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#435 2013-09-20 05:25:41

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#436 2013-09-20 09:59:59

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#437 2013-09-20 16:02:09

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#438 2013-09-23 05:50:41

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#439 2013-09-23 06:31:04

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#440 2013-09-23 07:39:19

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#441 2013-09-23 07:44:58

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#442 2013-09-23 08:50:21

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#443 2013-09-23 08:59:07

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#444 2013-10-12 05:56:39

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#445 2013-10-12 12:23:28

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#446 2013-10-12 15:57:47

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#447 2013-10-13 07:15:53

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#448 2013-10-13 12:15:53

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#449 2013-10-14 07:16:05

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#450 2013-11-06 10:08:11

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

Board footer