profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

graysky · 2012-11-13 17:18:16

graysky wrote:

3) What do your benchmark numbers look like in endpoints shown on the wiki page?

...in other words, run the benchmarks

pingpong · 2012-11-13 18:48:32

@graysky

Oh, okay. Here are the results:

# hdpram -Tt /dev/sda

/dev/sda:
 Timing cached reads:   11180 MB in  2.00 seconds = 5594.15 MB/sec
 Timing buffered disk reads: 736 MB in  3.00 seconds = 245.27 MB/sec

$ dd if=/dev/zero of=tempfile bs=1M count=1024 conv=fdatasync,notrunc

1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 4.20024 s, 256 MB/s

echo 3 > /proc/sys/vm/drop_caches

dd if=tempfile of=/dev/null bs=1M count=1024
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 4.12454 s, 260 MB/s

Finally

dd if=tempfile of=/dev/null bs=1M count=1024

1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 0.172948 s, 6.2 GB/s

Well, I have a Vertex 3 120 GB but looks like it much slower than what mulfone
posted result...

I guess a work around is to start psd.service after the system is booted.
Sounds ok?

graysky · 2012-11-13 19:43:24

pingpong wrote:

Well, I have a Vertex 3 120 GB but looks like it much slower than what mulfone posted result...
I guess a work around is to start psd.service after the system is booted.
Sounds ok?

Why? You will have to wait for it to sync at boot or after login...

Also, I posted your results to the wiki; please edit as you see fit.

https://wiki.archlinux.org/index.php/SS … ERTEX3_120

pingpong · 2012-11-13 19:54:08

Thanks for posting it. I was thinking about that.

Btw, how can I find out the firmware version?

Also, well if I put it in .xinitrc, I will have a faster boot time and usually firefox is not the first application that I fired up after login, so psd would definitely have the 2s it needs.
Does it make sense?

I know it sounds a bit silly, afterall we are just talking about 2s. But I learned something from this discussion, so that's worth all the typing.

alain · 2012-11-20 15:04:20

Hi all,

This daemon seems very useful, but according to the conf file, I guess it will only synchronize the "default" profile. Is-it correct?

In my family, we use the same (auto)-login, but each of us has its own profile (to save tabs).

Is-it possible to configure profile-sync-daemon to synchronize these profiles (for example, by defining the full path of the profile to synchronize?)

If not, I hope that this feature will be implemented one day

Thanks.

Alain

Last edited by alain (2012-11-20 15:04:46)

graysky · 2012-11-20 15:16:28

@Alain - Let me make sure I understand:

1) All your family uses the same Linux user.
2) You all have different profiles under that same user.

Are my first two statements correct?

If you are referring the the firefox browser, psd should sync all of your profiles assuming that are under $HOME/.mozilla/firefox which is the default. Post the output of:

% ls -l $HOME/.mozilla/firefox

Last edited by graysky (2012-11-20 15:17:19)

alain · 2012-11-20 15:27:37

Hi graysky,

you're completly right.

Ok, I didn't know it will sync all profiles under $HOME/.mozilla/firefox.
To bad for me, my profiles are not store here (I have a dedicated encrypted partition where i store my data, as well as firefox profile)
For example /DATA/AlainFolder/FirefoxProfile, /DATA/GrayskyFolder/FirefoxProfile, /DATA/GrayskyFolder/FirefoxProfileForJobAdvertisement ....

May be I should re-think my folder organisation to match your soft...

Is there a way to change the default location for all profiles ?

Many thanks.

Regards.
Alain

alain · 2012-11-20 15:51:17

Ok, I've checked the file profile-sync-daemon from git.
The main probleme (or advantage from user point of view) is that folders are hard coded:

(...)
google-chrome)
BACKUP="$homedir/.config/$browser-backup"
DIR="$homedir/.config/$browser"
ENAME="chrome"
;;
firefox)
BACKUP="$homedir/.mozilla/firefox-backup"
DIR="$homedir/.mozilla/firefox"
ENAME="$browser"
(...)

One solution from my point of view will be to externalize DIR and ENAME into psd.conf for each browsers (BACKUP may be calculate from DIR).
I'm not too familiar with bash, I d'ont know if it is possible to create an "array" of data.
For example in psd.conf:
DIR[firefox]=...
ENAME[firefox]=...

May be it is less convenient for updates?

Do you think it's a bad idea?

Thanks and congratulations for this software.

Regards.
Alain

graysky · 2012-11-20 17:50:33

alain wrote:

my profiles are not store here (I have a dedicated encrypted partition where i store my data, as well as firefox profile)
For example /DATA/AlainFolder/FirefoxProfile, /DATA/GrayskyFolder/FirefoxProfile, /DATA/GrayskyFolder/FirefoxProfileForJobAdvertisement ....

So you define within ~/.mozilla/firefox/profiles.ini in the Path variable? Example:

[Profile0]
Name=main
IsRelative=0
Path=/DATA/AlainFolder/FirefoxProfile

Last edited by graysky (2012-11-20 17:50:49)

alain · 2012-11-21 08:14:21

Exactly. It is the standard way of managing profiles within Firefox.
I then select the profile by selecting the good one from the pop-up menu showed by FireFow at startup (with -P option)

Do you think it is feasible to read this file to synchronize the paths ? If so, it will be a smart solution !

Regards
Alain

Last edited by alain (2012-11-21 08:27:50)

graysky · 2012-11-22 11:25:05

@alain - Dunno if I have the knowledge to do this... pull requests are welcomed

alain · 2012-11-22 15:19:36

graysky wrote:

@alain - Dunno if I have the knowledge to do this... pull requests are welcomed

Pleased to hear that. I'm sure that you will find solutions
Unfortunately, I'm not familiar with bash scripting, so it will be difficult to help you...

Anyway, thanks for your replies and congratulations for your work (and share).

Regards
Alain

Nudin · 2012-11-22 23:48:08

I was looking for something like that – great! But unfortunately it doesn't work for me. When starting it, the profiles (firefox and chromium) are moved to there backup-location. And a symlink is created to /tmp/michi-firefox, but the profile is not copied to this location. (folder /tmp/michi-firefox does not exist)
Heres my configuration:

$ profile-sync-daemon debug
::DEBUG MODE::
profile-sync-daemon version: 4.13

settings in /etc/psd.conf will make profile-sync-daemon manage the following browers/user combos when invoked:

browser to manage: chromium
owner/group:       michi:UNKNOWN
homedir:           /home/michi
backup target:     /home/michi/.config/chromium-backup

browser to manage: firefox
owner/group:       michi:UNKNOWN
homedir:           /home/michi
backup target:     /home/michi/.mozilla/firefox-backup

Is the unknown group the problem (and what causes this) or what am I doing wrong?

alain · 2012-11-23 08:32:43

@Nudin

I think that your home folder has no defined "group".
Check it by doing

$ls -al /home

You will see that the owner is "michi" and the group is a number, "1001" for example.

My suggestion is to add your user to "users" group (as root):

# gpasswd -a michi users

and to "chown" your home directory to michi.users (as root)

# chown -Rf michi.users /home/michi

I think this is the origin of your problem. But may be you should wait for Graysky advices....

Best regards
Alain

Last edited by alain (2012-11-23 08:35:15)

Nudin · 2012-11-23 12:23:21

@alain: thanks, that was the problem. Must have forgotten to restore some permissions while restoring backups.

alain · 2012-11-23 13:27:07

Nudin wrote:

@alain: thanks, that was the problem. Must have forgotten to restore some permissions while restoring backups.

You're welcome

graysky · 2012-11-23 14:08:40

@Alain - Thinking about how to parse the profile.ini file for firefox without a total rebuild of the script. No promises.

alain · 2012-11-23 14:15:30

@graysky - No pb, thanks to you

graysky · 2012-11-26 20:45:04

@alain - I got a start on this, but have encountered a roadblock. If anyone is willing to help, please see this thread and either respond directly there or send me a pull request. Thanks.

alain · 2012-11-27 08:40:05

Hi graysky,

I'm not familiar with bash script, so I can't help you so much. I feel that converting DIR into an array will be the simplest solution (it is not necessary for BACKUP and BROWSER, as the former can be calculated from DIR[index] and the latter doesn't change).
I will keep an eye on the other thread in case i can help...

Thank you again!!
Alain

barraponto · 2012-11-27 10:35:17

@Graysky my issues are gone with 4.x version.

I was thinking: can we add some behavior to avoid moving ReadItLater files to tpmfs, but symlinking it? ReadItLater is an extension, now rebranded Pocket, that can be configured to download your reading lists, with assets. I sometimes end up with hundreds of megabytes in my profile. And it's a quite popular extension.

graysky · 2012-11-27 22:14:26

@barraponto - Glad to hear it. You'll need to explain this readitlater/pocket thing. Which browser? Into which dir on the filesystem does it write lots of data? You're asking if it can be omitted from the sync?

graysky · 2012-11-27 22:20:02

Can I ask some of your guys to test out version 5 of psd? It isn't a total re-write but is has significantly changed the way psd works (internal code) thanks to both aesiris and alai. Note - I am assuming that you guys all have good back up strategies and have them freshened before running the testing version.

New features:
* psd will now parse through firefox/aurora ~/.mozilla/<BROWSER>/profiles.ini for syncs. This is powerful for users who keep their profiles outside of their home directory.
* psd will not start if managed browser(s) are running by a managed user. This is to safeguard the profiles against data loss.

I have been running it on my machine and have been banging away at it with no problems. I do want more people testing it before I call it 'stable' and update the stable branch.

https://github.com/graysky2/profile-syn … ync-daemon

How to test it
!STOP PSD FIRST!

1) Close your browsers
2) sudo systemctl stop psd.service
3) Copy the unstable code from github to your /usr/bin and `sudo chmod +x /usr/bin/profile-sync-daemon`
4) Start the daemon again

pingpong · 2012-11-28 07:15:32

@garysky

Looks like psd nows creates a symbolic link of the profile directory.
Could it be the reason why profile-cleaner isn't working anymore for firefox?
(It looks for the files in .mozilla/firefox/ directory.)

Edit: Just verified above by disabling psd and profile-cleaner works again.
So a workaround would be stop psd, clean the database then restart.

Last edited by pingpong (2012-11-28 07:35:49)

graysky · 2012-11-28 10:00:57

pingpong wrote:

...
Could it be the reason why profile-cleaner isn't working anymore for firefox?
(It looks for the files in .mozilla/firefox/ directory.)
Edit: Just verified above by disabling psd and profile-cleaner works again.
So a workaround would be stop psd, clean the database then restart.

Just a minor update I need to make to profile-cleaner. Thanks for the reminder.

Last edited by graysky (2012-11-28 22:16:39)

Arch Linux

#226 2012-11-13 17:18:16

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#227 2012-11-13 18:48:32

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#228 2012-11-13 19:43:24

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#229 2012-11-13 19:54:08

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#230 2012-11-20 15:04:20

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#231 2012-11-20 15:16:28

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#232 2012-11-20 15:27:37

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#233 2012-11-20 15:51:17

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#234 2012-11-20 17:50:33

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#235 2012-11-21 08:14:21

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#236 2012-11-22 11:25:05

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#237 2012-11-22 15:19:36

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#238 2012-11-22 23:48:08

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#239 2012-11-23 08:32:43

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#240 2012-11-23 12:23:21

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#241 2012-11-23 13:27:07

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#242 2012-11-23 14:08:40

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#243 2012-11-23 14:15:30

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#244 2012-11-26 20:45:04

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#245 2012-11-27 08:40:05

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#246 2012-11-27 10:35:17

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#247 2012-11-27 22:14:26

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#248 2012-11-27 22:20:02

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#249 2012-11-28 07:15:32

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

#250 2012-11-28 10:00:57

Re: profile-sync-daemon - keep browser profiles in tmpfs and sync'ed

Board footer