You are not logged in.

#1 2011-03-27 13:28:01

Dieter@be
Forum Fellow
From: Belgium
Registered: 2006-11-05
Posts: 1,959
Website

Dvcs-autosync: An open source dropbox clone... well.. almost

I found the Dvcs-autosync project on the vcs-home mailing list, which btw is a great list for folks who are doing stuff like maintaining their home directory in a vcs.
In short:

-  simple python tool (600 sloc), works with your dvcs of choice (mainly tested/used with git)
-  watches for inotify events, performs commits on changes (coalesces some changes together)
-  synchronizes with other clones (remotes), uses xmpp for push notifications
-  let's you know what's going on through libnotify popups or whatever
-  home page: http://mayrhofer.eu.org/dvcs-autosync
-  dvcs-autosync gitorious repo: http://gitorious.org/dvcs-autosync/dvcs-autosync
-  initial announcement on vcs-home: http://lists.madduck.net/pipermail/vcs- … 00314.html
-  AUR git package: http://aur.archlinux.org/packages.php?ID=47643

Use cases:

-  you have one or more trees you want to maintain under a VCS (because you want vcs advantages like history and whatnot) but it's not worth to spend time comitting manually
-  you want to backup some files transparently
-  you want to share / work with others easily
-  you like the idea of dropbox, but not the closed-source-ness or the vendor dependence
-  you work mainly with relatively small files (or.. read on)

Thoughts:

-  very simple to get started
-  simpler (and imho saner) code and implementation in comparison to sparkleshare
-  bound to the limitations of the dvcs. In case of git: no support for ownership, xattrs, and less suited for bigger files (although git-annex might help?)
-  You cannot store a git repo inside a git repo, I think (i.e. I don't think you can keep a -potentially dirty- git clone in dvcs-autosync)
-  For a real, filesystem-level dropbox-alike coda might be a better option, though I'm not sure how useful that project is right now

If it sounds like something you need, try it out. And contributions welcome.
Next up to the todolist: more clever heuristic for event coalescing, setting up bugtracker.


< Daenyth> and he works prolifically
4 8 15 16 23 42

Offline

#2 2011-03-30 00:44:35

Floft
Member
Registered: 2009-11-15
Posts: 36
Website

Re: Dvcs-autosync: An open source dropbox clone... well.. almost

I've been playing around with it for the last few days. The only thing I noticed is that I didn't see an option not to show notifications (in the version I had). That's easily fixed though... the way I did it: link removed I do like it overall.

Last edited by Floft (2011-06-16 19:04:54)

Offline

#3 2011-03-30 09:39:35

Dieter@be
Forum Fellow
From: Belgium
Registered: 2006-11-05
Posts: 1,959
Website

Re: Dvcs-autosync: An open source dropbox clone... well.. almost

that seems like a useful option; if you have github or gitorious (the project is on both) you can do a pull request there


< Daenyth> and he works prolifically
4 8 15 16 23 42

Offline

#4 2011-04-01 02:25:37

ltpl4y3r
Member
Registered: 2008-08-23
Posts: 28

Re: Dvcs-autosync: An open source dropbox clone... well.. almost

Do you know of any good resources for learning how to set this up from scratch? I am hoping to use this to sync between 3 different PCs and have never setup/used git without a PKGBUILD.

Offline

#5 2011-04-01 08:55:25

Dieter@be
Forum Fellow
From: Belgium
Registered: 2006-11-05
Posts: 1,959
Website

Re: Dvcs-autosync: An open source dropbox clone... well.. almost

ltpl4y3r wrote:

Do you know of any good resources for learning how to set this up from scratch? I am hoping to use this to sync between 3 different PCs and have never setup/used git without a PKGBUILD.

just install it from AUR with a tool like clyde, setting up is easy; instructions are on the project website.


< Daenyth> and he works prolifically
4 8 15 16 23 42

Offline

#6 2011-04-01 11:54:55

skodabenz
Banned
From: Tamilnadu, India
Registered: 2010-04-11
Posts: 382

Re: Dvcs-autosync: An open source dropbox clone... well.. almost

I wonder whether it allows maintaining custom arch repos like some users do you in dropbox.


My new forum user/nick name is "the.ridikulus.rat" .

Offline

#7 2011-04-01 20:17:15

Dieter@be
Forum Fellow
From: Belgium
Registered: 2006-11-05
Posts: 1,959
Website

Re: Dvcs-autosync: An open source dropbox clone... well.. almost

skodabenz wrote:

I wonder whether it allows maintaining custom arch repos like some users do you in dropbox.

hmm that's another interesting use of dropbox (or clone thereof).

answer: it does allow it, you just need to be aware that storing big files in git is not very efficient (your disk usage doubles), until we've figured out how to use git-annex in dvcs-autosync smile


< Daenyth> and he works prolifically
4 8 15 16 23 42

Offline

#8 2011-04-01 21:12:24

smartboyathome
Member
From: $HOME
Registered: 2007-12-23
Posts: 334
Website

Re: Dvcs-autosync: An open source dropbox clone... well.. almost

It looks like a project called Sharepoint on Github is doing something similar, and even uses python as well. Perhaps you two could combine efforts or could learn from each others' code?

Offline

#9 2011-04-02 09:01:09

Dieter@be
Forum Fellow
From: Belgium
Registered: 2006-11-05
Posts: 1,959
Website

Re: Dvcs-autosync: An open source dropbox clone... well.. almost

smartboyathome wrote:

It looks like a project called Sharepoint on Github is doing something similar, and even uses python as well. Perhaps you two could combine efforts or could learn from each others' code?

it's sharebox, and.. way ahead of you wink
see the comments @ http://dieter.plaetinck.be/dvcs-autosyn … lmost.html and discussion @ http://lists.madduck.net/pipermail/vcs- … 00351.html


< Daenyth> and he works prolifically
4 8 15 16 23 42

Offline

#10 2011-04-06 06:47:22

ngoonee
Forum Fellow
From: Between Thailand and Singapore
Registered: 2009-03-17
Posts: 6,839

Re: Dvcs-autosync: An open source dropbox clone... well.. almost

How useful would this be for automatically updating a local git repo? In other words I don't want to use all the networked stuff. Just something which would add . and commit -a whenever inotify says something happened.


Allan-Volunteer on the (topic being discussed) mailn lists. You never get the people who matters attention on the forums.
jasonwryan-Installing Arch is a measure of your literacy. Maintaining Arch is a measure of your diligence. Contributing to Arch is a measure of your competence.
Griemak-Bleeding edge, not bleeding flat. Edge denotes falls will occur from time to time. Bring your own parachute.

Offline

#11 2011-04-07 03:10:54

ltpl4y3r
Member
Registered: 2008-08-23
Posts: 28

Re: Dvcs-autosync: An open source dropbox clone... well.. almost

Okay, I have managed to get it working, for the most part. I think your instructions may be missing:

git add -S

before the first commit.

Anyway, I was wondering what use the login information is in the config. Am I going to need to make it so that it can ssh in w/o the need to supply a password?

DEBUG:root:Starting push timer with 5 seconds until push would occur (if no other changes happen in between)
DEBUG:root:Tick 1 / 5
DEBUG:root:Tick 2 / 5
DEBUG:root:Starting coalesce timer with 2 seconds until coalescing events for file /home/nick/autosync/01 Oya Shirazu.mp3 would occur (if no other changes happen in between)
DEBUG:root:Resetting already active coalesce timer to new timeout of 2 seconds until coalescing events for file /home/nick/autosync/01 Oya Shirazu.mp3 would occur
DEBUG:root:Tick 3 / 5
DEBUG:root:Tick 4 / 5
DEBUG:root:Tick 5 / 5
INFO:root:NOTIFICATION: Pushing changes: Pushing last local changes to remote repository
dropbox@192.168.2.201's password: INFO:root:Coalesce event triggered for file /home/nick/autosync/01 Oya Shirazu.mp3

I would like to note that to log into the server, it is a different login name and password than the workstation. Any hints on how to proceed?

EDIT:
Sorry, doesn't seem to be syncing between computers.. Back to the drawing board.

Last edited by ltpl4y3r (2011-04-07 03:13:47)

Offline

#12 2011-04-07 12:21:22

Dieter@be
Forum Fellow
From: Belgium
Registered: 2006-11-05
Posts: 1,959
Website

Re: Dvcs-autosync: An open source dropbox clone... well.. almost

ltpl4y3r wrote:

Okay, I have managed to get it working, for the most part. I think your instructions may be missing:

git add -S

before the first commit.

What? The instructions clearly mention to add to the index.

ltpl4y3r wrote:

Anyway, I was wondering what use the login information is in the config. Am I going to need to make it so that it can ssh in w/o the need to supply a password?

DEBUG:root:Starting push timer with 5 seconds until push would occur (if no other changes happen in between)
DEBUG:root:Tick 1 / 5
DEBUG:root:Tick 2 / 5
DEBUG:root:Starting coalesce timer with 2 seconds until coalescing events for file /home/nick/autosync/01 Oya Shirazu.mp3 would occur (if no other changes happen in between)
DEBUG:root:Resetting already active coalesce timer to new timeout of 2 seconds until coalescing events for file /home/nick/autosync/01 Oya Shirazu.mp3 would occur
DEBUG:root:Tick 3 / 5
DEBUG:root:Tick 4 / 5
DEBUG:root:Tick 5 / 5
INFO:root:NOTIFICATION: Pushing changes: Pushing last local changes to remote repository
dropbox@192.168.2.201's password: INFO:root:Coalesce event triggered for file /home/nick/autosync/01 Oya Shirazu.mp3

I would like to note that to log into the server, it is a different login name and password than the workstation. Any hints on how to proceed?

EDIT:
Sorry, doesn't seem to be syncing between computers.. Back to the drawing board.

use ssh keys


< Daenyth> and he works prolifically
4 8 15 16 23 42

Offline

#13 2011-04-16 09:53:32

ngoonee
Forum Fellow
From: Between Thailand and Singapore
Registered: 2009-03-17
Posts: 6,839

Re: Dvcs-autosync: An open source dropbox clone... well.. almost

ngoonee wrote:

How useful would this be for automatically updating a local git repo? In other words I don't want to use all the networked stuff. Just something which would add . and commit -a whenever inotify says something happened.

Tried this out, but keep getting the following:-

DEBUG:root:AutosyncJabberbot:unable to connect to server your XMPP id here.
Exception in thread Thread-7:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 530, in __bootstrap_inner
    self.run()
  File "/usr/bin/autosync.py", line 139, in run
    self.expire(self.arg)
  File "/usr/bin/autosync.py", line 401, in _real_push
    bot.send(sendto, 'pushed %s' % remoteurl)
  File "/usr/lib/python2.7/site-packages/jabberbot.py", line 179, in send
    self.send_message(mess)
  File "/usr/lib/python2.7/site-packages/jabberbot.py", line 166, in send_message
    self.connect().send(mess)
AttributeError: 'NoneType' object has no attribute 'send'

Of course, I have not setup any xmpp details. Doesn't seem to affect proper functioning (so far, fingers crossed). So, a question and a request:-

1. I guess wouldn't make sense to have this script without the networking bits, but if I wish to run it that way is it 'safe'? I assume nothing gets touched in the base directory in any case.
2. Could the .autosync-example file be included in the built package? Maybe as /etc/xdg/autosync/example or /usr/share/examples/something. Would make more sense, IMHO.


Allan-Volunteer on the (topic being discussed) mailn lists. You never get the people who matters attention on the forums.
jasonwryan-Installing Arch is a measure of your literacy. Maintaining Arch is a measure of your diligence. Contributing to Arch is a measure of your competence.
Griemak-Bleeding edge, not bleeding flat. Edge denotes falls will occur from time to time. Bring your own parachute.

Offline

#14 2011-04-16 10:09:42

Dieter@be
Forum Fellow
From: Belgium
Registered: 2006-11-05
Posts: 1,959
Website

Re: Dvcs-autosync: An open source dropbox clone... well.. almost

ngoonee wrote:

1. I guess wouldn't make sense to have this script without the networking bits, but if I wish to run it that way is it 'safe'? I assume nothing gets touched in the base directory in any case.

Not sure what you mean, but supporting `syncmethod = none` seems fairly trivial. (the variable already exists, but we don't use it yet)

ngoonee wrote:

2. Could the .autosync-example file be included in the built package? Maybe as /etc/xdg/autosync/example or /usr/share/examples/something. Would make more sense, IMHO.

true.
Right now my package copies some files, but Rene was gonna implement setup.py/python distutils/setuptools whatever that thing is called. So i'm waiting a bit for that to update my package.

Either way, you can create tickets for both issues @ https://github.com/rmayr/dvcs-autosync (or better: do pull requests smile


< Daenyth> and he works prolifically
4 8 15 16 23 42

Offline

#15 2011-04-16 10:27:42

ngoonee
Forum Fellow
From: Between Thailand and Singapore
Registered: 2009-03-17
Posts: 6,839

Re: Dvcs-autosync: An open source dropbox clone... well.. almost

Dieter@be wrote:
ngoonee wrote:

1. I guess wouldn't make sense to have this script without the networking bits, but if I wish to run it that way is it 'safe'? I assume nothing gets touched in the base directory in any case.

Not sure what you mean, but supporting `syncmethod = none` seems fairly trivial. (the variable already exists, but we don't use it yet)

Yes, that would be swell!

Dieter@be wrote:
ngoonee wrote:

2. Could the .autosync-example file be included in the built package? Maybe as /etc/xdg/autosync/example or /usr/share/examples/something. Would make more sense, IMHO.

true.
Right now my package copies some files, but Rene was gonna implement setup.py/python distutils/setuptools whatever that thing is called. So i'm waiting a bit for that to update my package.

Either way, you can create tickets for both issues @ https://github.com/rmayr/dvcs-autosync (or better: do pull requests smile

Hmm, the second is more a packaging suggestion for you, but yes if a 'proper' packaging is planned then I'll wait for that. I'm a python illiterate currently so I'll just file a ticket for the first smile. Thanks. Btw, why a github address when the project is gitorious?


Allan-Volunteer on the (topic being discussed) mailn lists. You never get the people who matters attention on the forums.
jasonwryan-Installing Arch is a measure of your literacy. Maintaining Arch is a measure of your diligence. Contributing to Arch is a measure of your competence.
Griemak-Bleeding edge, not bleeding flat. Edge denotes falls will occur from time to time. Bring your own parachute.

Offline

#16 2011-04-16 10:38:44

Dieter@be
Forum Fellow
From: Belgium
Registered: 2006-11-05
Posts: 1,959
Website

Re: Dvcs-autosync: An open source dropbox clone... well.. almost

Dieter@be wrote:

Hmm, the second is more a packaging suggestion for you, but yes if a 'proper' packaging is planned then I'll wait for that. I'm a python illiterate currently so I'll just file a ticket for the first smile. Thanks. Btw, why a github address when the project is gitorious?

we're still evaluating different options.  Rene is more of a gitorious fan (but it doesn't have a bugtracker), I'm the github fan.
the 2nd issue is definitely not just a packaging issue.  AFAIK the setup.py thingie should take care of putting all files where they belong (for a given prefix / "root tree"), so that the PKGBUILD file can just call python2 setup.py, similar to how you would deal with makefiles.  I'ld still make a ticket for your issue so that you're sure it doesn't get forgotten.


< Daenyth> and he works prolifically
4 8 15 16 23 42

Offline

Board footer

Powered by FluxBB