[SOLVED] Version control for ebooks, music, and pictures

catnap · 2017-06-30 21:23:29

I have fallen into the habit of keeping my code neatly under a version control. This can easily be accomplished by Bazaar and Git. Having a system of version control like this, allows me to easily synchronize my code between multiple computers and external hard drives. However, the problem becomes more difficult when dealing with ebooks, music, and pictures. These files take up so much space that version controling them is not efficient. Indeed, I have tried to put these files too under a version control system for the purposes of synchronization but this only resulted in the .bzr/ directory taking up unnecessarily large quantities of space on my hard drive.

Is there a way to "version control the file hiearchy", so that the version control keeps track of new files, deleted files, and renamed files but does not store information about the contents of the files apart from verifying by a checksum that the files have not changed?

Last edited by catnap (2017-07-02 00:41:07)

seth · 2017-07-01 07:27:15

rsync?

catnap · 2017-07-01 11:33:58

RSync is a very potent solution to many of these synchronization problems. But I wonder if it is able to update the file hierarchy between two systems. I have sometimes used the --delete switch to achieve a comparable effect but this incurs the risk of losing data when neither one of the systems is unequivocally more up to date than the other.

twelveeighty · 2017-07-01 12:01:06

catnap wrote:

one [...] is unequivocally more up to date than the other.

Please explain what it is exactly you want to achieve with some examples on when rsync wouldn't produce the desired outcome? For me, being "up to date" is a binary thing: it either "is" or "isn't", there's no grey area in between them.

seth · 2017-07-01 13:14:17

I assume conflict resolution (git merge), ie. when syncing systems A and B:
A/file1
A/file2
B/file1
B/file3

with A and B having mtime after the last sync.
=> Do you want to remove A/file2, B/file3, both or keep both?
This question is not dealt individually with by rsync, but controlled in global by whether you pass the --delete flag and which way you sync first.

This would lead to the question whether this is actually supposed to be a VERSION control system (the general rsync approach here is to cp -al the last backup) or rather a 5 line bash script that compares two directory trees and then asks for every deviation what to do with that file and either cp or rm it on either side.

x33a · 2017-07-01 13:47:10

I remember hearing about a backup tool based on git. Upon searching a bit, I think it was bup: https://github.com/bup/bup

catnap · 2017-07-01 18:55:58

twelveeighty wrote:

Please explain what it is exactly you want to achieve with some examples on when rsync wouldn't produce the desired outcome? For me, being "up to date" is a binary thing: it either "is" or "isn't", there's no grey area in between them.

A fair question. I think seth's answer is directly on the mark. You could need to synchronize two directories, each of which has been changed after the last synchronization. In this situation the --delete flag causes a loss of data.

x33a wrote:

I remember hearing about a backup tool based on git. Upon searching a bit, I think it was bup: https://github.com/bup/bup

I know that the forum is not for software recommendations. But just between you and me—thanks for the software recommendation.

catnap · 2017-07-01 19:43:41

seth wrote:

This would lead to the question whether this is actually supposed to be a VERSION control system

Version control for this situation might not be the traditional thing to do. But if the user wants this functionality, I do not see a problem in the aspiration. I also think, it is something that can practically be achieved, although the program for it might not yet exist. In any case, I could use some details in the simpler solution that does not entail a full version control. Is the aforementioned five line script available somewhere online?

x33a wrote:

I remember hearing about a backup tool based on git. Upon searching a bit, I think it was bup: https://github.com/bup/bup

I tried out bup and it will surely be useful for making backups. However, it seems like the program is not exactly what I need. As the name suggests, it is mostly for backing up data rather than keeping two file hierarchies synchronized.

catnap · 2017-07-01 21:31:01

A program called git-annex seems optimal for version controling large files. I'm trying it out and have had nothing but success so far. See http://git-annex.branchable.com/ for details.

eschwartz · 2017-07-02 06:03:34

Version control implies the need to keep a record of every single state, forever. All you really need is a file sync software which can sync the latest version of a media file to all devices, while deferring to the user whenever a file was modified from two locations without being properly synced.

Essentially, you want Dropbox or perhaps Owncloud.

Arch Linux

#1 2017-06-30 21:23:29

[SOLVED] Version control for ebooks, music, and pictures

#2 2017-07-01 07:27:15

Re: [SOLVED] Version control for ebooks, music, and pictures

#3 2017-07-01 11:33:58

Re: [SOLVED] Version control for ebooks, music, and pictures

#4 2017-07-01 12:01:06

Re: [SOLVED] Version control for ebooks, music, and pictures

#5 2017-07-01 13:14:17

Re: [SOLVED] Version control for ebooks, music, and pictures

#6 2017-07-01 13:47:10

Re: [SOLVED] Version control for ebooks, music, and pictures

#7 2017-07-01 18:55:58

Re: [SOLVED] Version control for ebooks, music, and pictures

#8 2017-07-01 19:43:41

Re: [SOLVED] Version control for ebooks, music, and pictures

#9 2017-07-01 21:31:01

Re: [SOLVED] Version control for ebooks, music, and pictures

#10 2017-07-02 06:03:34

Re: [SOLVED] Version control for ebooks, music, and pictures

Board footer