Recomendations for backing up my programming projects folder?

MarthaParkin · 2016-06-15 07:59:19

Bear in mind my total projects directory size is less than 3 mbs it is just I need it stored safely in several places and done regularly.

I had been using sync.com as a cloud service but they kindly have not bothered to make a linux version and I have now moved to purely linux so they are out.

I aim to have my projects backed up in 3 different places. So I would have it on my hard drive I work from of course; the sync cloud and the third I would periodically save to another part of my hard drive. I know it is better to save to a usb -rather than the same drive as it defeats the object somewhat- as a third which is what I plan to do I just never got round to it.

I'm wondering if there is any need for a cloud backup; having to deal with all the different proprietary services' rules and nuances of their software. I originally didn't bother with dropbox because they make it difficult to sync a folder that isn't in 'my documents'; you had to have the main folder in the dropbox folder and symbolic link to it whereas I wanted it the other way round.

I already use syncthing to sync files between my main comp and a server I use but that isn't really a backup since I don't plan to use that server long term it is just a monthly rental. The purpose of that is not having to edit projects multiple times and so I only have to edit the project on my local comp and it is edited on the server.

With the self made syncing like syncthing the drawback is I don't want to have say my own separate server on all the time at home since I am not producing huge amounts of work all the time which would need something to be on all the time vs a cloud backup which is convenient since you are always connected to the internet.

So now sync.com is out I was wondering what you guys would recommend. I do prefer the idea of my own set up so I don't have to rely on these third parties. I just need it to be reliable and something that is semi automated/doesn't require alot of hassle such that I would forget or avoid it.

I will start backing up to usb drive every couple of weeks but I'm thinking of how to cover the cloud service that I lost.

I have had recommendations to use git but isn't it not suited to the task since I want to backup the whole projects folder and not individual projects and it would be a pain having to track the changes? I dont really know as I havent learned git properly but that is what people have said.

I would like to be able to upload to some kind of cloud storage for peace of mind though I think.

Last edited by MarthaParkin (2016-06-15 07:59:48)

madpierre · 2016-06-15 08:24:53

I would like to be able to upload to some kind of cloud storage for peace of mind though I think.

The "peace of mind" part of your statement strikes me as funny- not in a bad or insulting way mind you. Basically, you're saying that you would rest better knowing that your prized data is on some unknown machine in an unknown location managed and "protected" by unknown persons of unknown character, all the while depending on an infrastructure of unknown distance, security, and health for unwavering and reliable access... Kidding aside, under some circumstances, the "cloud" can be an invaluable asset- i.e., if you travel a lot for business and/or pleasure, IMHO.

Frankly, I prefer the use of an encrypted external HDD and use Grsync for backups.

All the best.

Last edited by madpierre (2016-06-15 08:31:33)

dakota · 2016-06-15 08:29:12

Would rsync meet your needs?

You could set it up to automatically sync a folder on your hard drive to an external USB drive, say, every hour.

chris.m · 2016-06-15 08:34:48

Why are you not using a code hosting service like GitHub or BitBucket. Code repositories along with the versioning services they provide are the only way to go. You might think it seems like overkill for small projects, but its not. Our creations are important to us ... hence the reason you want a good backup system for your code

nbd · 2016-06-15 08:49:36

chris.m wrote:

Why are you not using a code hosting service like GitHub or BitBucket.

What if a project contains some valuable proprietary algorithm or a proprietary research in a new software technology...

jasonwryan · 2016-06-15 08:51:33

Tarsnap...

MarthaParkin · 2016-06-15 09:36:17

@madpierre I am not really worried about security but rather accidental deletion somehow. This is especially important now I'm using linux exclusively rather than windows because I'm still quite green to it and commands are irrevocable and no confirmations etc (I know I could add confirmations Im not looking for them particularly so long as my stuff if backed up so then Im free to experiment). My projects aren't worth much to other people. They are for a certain niche but that niche of ppl I doubt would be hackers trying to access the sites.

I saw tarsnap earlier. I'll read up on it.
Edit: tarsnap is no good I want free options only

I have been reading up on 'bup' which arch also has a repository for. Since it is based on git would it be compatible to upload it to bitbucket etc?

I am currently thinking bup for the whole project directory and git for individual projects.

Im leaning towards a git upload solution since it is such a huge website/movement/whatever you call it and well reputed and not likely to disappear any time soon unlike some of these other cloud services. One in one of the reviews I was reading had been shut down since the post had been made.

Last edited by MarthaParkin (2016-06-15 09:54:48)

chris.m · 2016-06-15 15:33:20

nbd wrote:

chris.m wrote:
Why are you not using a code hosting service like GitHub or BitBucket.
What if a project contains some valuable proprietary algorithm or a proprietary research in a new software technology...

Both of the repository examples I gave have options for private uploads. They are paid features on both, however, iirc BitBucket does allow 5 free private repositories.

teateawhy · 2016-06-15 17:10:45

What i did.
Get a raspberry pi, and plug in a usb stick. Install arch linux ARM.
Install deja-dup on my computer. Configure deja-dup to backup to the pi's usb stick every day.
The PI is slow but it is good enough for small files and few users, and it uses so little energy it's ok to keep it running.
For code you could run Gitlab on the pi.

Last edited by teateawhy (2016-06-15 17:12:52)

drcouzelis · 2016-06-15 17:51:48

Many people are suggesting local backup options but, isn't the original question in regards to remote backup options?

...remember, if you don't have a local backup and a remote backup, then you don't have a backup.

ngoonee · 2016-06-15 20:57:36

drcouzelis wrote:

Many people are suggesting local backup options but, isn't the original question in regards to remote backup options?
...remember, if you don't have a local backup and a remote backup, then you don't have a backup.

Depends, 'remote' (and it's cousin 'off-site') aren't scientific definitions of anything. For most people simply 'in a different room' is remote enough. For others, it would have to be a different state.

Steef435 · 2016-06-15 22:11:24

If you're going for git, GitLab allows for unlimited free private repositories AFAIK. I'm using it for a little private project right now and it works quite well so far. Their CI integration is awesome as well, you can hook up automatic tests (run on their servers for free or on your own) quite easily.

For quicker small backups of unversioned projects I use a free Dropbox account. The sync daemon can be interacted with using a tray icon or using their CLI tool (dropbox-cli in the aur).

There are tons of other (free) cloud storage options around of course.

Durden · 2016-06-16 14:39:48

I setup a VM on Vultr and run OwnCloud (soon to be NextCloud) instance on it with local disk encryption. Then I just keep it all in my owncloud folder and sync it across devices. Costs me 5 bucks a month but I also run it as a mail server and web server so I don't mind.

drcouzelis · 2016-06-16 17:11:27

ngoonee wrote:

Depends, 'remote' (and it's cousin 'off-site') aren't scientific definitions of anything. For most people simply 'in a different room' is remote enough. For others, it would have to be a different state.

That's a good point. My test case is, "If I were to come home and discover that my house had burned down (like my neighbor's did last year) would I still have my wedding photos and my children's baby photos?".

If the answer is "no", then I don't have a backup.

Soukyuu · 2016-06-16 17:43:43

You could take a look at gitlab, it offers free private repo hosting, as well as the server software, so you could self-host it on your own server if you don't trust their service. The package is even in the official arch repos.

edit: Or if you only work on the code yourself, then just push to a remote (bare) git repo located wherever you like.

edit2: I have no idea how I overlooked the other posts (especially Steef435 mentioning gitlab), sorry.

Last edited by Soukyuu (2016-06-16 17:54:27)

jjb2016 · 2016-06-16 22:33:16

I would use rsync. It will enable you to backup to local or remote storage securely (over ssh if you like). It can be setup to do periodic incrimental backups using --link-dest option and a bit of bash scripting. Of course it can also be automated, with cron for example. It's free ...
I use it for daily incrimental backups of all of my personal data and system config files - works wonderfully. If you want more info on how-to let me know.

numb3rz · 2016-06-16 22:54:04

GitHub should suffice most of your needs, though IIRC GitLab offers their software for free so you can host your own git repositories. If you're not comfortable with creating a separate git project for each small project then I guess you could give a go at a Duply + Amazon S3/Glacier setup.

Last edited by numb3rz (2016-06-16 22:56:29)

Arch Linux

#1 2016-06-15 07:59:19

Recomendations for backing up my programming projects folder?

#2 2016-06-15 08:24:53

Re: Recomendations for backing up my programming projects folder?

#3 2016-06-15 08:29:12

Re: Recomendations for backing up my programming projects folder?

#4 2016-06-15 08:34:48

Re: Recomendations for backing up my programming projects folder?

#5 2016-06-15 08:49:36

Re: Recomendations for backing up my programming projects folder?

#6 2016-06-15 08:51:33

Re: Recomendations for backing up my programming projects folder?

#7 2016-06-15 09:36:17

Re: Recomendations for backing up my programming projects folder?

#8 2016-06-15 15:33:20

Re: Recomendations for backing up my programming projects folder?

#9 2016-06-15 17:10:45

Re: Recomendations for backing up my programming projects folder?

#10 2016-06-15 17:51:48

Re: Recomendations for backing up my programming projects folder?

#11 2016-06-15 20:57:36

Re: Recomendations for backing up my programming projects folder?

#12 2016-06-15 22:11:24

Re: Recomendations for backing up my programming projects folder?

#13 2016-06-16 14:39:48

Re: Recomendations for backing up my programming projects folder?

#14 2016-06-16 17:11:27

Re: Recomendations for backing up my programming projects folder?

#15 2016-06-16 17:43:43

Re: Recomendations for backing up my programming projects folder?

#16 2016-06-16 22:33:16

Re: Recomendations for backing up my programming projects folder?

#17 2016-06-16 22:54:04

Re: Recomendations for backing up my programming projects folder?

Board footer