You are not logged in.

#1 2019-03-11 07:03:28

Xyne
Moderator/TU
Registered: 2008-08-03
Posts: 6,441
Website

alternatives to borg for backups

I've been using borg for backups for about a year on several systems and have been very satisfied with it. However, it recently stopped working due to dependency upgrades (see FS#61346 and FS#61684) and the maintainer has apparently made it clear that he is reluctant to implement easy fixes to ensure that the software continues to work for all users (see the flyspray discussions). The nail in the coffin for me is this: https://github.com/borgbackup/borg/issues/4343

A backup system that deliberately depends on deprecated software is clearly not something to rely on in the future so I am not looking for an alternative asap while I can still recover my data. The main features that I enjoyed in borg were the deduplication and the fuse mounting which made it easy to retrieve backups. Can anyone recommend something similar?

What backup system do you currently use and what features does it have?


edit: Just to be clear, I understand and agree with the argument that the software has to continue to support its original dependencies for minor updates. That doesn't mean that it has to reject support for evolving dependencies. If that were the intention, it should have been made clear that the software is only supported on specific setups.

Last edited by Xyne (2019-03-11 07:11:41)


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#2 2019-03-11 07:10:55

jasonwryan
Anarchist
From: .nz
Registered: 2009-05-09
Posts: 28,123
Website

Re: alternatives to borg for backups

Not exactly what you are looking for, but I have been using tarsnap for years and it is terrific. Colin Percival is the FreeBSD security officer and, as the tagline says, it is a solution for the truly paranoid.

It does deduplication, but because it is on S3, backup retrieval is slow. Slow, but cheap. This has never been an issue for me because the few times I have needed a backup, I'm not in that much of a hurry to get it -- the fact that I can get it is what counts. Waiting 10 minutes (on an NZ crap connection) is not a deal breaker as far as I am concerned.


Arch + dwm   •   Mercurial repos  •   Surfraw

Registered Linux User #482438

Online

#3 2019-03-11 07:14:35

waitnsea
Member
From: France
Registered: 2013-02-10
Posts: 34

Re: alternatives to borg for backups

Xyne wrote:

I've been using borg for backups for about a year on several systems and have been very satisfied with it. However, it recently stopped working due to dependency upgrades (see FS#61346 and FS#61684) and the maintainer has apparently made it clear that he is reluctant to implement easy fixes to ensure that the software continues to work for all users (see the flyspray discussions). The nail in the coffin for me is this: https://github.com/borgbackup/borg/issues/4343

A backup system that deliberately depends on deprecated software is clearly not something to rely on in the future so I am not looking for an alternative asap while I can still recover my data. The main features that I enjoyed in borg were the deduplication and the fuse mounting which made it easy to retrieve backups. Can anyone recommend something similar?

What backup system do you currently use and what features does it have?

Hi Xyne,
Very sad, same as you...
I keep on - for a while - to use my automatized borg-scripts as I can mount and extract backups with the python venv workaround, but I added manually rdiff-backups

Edit : rdiff-backup is very very far behind Borg for speed

Last edited by waitnsea (2019-03-11 08:43:46)

Offline

#4 2019-03-11 07:16:06

Xyne
Moderator/TU
Registered: 2008-08-03
Posts: 6,441
Website

Re: alternatives to borg for backups

Tarsnap looks good for what it is but I would prefer something that backs up to local encrypted disks. The offsite aspect of cloud backups with an extra level of redundancy is appealing but I just can't bring myself to trust my data to the cloud. I appreciate the suggestion nevertheless.


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#5 2019-03-11 07:21:04

jasonwryan
Anarchist
From: .nz
Registered: 2009-05-09
Posts: 28,123
Website

Re: alternatives to borg for backups

Fair enough. I'm the same (re. trusting the cloud): I also rsync all of my machines to a debian server in my garage smile


Arch + dwm   •   Mercurial repos  •   Surfraw

Registered Linux User #482438

Online

#6 2019-03-11 12:34:53

Everette88
Member
Registered: 2018-02-17
Posts: 32

Re: alternatives to borg for backups

You may take a look at restic: https://restic.net/

Offline

#7 2019-03-12 02:05:43

zosodk69
Member
Registered: 2005-01-30
Posts: 30

Re: alternatives to borg for backups

Xyne, I have considerable respect for you and am a consumer of your software.  I have your repository in my pacman.conf file.  I think, though, that in this case you're making the same mistake that many of us do in assuming that Arch users are the ONLY Linux users in existence:

Xyne wrote:

the developer has chosen to force unsupported dependencies on all of his users.

The reality is that this ONLY affects users of rolling releases like Arch.

Borg version 1.1 is probably more appropriately named borg-lts under the typical Arch naming convention.  It is in maintenance mode and the author isn't comfortable making drastic changes that could potentially introduce bugs.  Remember, this is backup software we're talking about.  Whether you or I think the necessary code change is simple is irrelevant.  The author clearly doesn't feel this way, and is uncomfortable introducing bugs into a maintenance release of his backup software.

I support software in enterprise environments and completely respect his risk/reward evaluation.  I think expecting the author to bend over backwards for Arch users, at the expense of the rest of the user base, is an example of the type of tunnel-vision which I personally have been guilty of on occasion.

What really frustrates me is the downright inflammatory discourse that some Arch representatives have directed at the borg author.  I've been pretty impressed with how much class Thomas Waldmann has demonstrated here.  A quick browse of the FS#61346 and you can see a particular TU (who should know better) throwing a millennial-neck-beard-temper-tantrum that his niche Linux distro isn't being respected above all else.  If this person's resume ever crossed my desk, a quick web search would show that he is basically unhireable for anything that involves collaborating on a team.


-=[dave]=-

Offline

#8 2019-03-13 15:42:44

Xyne
Moderator/TU
Registered: 2008-08-03
Posts: 6,441
Website

Re: alternatives to borg for backups

@Everette88
Thanks, I'll take a look at restic.

@zosodk69
I admit that I was indeed annoyed when I first posted. I agree with most of your points (the dev's priorities, the necessity of ensuring lts for a backup system, the unwarranted inflammatory discourse, etc.).I recognize that the problem is mainly the rolling-release nature of Arch and that I initially reacted to what the other Arch users had posted about the dev's reluctance to implement a fix.

I nevertheless would have hoped that the dev would take end-of-life dates of his dependencies* as an incentive to at least provide patches or a development branch so that others can move forward. I realize that this is easier said than done as an open source dev because most of the time the project is a hobby developed on a particular machine and shared with no more than the hope that someone might find it useful as-is.

Anyway, with no expectations or blame, it's clear that borg on Arch is not a reliable backup system despite the merits of borg on other systems.

* As I said, this was the nail in the coffin. I understand not wanting or being able to chase dependency updates on other systems, but relying on deps past their eol does not bode well for future support.


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#9 2019-03-13 16:04:47

zosodk69
Member
Registered: 2005-01-30
Posts: 30

Re: alternatives to borg for backups

@Xyne
I do agree with you that this makes borg a difficult solution to suggest to fellow Arch users.  I personally find the software to be too good to completely dismiss, and worth pursuing workarounds for the time being.  Cloud providers like rsync.net even support borg/attic as remote targets (which is my primary use for borg).

For my purposes, I've created a PKGBUILD that bundles borg and its dependencies into a virtualenv and puts it under /usr/share/borg.  Not ideal but it's a pretty simple solution to continue using Borg while we wait for 1.2 to finish.  I tried to post this as an alternative under the bug report but I got beat down pretty hard and had my attachment removed.  I tried to post to an alternative location to download the PKGBUILD but that was deleted pretty quickly.  I'm hoping I don't get banned from the whole internet for publishing it here:

# Maintainer: Lukas Fleischer <lfleischer@archlinux.org>
# Contributor: Vlad M. <vlad@archlinux.net>
# Contributor: Dave Kil <dave@thekilempire.com>

pkgname=borg
_pkgname=borgbackup
pkgver=1.1.9
pkgrel=2
pkgdesc='Deduplicating backup program with compression and authenticated encryption'
url='https://borgbackup.github.io/'
license=('BSD')
arch=('x86_64')
depends=('acl' 'lz4' 'openssl' 'xz' 'zstd')
optdepends=('openssh: repositories on remote hosts')
makedepends=('python-virtualenv')
checkdepends=('fakeroot')
provides=('borgbackup')
replaces=('borgbackup')
source=("https://github.com/$_pkgname/$pkgname/releases/download/$pkgver/$_pkgname-$pkgver.tar.gz"{,.asc})
sha256sums=('7d0ff84e64c4be35c43ae2c047bb521a94f15b278c2fe63b43950c4836b42575'
            'SKIP')
validpgpkeys=('6D5BEF9ADD2075805747B70F9F88FB52FAF7B393') # Thomas Waldmann <tw@waldmann-edv.de>

prepare() {
  virtualenv --python=python3 "$srcdir/virtualenv-tools3-env"
  source "$srcdir/virtualenv-tools3-env/bin/activate"
  pip install virtualenv-tools3

  virtualenv --python=python3 "$srcdir/borg-env"
}

build() {
  source "$srcdir/borg-env/bin/activate"
  cd "$srcdir/$_pkgname-$pkgver"
  pip install -r requirements.d/development.txt
  pip install -r requirements.d/docs.txt
  pip install -r requirements.d/fuse.txt
	pip install .

  source "$srcdir/virtualenv-tools3-env/bin/activate"
  virtualenv-tools --update-path="/usr/share/borg" "$srcdir/borg-env"
}

#check() {
#  source "$srcdir/borg-env/bin/activate"
#  cd "$srcdir/$_pkgname-$pkgver"
#  LANG=en_US.UTF-8 PYTHONPATH="$PWD:$PYTHONPATH" py.test --cov=borg \
#    --benchmark-skip --pyargs borg.testsuite -v \
#    -k 'not test_non_ascii_acl'
#}

package() {
  install -d "$pkgdir/usr/share/borg"
  cp -a "$srcdir/borg-env/"{include,lib,lib64} "$pkgdir/usr/share/borg"

  install -Dm755 "$srcdir/borg-env/bin/borg" \
    "$pkgdir/usr/bin/borg"
  install -Dm755 "$srcdir/borg-env/bin/borgfs" \
    "$pkgdir/usr/bin/borgfs"
  install -Dm755 "$srcdir/borg-env/bin/python3" \
    "$pkgdir/usr/share/borg/bin/python3"

  sed -i 's|^#!.*|#!/usr/share/borg/bin/python3|g' "$pkgdir/usr/bin/borg"
  sed -i 's|^#!.*|#!/usr/share/borg/bin/python3|g' "$pkgdir/usr/bin/borgfs"

  install -Dm644 "$srcdir/$_pkgname-$pkgver/scripts/shell_completions/bash/borg" \
    "$pkgdir/usr/share/bash-completion/completions/borg"
  install -Dm644 "$srcdir/$_pkgname-$pkgver/scripts/shell_completions/fish/borg.fish" \
    "$pkgdir/usr/share/fish/vendor_completions.d/borg.fish"
  install -Dm644 "$srcdir/$_pkgname-$pkgver/scripts/shell_completions/zsh/_borg" \
    "$pkgdir/usr/share/zsh/site-functions/_borg"

  install -Dm644 -t "$pkgdir/usr/share/man/man1/" "$srcdir/$_pkgname-$pkgver/docs/man/"*.1
  install -Dm644 "$srcdir/$_pkgname-$pkgver/LICENSE" "$pkgdir/usr/share/licenses/$pkgname/LICENSE"
}

-=[dave]=-

Offline

#10 2019-03-13 17:22:12

waitnsea
Member
From: France
Registered: 2013-02-10
Posts: 34

Re: alternatives to borg for backups

zosodk69 wrote:

For my purposes, I've created a PKGBUILD that bundles borg and its dependencies into a virtualenv and puts it under /usr/share/borg.  Not ideal but it's a pretty simple solution to continue using Borg while we wait for 1.2 to finish.  I tried to post this as an alternative under the bug report but I got beat down pretty hard and had my attachment removed.  I tried to post to an alternative location to download the PKGBUILD but that was deleted pretty quickly.  I'm hoping I don't get banned from the whole internet for publishing it here:

Thank you for the PKGBUILD - works well and I saved it in my archives just in case - it allows now the borg mount and borg extract commands
Note : I before installation had to receive GPG keys as follow :

$ gpg --keyserver keyserver.ubuntu.com --recv-keys "6D5B EF9A DD20 7580 5747 B70F 9F88 FB52 FAF7 B393"

Last edited by waitnsea (2019-03-13 17:45:01)

Offline

#11 2019-03-13 17:29:06

Alad
Wiki Admin/IRC Op/TU
From: Bagelstan
Registered: 2014-05-04
Posts: 2,004
Website

Re: alternatives to borg for backups

I tried to post this as an alternative under the bug report but I got beat down pretty hard and had my attachment removed.

That's just disgraceful, as is much of the other behavior from that particular TU. Thanks for posting your PKGBUILD here so that others can benefit.


Mods are just community members who have the occasionally necessary option to move threads around and edit posts. -- Trilby
Honest Alad's Package Emporium—Now with added bugs! (Grand reopening: December 1st 2018)

Offline

#12 2019-03-14 04:13:34

ngoonee
Forum Fellow
From: Between Thailand and Singapore
Registered: 2009-03-17
Posts: 7,244

Re: alternatives to borg for backups

Alad wrote:

I tried to post this as an alternative under the bug report but I got beat down pretty hard and had my attachment removed.

That's just disgraceful, as is much of the other behavior from that particular TU. Thanks for posting your PKGBUILD here so that others can benefit.

Without comment on 'other behaviour', the posted PKGBUILD is a hack and really shouldn't be distributed....


Allan-Volunteer on the (topic being discussed) mailn lists. You never get the people who matters attention on the forums.
jasonwryan-Installing Arch is a measure of your literacy. Maintaining Arch is a measure of your diligence. Contributing to Arch is a measure of your competence.
Griemak-Bleeding edge, not bleeding flat. Edge denotes falls will occur from time to time. Bring your own parachute.

Offline

#13 2019-03-14 05:37:08

waitnsea
Member
From: France
Registered: 2013-02-10
Posts: 34

Re: alternatives to borg for backups

ngoonee wrote:

the posted PKGBUILD is a hack and really shouldn't be distributed....

Happily, it has been... I reiterate my thanks

Offline

#14 2019-03-15 16:40:55

galvez_65
Member
Registered: 2018-06-29
Posts: 6

Re: alternatives to borg for backups

I know using Borg directly from Git is not recommend, but I've been using it now for about a month and it's working great. The issue is one it gets rebuilt often, and a regression error could creep in because it's still beta. But just saying 1.2 looks good.

Offline

#15 2019-04-16 05:41:57

bulletmark
Member
From: Brisbane, Australia
Registered: 2013-10-22
Posts: 439

Re: alternatives to borg for backups

Given that a backup program needs to be extremely stable and given that a python source program on Arch will always be perhaps a little brittle to Arch's bleeding edge python version changes, perhaps the Arch borg packager may instead consider basing the borg package on the official standalone binary releases? Alternately, would anybody object if I added (and maintained) a borg-bin package in the AUR for that? I have created a simple PKGBUILD that builds 64 bit, 32 bit, and various ARM packages.

Offline

#16 2019-04-16 06:25:42

waitnsea
Member
From: France
Registered: 2013-02-10
Posts: 34

Re: alternatives to borg for backups

bulletmark wrote:

basing the borg package on the official standalone binary releases ...simple PKGBUILD that builds 64 bit, 32 bit, and various ARM packages.

Hi,
It works well, and if you could maintain it for every stable version which will appear, it could be a very good deal for my needs. Thank you

Offline

#17 2019-04-16 06:29:15

bulletmark
Member
From: Brisbane, Australia
Registered: 2013-10-22
Posts: 439

Re: alternatives to borg for backups

waitnsea wrote:

if you could maintain it for every stable version which will appear, it could be a very good deal for my needs.

Yes, of course I would update immediately at every new borg release. Borg releases are fairly infrequent so that is easy to do.

Offline

#18 2019-04-16 06:39:34

waitnsea
Member
From: France
Registered: 2013-02-10
Posts: 34

Re: alternatives to borg for backups

No other backup program brings performance, speed, and services  of borg
Your solution could be good enough to go to community/ instead of aur/

Offline

#19 2019-04-16 07:22:02

fukawi2
Administrator
From: .vic.au
Registered: 2007-09-28
Posts: 5,758
Website

Re: alternatives to borg for backups

Everette88 wrote:

You may take a look at restic: https://restic.net/

I used borg at one stage, and went through duplicity and attic as well... I found restic to be by far the best IMHO. Simple, fast, reliable. I would strongly recommend anyone looking for a new backup tool to at least evaluate it.

Offline

#20 2019-04-16 09:53:55

waitnsea
Member
From: France
Registered: 2013-02-10
Posts: 34

Re: alternatives to borg for backups

fukawi2 wrote:
Everette88 wrote:

You may take a look at restic: https://restic.net/

I used borg at one stage, and went through duplicity and attic as well... I found restic to be by far the best IMHO. Simple, fast, reliable. I would strongly recommend anyone looking for a new backup tool to at least evaluate it.

Hi fukawi2
Following you I Tried again restic - and again couldn't get the equivalent of the(very fast) "Mount" function in borg (to only see and control then simply unmount) - neither to include hidden files
Neithertheless it gave a speed backup and is a good option

Offline

#21 2019-04-16 11:06:02

fukawi2
Administrator
From: .vic.au
Registered: 2007-09-28
Posts: 5,758
Website

Re: alternatives to borg for backups

waitnsea wrote:

equivalent of the(very fast) "Mount" function in borg (to only see and control then simply unmount)

restic has a 'mount' option, although I don't use it. Are you saying it doesn't work how you need/expect?

Offline

#22 2019-04-16 16:08:04

waitnsea
Member
From: France
Registered: 2013-02-10
Posts: 34

Re: alternatives to borg for backups

You're right : restic-mount, sorry I had missed it...

Offline

#23 2019-04-16 22:33:43

bulletmark
Member
From: Brisbane, Australia
Registered: 2013-10-22
Posts: 439

Re: alternatives to borg for backups

I added borg-bin to the AUR.

Offline

#24 2019-04-19 14:05:19

Thme
Member
From: Raleigh NC
Registered: 2012-01-22
Posts: 105

Re: alternatives to borg for backups

I personaly use rsync and a separate encrypted partition, hard linking options for root and dotfile directory trees. Wrote a script a few years ago to automate and rotate backups locally.  I do mine locally however there's many other uses and though it's old it can be used in conjuction with SSH to transfer securely over a network.  Here's the link to my github https://github.com/Thme/rotorsync I'm sure it can be re-purposed. Rsync has few dependencies and is pretty universal on most systems. I can fix a borked system by reversing the paths and using the --del option to restore any backup
Also note the script uses --link-dest= and links to the previous backup this creates a chain of differential backups with less disk usage and the simplicity that removing the old backups including the first won't break the chain.

Edit:This script was mainly written for personal use on a laptop. My goal was to have a configurable number backups(7 for me) and frequency(in my case daily) of backups. I kept a snapshot partition approximately 2.5 times the size of my root partition and 7 rotating snapshots with rsync never execeded the partitions limit in fact those 7 weren't much bigger than my root(maybe +3-5gb at most depending on how much I actually changed or updated)

Last edited by Thme (2019-04-19 18:37:55)


"Hidden are the ways for those who pass by, for light is perished and darkness comes into being." Nephthys:
Ancient Egyptian Coffin Texts

Offline

#25 2019-05-01 10:10:17

zerophase
Member
Registered: 2015-09-03
Posts: 189

Re: alternatives to borg for backups

We could improve borg by removing the need for msgpack (Issue 938) by using one of the alternatives proposed. Some of them actually sound like they might significantly improve borg's performance.

On the otherhand, if we don't improve borg rsnapshot is looking for a new maintainer if anyone has the time. I believe deduplication is one of the features proposed, but left with only partial support implemented, or only an implementation for one of many use cases. Has the advantage of being fast to browse.

Offline

Board footer

Powered by FluxBB