You are not logged in.

#1 2013-09-11 12:32:23

Kotrfa
Member
Registered: 2012-10-25
Posts: 213

[SOLVED]Copy.com storage service - optimal backup

Hello,

I just moved from Dropbox to Copy  (which by the way has linux app and this great promo - 20 GB for free) and I have enough storage to make the backup of my whole system (120 GB SSD).

As you probably know, it works on "drop folder" idea - you have one folder (called "Copy") and stuffs are synchronized if and only they are inside the Copy folder. For complete backup It's highly impractical - I can't sync all my .dotfiles in my home folder or better - my /etc (only "non-danger" files) folder.

I find a little workaround using sym links (ln -s src dst), but it still doesn't solve it at all, because my Copy folder is inside my home folder, so I still can't sync my whole home folder - it would be cycle...

My idea was to make Copy folder in "/" and sym link all directories I want to sync (/home, /etc, /opt, /var...), but it would mean that I have to run Copy app with root permissions and I'm not sure it is correct.

Thanks

Last edited by Kotrfa (2013-09-16 08:59:51)

Offline

#2 2013-09-11 15:22:54

WonderWoofy
Member
From: Los Gatos, CA
Registered: 2012-05-19
Posts: 8,414

Re: [SOLVED]Copy.com storage service - optimal backup

No you don't have to run it as UID 0.  Just put this "copy" directory wherever the heck you want it.  Then make sure that it is owned by you (and optionally your primary group) with chown, and make sure it is writeable for at least you.  If the permissions need to be changed, use chmod.

FWIW, I don't like to litter / with extraneous directories without a really good reason.  So I would probably put it somewhere else (like maybe /mnt/copy or /media/copy).

Offline

#3 2013-09-11 18:03:12

Kotrfa
Member
Registered: 2012-10-25
Posts: 213

Re: [SOLVED]Copy.com storage service - optimal backup

That's the problem - even when I change ownership etc by chmod/chown for /mnt (/mnt/Copy), I can't change Copy folder to this directory. CopyAgent (Copy X application) give me error: "Fail to move your Copy folder" without any explanation (even when I run it in terminal).

ls -l /mnt

drwxr-xr-x   2 dan  users  4096 Sep 11 19:34 mnt

ps aux|grep Copy

dan       1356  2.2  0.9 1475916 57644 ?       Ssl  19:32   0:40 CopyAgent

Offline

#4 2013-09-12 12:22:06

alexjj
Member
Registered: 2013-02-10
Posts: 11

Re: [SOLVED]Copy.com storage service - optimal backup

I want to run this headless...I installed it from the AUR and started the copy manually with CopyConsole -username=email -root=location which all worked fine. How do I set this up as a service? There's a systemctl file included which I tried using (with copy-agent@email.service) but nothing seems to happen. Any suggestions?

cat /usr/lib/systemd/system/copy-agent@.service
[Unit]
Description=CopyConsole
After=local-fs.target network.target

[Service]
Type=simple
ExecStart=/usr/bin/CopyConsole -daemon
ExecReload=/bin/kill -HUP $MAINPID
KillMode=process
Restart=always
User=%I

[Install]
WantedBy=multi-user.target

Offline

#5 2013-09-12 12:59:32

Kotrfa
Member
Registered: 2012-10-25
Posts: 213

Re: [SOLVED]Copy.com storage service - optimal backup

I think you should start your own thread... This haven't absolutely nothing to do with my question.

Last edited by Kotrfa (2013-09-12 12:59:59)

Offline

#6 2013-09-12 13:26:15

skanky
Member
From: WAIS
Registered: 2009-10-23
Posts: 1,847

Re: [SOLVED]Copy.com storage service - optimal backup

You might find that the sync software will detect a circular reference via linking to home and not enter the ~/Copy directory.
They have a help site where you can enter questions, so it might be worth posting the issue there.


"...one cannot be angry when one looks at a penguin."  - John Ruskin
"Life in general is a bit shit, and so too is the internet. And that's all there is." - scepticisle

Offline

#7 2013-09-12 15:37:26

kicka
Member
From: Sicily / Italy
Registered: 2012-10-18
Posts: 31

Re: [SOLVED]Copy.com storage service - optimal backup

my 2 cents here...

1 - You can install copy with the folder wherever you want, just remember that, during installation, you have to select the uplevel folder of the copy folder.

Example:

I want to install my copy folder here:

/media/Storage/Copy

then I have to select as folder, in the installation program, /media/Storage because if you put /media/Storage/Copy it will create another copy folder inside the last Copy folder as you can see below:

/media/Storage/Copy/Copy

The same tecnique can be used in other cloud service like dropbox, owncloud, etc... My copy folder is outside the home folder as well, and works flawlessy.

2 - Instead of symlink everything why don't you backup via tar everything (but the copy folder) and then put the single tar.gz file into the copy folder?

sudo tar -cvpzf [folder path address where you want to put the file] --exclude=/mnt  --exclude=/media  --exclude=/tmp  --exclude=/sys  --exclude=/proc  --exclude=/[Copy Folder path] --exclude=/[folder path address where you want to put the file]

example:

sudo tar -cvpzf /backup.tar.gz  --exclude=/mnt  --exclude=/media  --exclude=/tmp  --exclude=/sys  --exclude=/proc  --exclude=/Copy --exclude=/backup.tar.gz

to Extract:

sudo tar -xvpzf [folder path address where you want to put the file] -C /[folder path where you want to start untar it]

In our case example:

sudo tar -xvpzf /backup.tar.gz -C /

Offline

#8 2013-09-12 20:22:04

Kotrfa
Member
Registered: 2012-10-25
Posts: 213

Re: [SOLVED]Copy.com storage service - optimal backup

Ok, I finally moved my Copy folder to /mnt/Copy and it is working fine.

Thanks for advice about backups, but thats what I want Copy use for - incremental backups without making elephant files. It works great with simlinks for /boot, /etc, /opt, /usr and so on, but doesn't works with files/directories which aren't readable by user (of course). So my solution is:

- I have Copy folder at "/mnt/Copy"
- there I have sim links to my home directories, which I want sync and also "backup" directory
- inside backup directory I have sim links to "non-problem" system directories (/etc, /boot, /opt, /usr...)
- directories and files which Copy can't sync (because of permissions) I'm gonna tar and put inside this backup directory

Can you help me with script for find all files/directories, which I can't read and tar only them as root (in crontab I guess)?

I guess I'm looking for these files:

find / -user root -perm 700

then feed tar with these and maybe add some password protection using zip or ccrypt or something similar and save it to Copy directory.

These will save my time and storage. Does anyone has better solution? Are there any dangers from this? Thanks

Offline

#9 2013-09-13 01:56:10

cfr
Member
From: Cymru
Registered: 2011-11-27
Posts: 7,198

Re: [SOLVED]Copy.com storage service - optimal backup

I think we should step back a little.

What do you want your backup to achieve?

For example:
1) You want to be able to restore your system, as is, from the backup with a straightforward command or couple of commands i.e. without re-installing Arch or whatever.
2) You basically want to save user data and system config. In case of disaster, you're happy to reinstall Arch but you want a record of your system configuration and all of your user data.

Those are just 2 possibilities.

The reason I'm asking is because I can't quite see the point of the way you are setting it up - you seem to be backing up everything (which suggests option 1) but are doing it in a way which doesn't capture everything you need to know about everything (which suggests option 2 or something else). If, for example, you backup user-readable configuration files from /etc but they are all owned by your user, you don't have a straightforward way to restore those because the permissions and ownership etc. will be wrong.

Moreover, it is more complicated than looking for directories and files owned by root with permissions 700. For one thing, there are root-owned files with permissions such as 600. There are also all the files owned and readable by a user other than you but not owned by root. For example, /var/lib/mysql on my system is owned by mysql:mysql and has permissions 700.


CLI Paste | How To Ask Questions

Arch Linux | x86_64 | GPT | EFI boot | refind | stub loader | systemd | LVM2 on LUKS
Lenovo x270 | Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz | Intel Wireless 8265/8275 | US keyboard w/ Euro | 512G NVMe INTEL SSDPEKKF512G7L

Offline

#10 2013-09-13 10:41:34

Kotrfa
Member
Registered: 2012-10-25
Posts: 213

Re: [SOLVED]Copy.com storage service - optimal backup

Understand - It is very good point. Of course I would prefer the second option. The problem is that I don't want to do every day big package (cca 10 GB with script which I include lower), which take time and also it is imposible to upload it (in one day...)
Is there some software, which can make incremental backups and would be possible to restore my system in easy way?

By the way the script for backup looks like this:

tar --exclude=/mnt  --exclude=/run  --exclude=/tmp --exclude=/sys  --exclude=/proc --exclude=/usr/local/texlive --exclude=/usr/local/Wolfram --exclude=/home/dan/multimedia/Music --exclude=/home/dan/multimedia/Videos --exclude=/home/dan/Downloads --exclude=/home/dan/GitHub --exclude=/home/dan/backups --exclude=/var/cache  -cpzf  /mnt/Copy/backups/530uarch/backup$(date +%Y%m%d).tar.gz /*

Offline

#11 2013-09-13 12:16:20

progandy
Member
Registered: 2012-05-17
Posts: 5,322

Re: [SOLVED]Copy.com storage service - optimal backup

Kotrfa wrote:

Is there some software, which can make incremental backups and would be possible to restore my system in easy way?

There is rdiff-backup in community. You can also search for "incremental" in the AUR and find many applications.
rdiff-backup uses reverse diffs, so you always have a copy of the current state. If you want something older, you can get it with specifying parameters to rdiff-backup. You won't have a tar archive, but a directory that rdiff modifies when the backup is updated. exclude and include are supported.

Last edited by progandy (2013-09-13 12:23:12)


| alias CUTF='LANG=en_XX.UTF-8@POSIX ' | alias ENGLISH='LANG=C.UTF-8 ' |

Offline

#12 2013-09-13 13:04:27

Kotrfa
Member
Registered: 2012-10-25
Posts: 213

Re: [SOLVED]Copy.com storage service - optimal backup

Thank you for your answer and pation.

I don't want to make next copies of my files and then copy this copies to Copy (nice sentence btw big_smile ). I want to use Copy as "incremental backup". Where is problem with that? User permissions? Why cfr thinks it is impossible with sym links? Thanks

Offline

#13 2013-09-13 21:39:44

cfr
Member
From: Cymru
Registered: 2011-11-27
Posts: 7,198

Re: [SOLVED]Copy.com storage service - optimal backup

Kotrfa wrote:

I don't want to make next copies of my files and then copy this copies to Copy (nice sentence btw big_smile ).

Come again? Very confused...

I want to use Copy as "incremental backup". Where is problem with that? User permissions? Why cfr thinks it is impossible with sym links? Thanks

You won't be able to restore those files as they are because the ownership and permissions will be wrong. So long as the copying is done by your regular user, the copies will have the wrong ownership, permissions and ACLS and capabilities (where applicable). Except where you are copying files which you own, of course. So you couldn't just copy a config file back to /etc, for example. You'd need the rest of the information in order to restore it.


CLI Paste | How To Ask Questions

Arch Linux | x86_64 | GPT | EFI boot | refind | stub loader | systemd | LVM2 on LUKS
Lenovo x270 | Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz | Intel Wireless 8265/8275 | US keyboard w/ Euro | 512G NVMe INTEL SSDPEKKF512G7L

Offline

#14 2013-09-13 21:52:34

Kotrfa
Member
Registered: 2012-10-25
Posts: 213

Re: [SOLVED]Copy.com storage service - optimal backup

I think I get it.

I decided to use luckybackup on  my external HDD and Copy use just for the "little" things. Just one more question - what folders I can exclude from backup? I tried to google that, but sometimes there are different directories on different distros. I make complete backup of "/" instead of these:

/mnt
/run
/tmp
/sys
/proc
/var/cache
/var/tmp

Will I be able to restore whole system with this, or I can't exclude some of them? Can I exclude something else (something what is significant).

Last edited by Kotrfa (2013-09-13 21:54:14)

Offline

#15 2013-09-13 22:26:09

cfr
Member
From: Cymru
Registered: 2011-11-27
Posts: 7,198

Re: [SOLVED]Copy.com storage service - optimal backup

Here's the essence of what I use with rsync:

# Include
+ /dev
+ /media
+ /mnt/<networked-filsystem>
+ /mnt/generic
+ /mnt/generic/var
+ /mnt/<backup-usb-drive>
+ /etc/fstab\.*

# Exclude
- /dev/*
- /proc/*
- /run/*
- /sys/*
- /tmp/*
- lost+found/
- /media/*
- /mnt/<networked-filesystem>/*
- /mnt/generic/*
- /mnt/<backup-usb-drive>/*
- /etc/fstab
- /var/lib/pacman/sync/*
- /home/*/.gvfs/
- /home/*/wuala/WualaDrive/
- /home/*/.mozilla/firefox/*.default/Cache/*
- /home/*/.cache/chromium/*
- /boot/isos/
- /boot/efi/*.iso

Note that you probably want to include fstab - my reasons for excluding it are that I have a separate version for my external backup drive. gvfs and WualaDrive are excluded because they cause problems.

The top block essentially is to ensure that the directories and mount points are backed up even when their contents is excluded by the second block. I don't know how the application you're using works but this is the basics of my setup (which was originally based on the wiki page on using rsync for full system backup so you might look there, too).


CLI Paste | How To Ask Questions

Arch Linux | x86_64 | GPT | EFI boot | refind | stub loader | systemd | LVM2 on LUKS
Lenovo x270 | Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz | Intel Wireless 8265/8275 | US keyboard w/ Euro | 512G NVMe INTEL SSDPEKKF512G7L

Offline

#16 2013-09-15 14:23:08

Kotrfa
Member
Registered: 2012-10-25
Posts: 213

Re: [SOLVED]Copy.com storage service - optimal backup

Thanks for you help. I don't see a point why I should backup up mount points, when there is nothing mounted except the backup HDD. luckybackup is basicly frontend for rsync and diff. I'm getting these errors:

rsync: opendir "/dev/vboxusb" failed: Permission denied (13) 
rsync: opendir "/etc/NetworkManager/system-connections" failed: Permission denied (13) 
rsync: opendir "/etc/cups/ssl" failed: Permission denied (13) 
rsync: opendir "/etc/lvm/cache" failed: Permission denied (13) 
rsync: opendir "/etc/polkit-1/rules.d" failed: Permission denied (13) 
rsync: opendir "/etc/sudoers.d" failed: Permission denied (13) 
rsync: opendir "/opt/torchroot/var/lib/tor" failed: Permission denied (13) 
rsync: opendir "/root" failed: Permission denied (13) 
rsync: opendir "/usr/share/polkit-1/rules.d" failed: Permission denied (13) 
rsync: opendir "/var/db/sudo" failed: Permission denied (13) 
rsync: opendir "/var/lib/NetworkManager" failed: Permission denied (13) rsync: opendir "/var/lib/bluetooth" failed: Permission denied (13) rsync: opendir "/var/lib/mysql" failed: Permission denied (13) 
rsync: opendir "/var/lib/tor" failed: Permission denied (13) rsync: opendir "/var/lib/transmission" failed: Permission denied (13) 
rsync: opendir "/var/lib/udisks2" failed: Permission denied (13) 
rsync: opendir "/var/spool/cups" failed: Permission denied (13)

but backup is okay. Should I be concerned about this? Should I run backup as superuser?

Thanks

Offline

#17 2013-09-16 00:51:55

cfr
Member
From: Cymru
Registered: 2011-11-27
Posts: 7,198

Re: [SOLVED]Copy.com storage service - optimal backup

You need to run it with root privileges for a system backup.


CLI Paste | How To Ask Questions

Arch Linux | x86_64 | GPT | EFI boot | refind | stub loader | systemd | LVM2 on LUKS
Lenovo x270 | Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz | Intel Wireless 8265/8275 | US keyboard w/ Euro | 512G NVMe INTEL SSDPEKKF512G7L

Offline

#18 2013-09-16 08:59:38

Kotrfa
Member
Registered: 2012-10-25
Posts: 213

Re: [SOLVED]Copy.com storage service - optimal backup

Thank you. I think it is solved.

Offline

Board footer

Powered by FluxBB