You are not logged in.

#1 2021-12-07 17:29:42

Cvlc
Member
Registered: 2020-03-26
Posts: 248

[SOLVED] Rsync as cp alternative, Btrfs CoW

Hi

is the following tip from Rsync#As cp/mv alternative valid for a btrfs system with CoW ?

To use sane defaults quickly, you could use some aliases:

function cpr() {
  rsync --archive -hh --partial --info=stats1,progress2 --modify-window=1 "$@"
} 
function mvr() {
  rsync --archive -hh --partial --info=stats1,progress2 --modify-window=1 --remove-source-files "$@"
}

i just tried it to move a video file and I was surprised that it took 3s, when it's instantaneous with cp -a

[edit] Well actually I tried the mvr function, so not sure this has anything to do with CoW.... but still surprised that it took 3s, doesn't feel like it's doing it right.

Last edited by Cvlc (2021-12-17 03:30:45)

Offline

#2 2021-12-07 17:40:31

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 27,538
Website

Re: [SOLVED] Rsync as cp alternative, Btrfs CoW

I have no btrfs experience, but if this move was within the same filesystem, then cp should be notably faster than rsync for the reasons clearly outlined in the section of the wiki you link to.  Why would it surprise you that rsync took longer?


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#3 2021-12-07 20:10:00

progandy
Member
Registered: 2012-05-17
Posts: 5,047

Re: [SOLVED] Rsync as cp alternative, Btrfs CoW

As of coreutils 9.0, cp does try to automatically use reflinks if the filesystem supports that. rsync does not support it: https://github.com/WayneD/rsync/issues/153
For mv it is similar, on the same filesystem it is possible to just move the inode reference into the new directory without having to move the file contents. It may be possible to use --link-dest with rsync to use hardlinks to simulate that behaviour, but I am not sure whether the combination --link-dest with --remove-source-files is supported.

Last edited by progandy (2021-12-07 20:11:10)


| alias CUTF='LANG=en_XX.UTF-8@POSIX ' |

Offline

#4 2021-12-09 16:44:41

Cvlc
Member
Registered: 2020-03-26
Posts: 248

Re: [SOLVED] Rsync as cp alternative, Btrfs CoW

OK thanks. I'll look deeper into it but it does confirm that rsync is not a viable alternative to cp / mv on a btrfs system

Offline

#5 2021-12-09 18:19:49

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 27,538
Website

Re: [SOLVED] Rsync as cp alternative, Btrfs CoW

This should have nothing to do with btrfs.  Using rsync in place of cp or mv to copy/move content within the same filesystem would be horrible on *any* filesystem.  That's not what rsync is for - it's the wrong tool for the job.


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#6 2021-12-14 16:47:57

Cvlc
Member
Registered: 2020-03-26
Posts: 248

Re: [SOLVED] Rsync as cp alternative, Btrfs CoW

Should the wiki be corrected then ?

Rsync#As cp/mv alternative

Offline

#7 2021-12-14 16:54:35

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 27,538
Website

Re: [SOLVED] Rsync as cp alternative, Btrfs CoW

I don't see why.  That's a page all about rsync which is introduced from the start as a tool primarily used for transferring files between different machines (so different filesystems is implied) not advice for btrfs systems.  While rsync can be used within a filesystem, and there may even be very good use cases for doing so, that's not the primary role of the tool.  This is spelled out explicitly in that very section:

wiki wrote:

Files can be copied locally as with cp, but the motivating purpose of rsync is to copy files remotely, i.e. between two different hosts.

The 'r' in rysnc as well as the r in the cpr and mvr stand for remote.  Taking tools intended for use between remote systems but using them within a filesystem and noting poor performance is a bit silly, and does not mean those tools don't have value.  Just because hammers suck at driving screws doesn't mean the hardware store should stop selling hammers.

Last edited by Trilby (2021-12-14 16:57:52)


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#8 2021-12-16 02:51:20

Cvlc
Member
Registered: 2020-03-26
Posts: 248

Re: [SOLVED] Rsync as cp alternative, Btrfs CoW

Well, I respectfully disagree :

The Core Utilities article has a cp_alternatives section which only says :

Using rsync#As cp/mv alternative allows you to resume a failed transfer, to show the transfer status, to skip already existing files and to make sure of the destination files integrity using checksums.

The link brings you to the rsync section which starts with

rsync can be used as an advanced alternative for the cp or mv command, especially for copying larger files:

$ rsync -P source destination

The part that you quoted says that it can be used locally, and does not mention at all that it's a bad idea to do so...

Like you said this has nothing to do with Btrfs, I use Rsync for backups and I know that it is essentially for remote systems, but when I read this I thought "oh yeah great idea".

It turns out not to be one which is why I think it's misleading. This thread wouldn't exist otherwise, and if I misunderstood others will.

The hardware store shouldn't stop selling hammers, but it shouldn't store the screwdrivers next to the nails and the hammers next to the screws....

thanks for your help

Last edited by Cvlc (2021-12-16 03:08:12)

Offline

#9 2021-12-16 04:12:41

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 27,538
Website

Re: [SOLVED] Rsync as cp alternative, Btrfs CoW

Ok, so you see a different page as misleading.  In the case of the prior-existing content on the core utilities page, I'd completely agree.  That was bad advice there and I see you've flagged it as disputed - though it's not just useful for the reasons stated for remote media, but anytime something is being moved between different filesystems.  Resuming a within-filesystem move is a non-issue as such a move should be effectively instantaneous regardless of the size of the file - for a copy, on a non-COW/ROW filesystem there is at least some theoretical possibility of getting the benefits mentioned in that section of the core utilities wiki, but it still seems trivial and silly to me.  But for a move it's completely nonsensical which is likely why a wiki admin moved it from a cp/mv alternative to just a cp alternative.  That said, there is a bunch of nonsense on that page, which is kind of sad for a page for such an important package.

Last edited by Trilby (2021-12-16 04:31:13)


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#10 2021-12-17 03:29:47

Cvlc
Member
Registered: 2020-03-26
Posts: 248

Re: [SOLVED] Rsync as cp alternative, Btrfs CoW

Thanks for the very interesting answers, I've updated the flag.

Have a good day !

Last edited by Cvlc (2021-12-17 03:30:24)

Offline

Board footer

Powered by FluxBB