[SOLVED] BTRFS: nodatacow on subvolume

nanohard · 2016-06-10 17:47:29

I am just now a part of the glorious Arch master race.

Question:
I have nodatacow on my mount options for several subvolumes, and I believe that option is being respected, but when I do lsattr I don't see "C"; just all dashes.
Is this expected behavior? I can't find any info on whether "C" should display if it's a subvolume mounted with options, rather than a file or directory that had chattr +C performed on it.
Other than seeing "C", I have no idea how to tell if CoW is on or off. Is there another way to tell?

Example (/vm is a subvolume where I want to keep my virtual machines):

$ lsattr -d /vm
---------------- /vm

Output of cat /proc/mounts (scroll to bottom, /dev/sda and /dev/sdb):

proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0
sys /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
dev /dev devtmpfs rw,nosuid,relatime,size=8135644k,nr_inodes=2033911,mode=755 0 0
run /run tmpfs rw,nosuid,nodev,relatime,mode=755 0 0
/dev/sda2 / btrfs rw,relatime,compress=lzo,ssd,space_cache,subvolid=258,subvol=/root 0 0
/dev/sda2 /usr btrfs rw,relatime,compress=lzo,ssd,space_cache,subvolid=259,subvol=/usr 0 0
securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0
tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0
devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0
tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0
cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0
pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0
efivarfs /sys/firmware/efi/efivars efivarfs rw,nosuid,nodev,noexec,relatime 0 0
cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0
cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0
cgroup /sys/fs/cgroup/net_cls cgroup rw,nosuid,nodev,noexec,relatime,net_cls 0 0
cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0
cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0
cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0
cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0
cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0
systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=25,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0
hugetlbfs /dev/hugepages hugetlbfs rw,relatime 0 0
mqueue /dev/mqueue mqueue rw,relatime 0 0
configfs /sys/kernel/config configfs rw,relatime 0 0
debugfs /sys/kernel/debug debugfs rw,relatime 0 0
/dev/sda1 /boot vfat rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,errors=remount-ro 0 0
/dev/sda2 /opt btrfs rw,relatime,compress=lzo,ssd,space_cache,subvolid=260,subvol=/opt 0 0
/dev/sda2 /code btrfs rw,relatime,compress=lzo,ssd,space_cache,subvolid=261,subvol=/code 0 0
/dev/sdb1 /home btrfs rw,nodev,relatime,nodatasum,nodatacow,space_cache,subvolid=259,subvol=/home 0 0
/dev/sdb1 /tmp btrfs rw,nodev,relatime,nodatasum,nodatacow,space_cache,subvolid=258,subvol=/tmp 0 0
/dev/sdb1 /var btrfs rw,nodev,relatime,nodatasum,nodatacow,space_cache,subvolid=257,subvol=/var 0 0
/dev/sdb1 /vm btrfs rw,nodev,relatime,nodatasum,nodatacow,space_cache,subvolid=260,subvol=/vm 0 0
tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=1627828k,mode=700,uid=1000,gid=100 0 0

Last edited by nanohard (2016-06-11 06:24:24)

Daerandin · 2016-06-10 23:17:58

Btrfs do not support mounting subvolumes from the same partition with different settings regarding COW. If you first mount one subvolume from the partition with COW enabled (the default), then all the other subvolumes on the same partition will also be mounted with COW enabled even if you specified nodatacow in fstab.

If you want everything on a particular subvolume to have COW disabled, but have other subvolumes on the same partition with COW enabled, then I suggest that you take a slightly different approach. First create the subvolume, then use chattr to set +C for that subvolume before creating/copying any files or directories there. Using chattr +C on existing files that are not empty has undefined behaviour, so make sure to set this attribute as soon as the subvolume is created.

And in case you are wondering: Mounting with nodatacow will not change the file attributes. So even if you mount successfully with nodatacow, lsattr will still not show C for any files on that subvolume. When a subvolume is successfully mounted with nodatacow, this will also show up in the output of the mount command.

Slithery · 2016-06-10 23:27:36

@Daerandin

I was going to post the same reply earlier, but looking at the mount output all of the subvolumes on sdb1 are mounted with the nodatacow option.
I'm interested in knowing the answer to this question as well.

Daerandin · 2016-06-11 00:15:37

slithery wrote:

@Daerandin
I was going to post the same reply earlier, but looking at the mount output all of the subvolumes on sdb1 are mounted with the nodatacow option.
I'm interested in knowing the answer to this question as well.

You are correct, I didn't see that. So according to that output, COW is disabled for all subvolumes on /dev/sdb1. I suspect nodatacow is a mount option for the /home subvolume. This ensures all other subvolumes on the same partition will be mounted with nodatacow even if they don't specify it. Seeing OP's fstab would be helpful.

I do want to point out one thing from the btrfs wiki:

btrfs wiki wrote:

nodatacow
Do not copy-on-write data for newly created files, existing files are unaffected. This also turns off checksumming! IOW, nodatacow implies nodatasum. datacow is used to ensure the user either has access to the old version of a file, or to the newer version of the file. datacow makes sure we never have partially updated files written to disk. nodatacow gives slight performance boost by directly overwriting data (like ext[234]), at the expense of potentially getting partially updated files on system failures. Performance gain is usually < 5% unless the workload is random writes to large database files, where the difference can become very large. NOTE: switches off compression !

So keep in mind that nodatacow will only affect newly created files. If the files were copied over to the subvolume while COW was enabled, then it will remain enabled for those files. I personally prefer to use chattr +C on subvolumes before I start putting files there, since I usually have several subvolumes on the same partition, some of which I want COW enabled.

nanohard · 2016-06-11 06:22:02

Thank you for your replies; marking solved.
As requested, here is my fstab. Not sure why "subvol" is there twice, I think this was generated by mkinitcpio and then I added /boot to it.
Folders I copied from a backup into /home didn't have +C, but newly created folders did. Looks like I won't be copying over any VMs and will be creating them from scratch. Hmm... but it looks like the files I copied from my backup inside those folders actually do have +C. I think my confusion is that the BTRFS info is lacking in this detail (on the internet in general, not just the Arch wiki). I did research this for hours without finding the answer in plain english.

sda = SSD
sda1 = /boot (fat32)
sda2 = /root, /usr, /opt, /code (btrfs, CoW on all subvolumes)

sdb = HDD
sdb1 = /home, /var, /tmp, /vm (btrfs, nodatacow on all subvolumes)

fstab:

# 
# /etc/fstab: static file system information
#
# <file system>	<dir>	<type>	<options>	<dump>	<pass>
# /dev/sda2 LABEL=ssd
UUID=bf08f98e-b45f-4827-bc4c-0f0c8ce32453	/         	btrfs     	rw,relatime,compress=lzo,ssd,space_cache,subvolid=258,subvol=/root,subvol=root	0 0

# /dev/sda1 LABEL=boot
UUID=FACA-B46D					/boot		vfat		rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,errors=remount-ro	0 0

# /dev/sda2 LABEL=ssd
UUID=bf08f98e-b45f-4827-bc4c-0f0c8ce32453	/usr      	btrfs     	rw,relatime,compress=lzo,ssd,space_cache,subvolid=259,subvol=/usr,subvol=usr	0 0

# /dev/sda2 LABEL=ssd
UUID=bf08f98e-b45f-4827-bc4c-0f0c8ce32453	/opt      	btrfs     	rw,relatime,compress=lzo,ssd,space_cache,subvolid=260,subvol=/opt,subvol=opt	0 0

# /dev/sda2 LABEL=ssd
UUID=bf08f98e-b45f-4827-bc4c-0f0c8ce32453	/code     	btrfs     	rw,relatime,compress=lzo,ssd,space_cache,subvolid=261,subvol=/code,subvol=code	0 0

# /dev/sdb1 LABEL=hdd
UUID=d799b9a8-8fd8-40e5-80a7-1821ad7c2bbe	/home     	btrfs     	rw,nodev,relatime,nodatasum,nodatacow,space_cache,subvolid=259,subvol=/home,subvol=home	0 0

# /dev/sdb1 LABEL=hdd
UUID=d799b9a8-8fd8-40e5-80a7-1821ad7c2bbe	/var      	btrfs     	rw,nodev,relatime,nodatasum,nodatacow,space_cache,subvolid=257,subvol=/var,subvol=var	0 0

# /dev/sdb1 LABEL=hdd
UUID=d799b9a8-8fd8-40e5-80a7-1821ad7c2bbe	/tmp      	btrfs     	rw,nodev,relatime,nodatasum,nodatacow,space_cache,subvolid=258,subvol=/tmp,subvol=tmp	0 0

# /dev/sdb1 LABEL=hdd
UUID=d799b9a8-8fd8-40e5-80a7-1821ad7c2bbe	/vm       	btrfs     	rw,nodev,relatime,nodatasum,nodatacow,space_cache,subvolid=260,subvol=/vm,subvol=vm	0 0

Last edited by nanohard (2016-06-11 06:33:24)

Daerandin · 2016-06-11 08:06:04

Since you have nodatacow for all sdb1 subvolumes, then it will work as intended. Keep in mind that a copy of a file is a newly created file. If you take a backup of all subvolumes on sdb1, then delete everything in those subvolumes, then copy everything back from your backup, then you will have COW disabled for all files in those subvolumes as everything will be newly created files.

Reading the btrfs wiki should be useful for you: https://btrfs.wiki.kernel.org/index.php/Main_Page

Be sure to check out the Guides and information section there, most of the articles linked to from there will be useful.

Soukyuu · 2016-06-11 09:52:01

Just curious, why do you have the 'nodatacow' option active? It disables most of the useful stuff btrfs offers (checksumming, snapshots not taking much space). If you are worried about performance of virtual machines etc, it's better to enable nodatacow for specific directories using chattr +C

nanohard · 2016-06-11 14:28:09

Daerandin wrote:

Keep in mind that a copy of a file is a newly created file.

Yeah, I sort of realized/considered this last night, thinking through how rsync or cp actually works in a theoretic manner, that a new file would have to be "touched" and then filled.
Thanks for the sweet link. I've probably read that one, and maybe one or two links from that, but I'll be reading it again I'm sure.

Soukyuu wrote:

Just curious, why do you have the 'nodatacow' option active? If you are worried about performance...

Indeed, using this new filesystem that I've never worked with before has me a bit nervous. This is my first time with a solid state drive, and my first time with BTRFS.

Would assigning -C to folders on /home (mounted with nodatacow) achieve something, or would it be ignored, or would it crash my system and start a loop of self-learning that would then burn the world to ashes?

I have to say, I'm very impressed with the Arch wiki and community. Thank you!

Last edited by nanohard (2016-06-11 14:41:12)

Soukyuu · 2016-06-11 16:15:02

From what I know, if you have the subvolume mounted with nodatacow, the +C attribute is not getting checked. So doing chattr -C wouldn't do anything as the flag isn't set.

I use btrfs with compression=lzo on my / and so far there were no adverse effects. Personally, if you disable COW, you might as well just use ext4.

Last edited by Soukyuu (2016-06-11 16:23:49)

Daerandin · 2016-06-11 17:28:16

I also agree that COW is a very important feature of btrfs and one of the reasons why I use it. I personally have two subvolumes for which I have manually set +C to ensure all newly created files in the subvolume do not use COW. These subvolumes are mounted on my SQL datadir and the directory for virtual machines. I do not use the nodatacow for those subvolumes, I have just set the actual subvolume to have the C attribute. This is strictly not required as you can simply set an empty directory to have the C attribute and it would apply it to all files created in the directory, but I prefer to keep it on separate subvolumes.

Arch Linux

#1 2016-06-10 17:47:29

[SOLVED] BTRFS: nodatacow on subvolume

#2 2016-06-10 23:17:58

Re: [SOLVED] BTRFS: nodatacow on subvolume

#3 2016-06-10 23:27:36

Re: [SOLVED] BTRFS: nodatacow on subvolume

#4 2016-06-11 00:15:37

Re: [SOLVED] BTRFS: nodatacow on subvolume

#5 2016-06-11 06:22:02

Re: [SOLVED] BTRFS: nodatacow on subvolume

#6 2016-06-11 08:06:04

Re: [SOLVED] BTRFS: nodatacow on subvolume

#7 2016-06-11 09:52:01

Re: [SOLVED] BTRFS: nodatacow on subvolume

#8 2016-06-11 14:28:09

Re: [SOLVED] BTRFS: nodatacow on subvolume

#9 2016-06-11 16:15:02

Re: [SOLVED] BTRFS: nodatacow on subvolume

#10 2016-06-11 17:28:16

Re: [SOLVED] BTRFS: nodatacow on subvolume

Board footer