You are not logged in.

#1 2013-08-06 09:19:16

geekinthesticks
Member
From: England
Registered: 2011-09-21
Posts: 143
Website

Some Questions About btrfs and RAID.

I have read through the wiki and am not sure if what I want to do is possible/sensible. I am going to rebuild my MythTV backend. At the moment the media are stored on RAID1 (mirrored) drives and  I have two sets of mirrors. What I would like to do is to have a situation where the drives are still mirrored, but the OS sees the two separate mirrored sets as one continuous drive. If one of the drives in a mirrored set fails, I don't want to use any data.

It looks as though btrfs with subvolumes may do what I want. Can anyone confirm this or offer a better alternative suggestion? Also the last time I looked at btrfs there were no file system recovery tools like e2fsck. Is this still the case?

Offline

#2 2013-08-06 09:56:37

graysky
Wiki Maintainer
From: :wq
Registered: 2008-12-01
Posts: 10,595
Website

Re: Some Questions About btrfs and RAID.

If this is your data drive, I dunno how btrfs handles large mpeg2 streams.  It is not recommended to use btrfs' CoW features for large files that change often like VM disk images... I am not saying it is bad, just that I would find out before converting an entire filesystem over.


CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#3 2013-08-06 11:55:43

arokh
Member
Registered: 2013-05-30
Posts: 35

Re: Some Questions About btrfs and RAID.

What you are describing is called RAID 10 (striped mirror), and requires 4 drives. It's supported directly by BTRFS or by using mdadm.

Here's a description of subvolumes: https://btrfs.wiki.kernel.org/index.php … Subvolumes

Subvolumes is a BTRFS feature that basically lets you have different filesystem roots inside the same volume. You can then mount different subvolumes with mount options, do snapshots of specific subvolumes etc. It's got nothing to do with joining devices.

BTRFS should handle large media just fine. It's true that COW should be disabled on large database files that receive small updates. This is because a small write will cause BTRFS to copy the entire file to new location before issuing the update to ensure data consistency, causing lots of I/O. Not true for large media files of course because they do not change.

Last edited by arokh (2013-08-06 11:56:51)

Offline

#4 2013-08-08 07:57:08

teekay
Member
Registered: 2011-10-26
Posts: 271

Re: Some Questions About btrfs and RAID.

graysky wrote:

If this is your data drive, I dunno how btrfs handles large mpeg2 streams.  It is not recommended to use btrfs' CoW features for large files that change often like VM disk images... I am not saying it is bad, just that I would find out before converting an entire filesystem over.

It's possible to disable CoW on a per file or per directory basis.

Say you have a folder where you record streams into (i.e. file keeps changing over hours, while you have a job that does a snapshot every hour => bad waste of space here)

chattr +C /path/to/records/folder

will disable CoW for newly created files in that folder.

Offline

#5 2013-08-08 08:39:07

graysky
Wiki Maintainer
From: :wq
Registered: 2008-12-01
Posts: 10,595
Website

Re: Some Questions About btrfs and RAID.

@tee - Right but if he will have to do this for every new recording that his system captures.  Not practical.


CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#6 2013-08-08 08:41:47

geekinthesticks
Member
From: England
Registered: 2011-09-21
Posts: 143
Website

Re: Some Questions About btrfs and RAID.

Thanks for all the tips. I'll implement RAID10. Being able to disable COW on a per directory basis is helpful. The mysql database gets many updates per minute during recordings.

Offline

#7 2013-08-08 08:51:50

teekay
Member
Registered: 2011-10-26
Posts: 271

Re: Some Questions About btrfs and RAID.

graysky wrote:

@tee - Right but if he will have to do this for every new recording that his system captures.  Not practical.

Not if you set +C on the directory in which the recordings get stored.

man chattr wrote:

A  file with the 'C' attribute set will not be subject to copy-on-write updates.  This flag is only supported on file systems which perform copy-on-write.  (Note:
For btrfs, the 'C' flag should be set on new or empty files.  If it is set on a file which already has data blocks, it is undefined when the  blocks  assigned  to
the  file  will be fully stable.  If the 'C' flag is set on a directory, it will have no effect on the directory, but new files created in that directory will the
No_COW attribute.)

That also works for newly created subdirectories. A test:

└» mkdir test
└» chattr +C test
└» lsattr|grep test
---------------C ./test
└» mkdir test/foo test/bar
└» lsattr test
---------------C test/foo
---------------C test/bar
└» touch test/foo/file1.txt
└» lsattr test/foo/
---------------C test/foo/file1.txt

I find that very practical actually. I use it for KVM images, postgres data files, recordings and other things.

Last edited by teekay (2013-08-08 09:00:18)

Offline

#8 2013-08-08 09:36:18

graysky
Wiki Maintainer
From: :wq
Registered: 2008-12-01
Posts: 10,595
Website

Re: Some Questions About btrfs and RAID.

@tee - I stand corrected.  Makes sense actually that the setting would work at the dir level.


CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

Board footer

Powered by FluxBB