ZFS for backups with low memory, Is it safe?? Is there an alternative?

hydrosIII · 2014-05-13 01:59:11

Hi. Im about to setup a backup server using and old pc. I will use 64 bit pc with 2 GB of RAM and a dual core CPU of 1 GHZ.
The purpose of this backup pc is to do some backups from another server, mostly using utilities like rsync. I will be backing up arooung 100GB of data.
Im using arch linux for this. ( maybe another linux distro but definitely linux and not FreeBSD or Opensolaris)

The plan is to automatically turn on the backup server for a couple of hours during the night do the backup job and just turn off.
This is really a simple backup task, but i figured i could take advantage of the ZFS filesystem. I will use two hard disk of 1 Tb assembled a RAiD 1 array or its equivalent in ZFS. I was almost about to do this when checking the documentataion of ZFS i just ended up in the FreeNas Website. FreeNAS is a BSD distro focused on NAS. There i read that it was just suported to use the minimum of 8GB of RAM to run ZFS. I just couldnt believe it, but in every part of the site says : "If you don't have 8G of RAM then you could lose data."
What???I mean, 8G, just to run a filesystem??? Researching further the FreeBsd website recommends 1G (http://www.freebsd.org/doc/handbook/fil … s-zfs.html) as the minimum to use ZFS. The arch wiki doesnt says anything about minimum RAM but says the amount of RAM can be tunned. Is it really safe to run a backup server with the ZFS filesystem without the full 8G of RAM? FreeNAS website also recommends a special type of RAM that is less propense to corruption. The Free NAS website cites a numerous numer of cases of people loosing z-pools because of low memory. Buying more RAM is just not possible at the moment. The actual server ( the one im backing up) tuns on 2G of memory and jus uses 50% of it, so it wouldn'nt make sense to use a server of 8G just to backup the data in it.

Has anyone experience in this subject?? Is ZFS really realiable both the fuse and the kernel module versions??
I really wanty the backup machine to be realiable, and i dont want to be watching the backups or checking for possible memory corruption. Shoud i use ZFS or maybe a more conservative solution like ext4 + backup utility.?

Im mostly looking at zfs because of the snapshot feature.

I will appreciate some feedback. Thank you.

WonderWoofy · 2014-05-13 02:30:10

I believe that the insane RAM requirements is a result of ZFS's automagic deduplication. My experience with ZFS is minimal, but I have also read that if you don't have much data (100G isn't so very much) that it won't take nearly that much for it to run properly.

I think the biggest point of failure for such a setup wouldn't be your lack of RAM, but rather the fact that you would be relying on an out of tree kernel module for something as crucial as the running filesystem. The last time I tried, I ended up with circular dependencies on updates between the spl stuff and the zfs stuff. Maybe using devtools or the unofficial repo that demizerone maintains might solve that though.

Still it would probably be wiser to use a native module/filesystem. Snapshots are really cool, and you could use btrfs, but that too might also be asking for trouble since it is not considered entirely stable still. But in that regard... ZoL isn't necessarily time tested either.

hydrosIII · 2014-05-13 04:08:01

Mmm so you recommend using btrfs to go native?. I think that the zfs-lts package resolves the issue of relying in the zfs kernel module. I have read about btrfs and there are a lot of warning signs around posts and information over the internet saying it is not production ready.

I could use FreeBSD or opensolaris as a backup NAS, i thought about that to go native, but the thing the server is not just a backup server, It is supposed to be a replacement of the original server. I will copy the services and configurations of the original server to this backup server. So it will have the services prepared in one partition and other partitions would be just dedicated to backups. The services are going to be turned off by default. I mostly run a samba server, sftp server, a little web server and an openerp server. All of this services are isolated in easy to transport linux containers (that are with no data, just the running service). So in case the original server goes down i could just turn on the linux containers populate them with the new data, and restore the entire system. Thus minimizing the data to be backed up and the downtime. So that is why i cannot use another OS it must be linux.

Also there would be two partitions at least, one with the system formatted in ext4 and the other in zfs, i have no idea if this is possible with just two disks that are going to be in RAID 1. If not i will just use a usb stick to boot the machine and store everything in the hard disk.
That would prevent that an upgrade or a failure of the zfs-modulre prevents the system to boot.

Any other thoughts about memory??

WonderWoofy · 2014-05-13 04:28:20

hydrosIII wrote:

Mmm so you recommend using btrfs to go native?. I think that the zfs-lts package resolves the issue of relying in the zfs kernel module. I have read about btrfs and there are a lot of warning signs around posts and information over the internet saying it is not production ready.

The thing is, the ZoL project is still in its infancy. Although they make big claims as being stable, it took ZFS years to become stable on Solaris. I don't know that I would recommend btrfs either for a server containing backups, but for my personal laptop it has proven quite stable for me.

I don't think that using the zfs-lts package will keep you from having to rebuild the modules either. From time to time a given kernel release has what is called a binary module breaker. When this happens, you will have to rebuild all third party modules. You can see when this happens when the kernel updates are released to [testing] along with things like r8168, virtualbox modules, etc to [community-testing].

graysky · 2014-05-13 10:18:53

I have an atom based box with 2 gigs of memory that has a 2x 1 TB drives in mirror mode just like you're planning that runs with no issues. I scrub it one a month and never found corruption... Knock on wood. I have the zpool about 40 % full. For system updates, I recommend using demz's repo for the spl and zfs utils. He keeps it synced up with Arch and it is easy to use.

EDIT: you mentioned two disks in raid1 or zfs equivalent... To my knowledge, 3 drives are the min requirement for raidz1. 2 drives can be used in mirror mode.

Last edited by graysky (2014-05-13 10:28:29)

hydrosIII · 2014-05-22 06:05:28

Sorry for the late reply i have been a litlle busy. during these time i took the risk and put zfs to work in the backup server. To be fair i just disassembled another old machine and got 2G extra of RAM so now the server has 4G in total .The true is that zfs eats all the memory leving roughly 100 mb free, But i think is just keeps the memory because it seems used even when not doing anything in the filesystem. Also i can run other processes and the memory use remains the same. Seems that zfs is just retaining the free memory but gives other processes the memory when requested.

I dont know if its stable i have being doing some tests but so far RAIDZ2 seems great. Also researching a little more of backups it seems that rsync can use hard links to create snapshots also.Interesting. I have found this web page that uses that technique to create the same thing i'm willing to do without ZFS.

http://anouar.adlani.com/2011/12/how-to … n-osx.html

So i asked myself.Will that be a better solution ?

Im relying on zfs to take snapshots and also use compression ( i have set zfs to use gzip)

So which solutions will be the most reliable and KISS. ZFS dependable backup or ext4 + RAID + rsync script + tar ?

chaonaut · 2014-05-22 07:56:56

hydrosIII wrote:

Mmm so you recommend using btrfs to go native?. I think that the zfs-lts package resolves the issue of relying in the zfs kernel module. I have read about btrfs and there are a lot of warning signs around posts and information over the internet saying it is not production ready.

btrfs is a way to go.
i'm using it on my build / backup server for last 2 years (either on mdadm raid and btrfs raid).
it works. and since 3.12, it works really ok. (excluding database & virtual machine images storage, but i use ext4 or xfs for that.)
i've experimented with zfs on linux recently. what i can say confidently — too slooooooooooow (anyone can see it without any benchmarks, just rsync'ing a few gigabytes to zfs partition). btrfs is a lot faster, especially in last kernels (starting from 3.12).
in fact, i'm still absolutely fascinated with all that fancy-schmancy zfs features. but its actual speed on linux broke my heart.

Last edited by chaonaut (2014-05-22 08:03:58)

hydrosIII · 2014-05-22 08:20:57

I have not benchmarked the zfs installation. But im staying away from btrfs it may be as you say but in the btrfs wiki there a lot of warnings signs saying , "Data corruption!!!!" . Also im using now raidz2 with 4 disks, the btrfs wiki clearly says that btrfs with raid 6 is still experimental.

Answering my previous question about zfs vs normal backup solutions, it seems that zfs (or similar ) is the way to go. I have investigated using rdiff-backup but it seems to be risk of corruption of the process is interrupted. Also the hard link approach in rsync seems to be not quite flexible. see: http://www.jcea.es/artic/backup02.htm (im sorry, it is in spanish)

Reading a little further i came across nilfs, i have never heard of at this filesystem and in fact there are only a few references in the internet (it doesn't have a page the arch wiki !!) but it seems to be stable and production ready, and it seems like a predecessor of btrfs, it is available in the mainline kernel and it supports snapshots.

Taking a look at the homepage ( http://nilfs.sourceforge.net/) seems like the project is alive and actively mantained, maybe it is not a mainstream filesystem but it is included in debian and in arch. Pretty interesting, it solves the memory problem and the snapshot problem, the only flaw is that i cannot find good docs about it, maybe the manual pages will do it . Im gonna give it a try before making the full compromise with zfs, it deserves it since it is in the mainline kernel and it doesn't have warning signs all around it.

Any thoughts?

hydrosIII · 2014-05-22 08:24:53

chaonaut wrote:

btrfs is a way to go.
i'm using it on my build / backup server for last 2 years (either on mdadm raid and btrfs raid).
it works. and since 3.12, it works really ok. (excluding database & virtual machine images storage, but i use ext4 or xfs for that.)

Just out of curiosity, can you use BTRFS in top of an md raid device?, for example in a RAID 6 level. Because that would solve the problem of the instability of the BRTFS 5 or 6 integrated RAID. Am i right??

chaonaut · 2014-05-22 08:31:09

hydrosIII wrote:

Just out of curiosity, can you use BTRFS in top of an md raid device?, for example in a RAID 6 level. Because that would solve the problem of the instability of the BRTFS 5 or 6 integrated RAID. Am i right??

sure, i can. i'm actually using it over md raid level 1 on my root partition, but it can be used over any md raid level, same as any other filesystem.
using or not using btrfs raid features is absolutely at your option.

theking2 · 2014-05-27 18:55:30

ZFS is a no brainer to install (compared to a md+lvm+xfs, I used to have) .

My experience is of course only a week or so old. but I must say I'm impressed with ZFS speed and trouble shooting. For over 10 years now I've deployed a NAS with XFS and now switched to ZFS with it first big task to survive restoring 1.5T from various external USB harddisks that I gatherd to keep all my films, TV shows and music and backups of the various laptops and PCs. It worked fine despite a USB cable problem cause numerous kernel panics. ZFS seemed unimpressed, just merely pointed out that some files contained errors, which were easily replaced. After scrubbing it seems that every thing is fine.

This runs on a 2G RAM/Atom board. not sure if 2G is regarded as low memory.
I still think that having a seperate device for / and /boot and store the NAS bulk on a raid[5|z1] set has made live so much easier.

I don't think there is an alternative :-)

Last edited by theking2 (2014-05-27 19:11:11)

seiichiro0185 · 2014-05-28 07:23:27

I also use ZFS on my NAS since quite some time, and it runs stable and without a problem even surviving several kernel panics / freezes of the machine due to other issues. I use the in-kernel variant from the already mentioned demz-repo. I can't complain about the speed either, getting about 80MB/s from a RAIDZ1 on 4 luks-encrypted WD Green drives internal and about 50MB/s over a 1GBit Ethernet Link.

The current system is a Pentium G CPU with 8GB of RAM, but it was running fine (although a little slower because of the crypto) on a AMD E350 with 2GB of RAM. What really uses a lot of RAM and decreases Performance quite a bit is the internal deduplication, especially for large datasets. But this feature is turned off by default.

This setup runs flawless for > 1 year, I do an automatic scrub twice a month and have not seen any corruption so far. Like theking2 I run the root system from a separate device (usb 3.0 stick) with ext4 as filesystem.

hydrosIII · 2014-06-11 22:21:13

Hi, im sticking with ZFS, reviewing my configuration i just figured out that ZFS was only using 500 mb of ram in my setup. I just got confused because of the kernel memory allcation: see more here:http://www.linuxatemyram.com/. I thinks requirements of zfs depends on the amount of data. Also im not using deduplication. So i think it is safe to use with relatively low ram, The guys at FreeNas are just being extra careful, or speaking with more conservative standards or working with large datasets. I think than 1 GB is good to run ZFS for a home server or a small server.

Thanks to all the persons that posted.

Marcher · 2014-06-14 03:30:24

ZFS uses your system memory as a cache (faster to read from here than a spinny disk or even an SSD), and it will use as much memory as possible for this cache (tunable though). I am running 4 gig on my machine, the more the better. Don't worry about dedup, the savings in space is not worth it for the memory requirements. Compression is great however. Next time you setup a ZFS machine try to get a system that supports ECC memory, this will help to strengthen your data resiliency with one less point of failure. Periodic snapshots should also help to prevent corrupted files (though this should never happen in a ZFS system).

hydrosIII · 2014-06-21 07:11:09

ECC memory? i have heard about it, not really sure how to distinguish it. Maybe i have it . I just bought and old rack server. Any way to figure it out from command line without disassembling the server

ukhippo · 2014-06-21 10:15:29

hydrosIII wrote:

ECC memory? i have heard about it, not really sure how to distinguish it. Maybe i have it . I just bought and old rack server. Any way to figure it out from command line without disassembling the server

dmidecode -t memory | grep 'Error Correct'

Arch Linux

#1 2014-05-13 01:59:11

ZFS for backups with low memory, Is it safe?? Is there an alternative?

#2 2014-05-13 02:30:10

Re: ZFS for backups with low memory, Is it safe?? Is there an alternative?

#3 2014-05-13 04:08:01

Re: ZFS for backups with low memory, Is it safe?? Is there an alternative?

#4 2014-05-13 04:28:20

Re: ZFS for backups with low memory, Is it safe?? Is there an alternative?

#5 2014-05-13 10:18:53

Re: ZFS for backups with low memory, Is it safe?? Is there an alternative?

#6 2014-05-22 06:05:28

Re: ZFS for backups with low memory, Is it safe?? Is there an alternative?

#7 2014-05-22 07:56:56

Re: ZFS for backups with low memory, Is it safe?? Is there an alternative?

#8 2014-05-22 08:20:57

Re: ZFS for backups with low memory, Is it safe?? Is there an alternative?

#9 2014-05-22 08:24:53

Re: ZFS for backups with low memory, Is it safe?? Is there an alternative?

#10 2014-05-22 08:31:09

Re: ZFS for backups with low memory, Is it safe?? Is there an alternative?

#11 2014-05-27 18:55:30

Re: ZFS for backups with low memory, Is it safe?? Is there an alternative?

#12 2014-05-28 07:23:27

Re: ZFS for backups with low memory, Is it safe?? Is there an alternative?

#13 2014-06-11 22:21:13

Re: ZFS for backups with low memory, Is it safe?? Is there an alternative?

#14 2014-06-14 03:30:24

Re: ZFS for backups with low memory, Is it safe?? Is there an alternative?

#15 2014-06-21 07:11:09

Re: ZFS for backups with low memory, Is it safe?? Is there an alternative?

#16 2014-06-21 10:15:29

Re: ZFS for backups with low memory, Is it safe?? Is there an alternative?

Board footer