You are not logged in.
prelude: I'm not sure which is the correct forum to post this, please feel free to move around if needed.
Fellow archers,
I'm trying to build a decent NAS from some old hardware I've found but could use some help. The cpu used is a (single core) AMD Athlon XP 2100+ with 512mb ram and I bought a 4-port PCI LSI MegaRaid SATA Raid card from ebay and hooked 3 new Samsung 500Gb drives on it. I configured a RAID5 array in the raid card bios so it's seen as a single drive from Linux and used an LVM2 setup on the top of it with separate volumes for root, storage and backup. All logical volumes are using XFS as this is the fs I've been trusting with my data for the last 6+ years on all my boxes.
Now the problems.. "hdparm -t /dev/sda" gives a decent read performance with 50MB/sec - not the fastest, but acceptable given the old CPU driving the setup. However, real-world usage like copying a file in the same volume, moving to an other volume or copying over gigabit network has never shown anything faster than 3-4MB/s.
According to top, eg. during file copy 'cp' takes an average 2-3% (with a max of ~10%) cpu usage and about 95% of the time is spent in waiting.. memory does not look limited either with an average 23Mb of swap used and 240Mb used as cache. What I tried to do to improve performance (with no success) is setting the scheduler to 'noop' for sda, increasing the number of requests to 1024 and set the read ahead to 256 but I think that'd be fine-tuning and I think I'm heavying a more serious performance loss here on something..
I'm willing to spend money on upgrading any components if required, but unfortunately I'm not sure what's causing the I/O bottleneck.. any ideas?
Offline
Which kernel are you using? .35 has issues with I/O congestion (I don't know if it matters).
Offline
not using .35 yet..
actually with switching writethrough cache to writeback cache in the megaraid card's bios, write performance improved to 15-20Mb/s which is lightyears ahead of the previous values.. but I'm still open for suggestions.
Offline
You should do data collecting first.
hdparm is worthless. Doesn't really tell anything really.
Try bonnie++ -s <2x amount of your ram> -d <folder on your RAID5> -u <your user>
e.g "bonnie++ -s 8192M -d /raid -u colyte"
You could also possibly run "bonnie++ -s <2x amount of your ram> -d <folder on your RAID5> -u <your user > file.csv" to get the csv output as well. (It will still provide the text output in your standard output (console))
Run it 3 times or so just to cover you basis.
You could also make use of the following optimizations and try the bonnie++ benchmark again:
blockdev --setra 65536 /dev/md0
blockdev --setra 2048 /dev/sdx /dev/sdx /dev/sdx
I assume that you haven't put this into use yet, if you could provide test results with ext4 in addition to XFS that would be neat.
// colyte
Offline
colye, thanks a lot for your suggestions, it was really helpful!
I did not yet have time for extensive measures but running bonnie in a for cycle gave results like this:
Version 1.96 ------Sequential Output------ --Sequential Input- --Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
epsilon 1G 417 99 38237 13 19158 8 594 86 53733 13 280.8 6
Latency 22516us 1324ms 703ms 659ms 28124us 178ms
Version 1.96 ------Sequential Create------ --------Random Create--------
epsilon -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 2234 18 +++++ +++ 2749 21 2799 23 +++++ +++ 2502 21
Latency 190ms 276us 33168us 63822us 44us 98876us
1.96,1.96,epsilon,1,1283465486,1G,,417,99,38237,13,19158,8,594,86,53733,13,280.8,6,16,,,,,2234,18,+++++,+++,2749,21,2799,23,+++++,+++,2502,21,22516us,1324ms,703ms,659ms,28124us,178ms,190ms,276us,33168us,63822us,44us,98876us
which is a bit difficult both to read and to interpret without any previous bonnie experience.. can you please help highlighting any problems with outputs like this?
Offline
Maybe this will help http://www.linux.com/archive/feature/139742
Offline