You are not logged in.
Wondering if someone could give me some insight into this: I just purchased an ARM based Netgear ReadyNAS 104 and I have a WD Caviar Green 3 TB drive in it (will be adding 3 more of the same shortly) and it formatted the drive with BTRFS and placed the drive in a X-RAID2 (RAID 5 apparently) set. I'm trying to transfer over about 2.5 TB from my Arch Linux PC to the NAS but it's being insanely slow. I have no real experience with RAID and NAS other than my dad's WD Raptors being in RAID 0.
The drive I'm transferring from is the same Caviar Green 3 TB that is inside the NAS but this one is formatted with XFS. I've tried NFS, SMB and FTP but all are either slow or unstable. NFS was clocking in at about 40-50 MB/sec but would regularly drop down to a few hundred KB/sec or just stop, then it would climb back up. SMB would reach up to 100 MB/sec but would drop down to a few hundred KB/sec or stop occasionally like NFS would. FTP stays at a solid 3-6 MB/sec with 3 simultaneous transfers.
I left it run over night, hoping that everything would be done in the morning, but to my dismay I woke up and found that the NAS had crashed and had only transferred about 180 GB. I rebooted it and set it up to transfer again while I would be at work, this time lowering the simultaneous transfers from 10 down to 3. It has been going for about 3 hours and all seems well. I can't see the actual transfer speeds because the web interface won't load for some reason but I can SSH into and see that it is indeed transferring files.
After logging in and running HTOP I can see that the processor is maxed out at 100% with each thread consuming around 25-35% of the total CPU time, meanwhile the load on the router looks fine. The router itself is a Linksys e3200 flashed with Tomato by Shibby and both the NAS, the router and my PC are connected to gigabit ethernet ports along with either CAT5e or CAT6 cable (can't remember).
Before I transferred this huge amount of data I tested it out by transferring about 50 GB via NFS (via a mounted share) and it said it was going to take about a half hour to transfer. I didn't check the CPU usage of the NAS during this.
So my question is, what do you think the bottleneck is? Is it simply that the processor on the NAS can't keep up with everything and is slowing the transfer down? Is my router being a piece of crap (like usual)? Any hints on how to diagnose this?
Edit: decided to stop using FTP and decided to try using Rsync with the --progress flag to see what the actual transfer speed is and it's staying steady at around 50-70 MB/sec, most of the time but now it's currently at 7 MB/sec with CPU usage at 100%.
Here's the output of what I'm experiencing:
[bran@ra /mnt/media]$ sudo rsync --progress -r --remove-source-files * 192.168.1.111:/data/Media/
root@192.168.1.111's password:
sending incremental file list
tv/American Horror Story/Season 01/S01E07 - Open House.mkv
2,341,606,015 100% 77.74MB/s 0:00:28 (xfr#1, ir-chk=1019/1086)
tv/American Horror Story/Season 01/S01E07 - Open House.nfo
3,487 100% 5.25kB/s 0:00:00 (xfr#2, ir-chk=1018/1086)
tv/American Horror Story/Season 01/S01E07 - Open House.tbn
18,331 100% 27.54kB/s 0:00:00 (xfr#3, ir-chk=1017/1086)
tv/American Horror Story/Season 01/S01E08 - Rubber Man.mkv
2,342,160,895 100% 75.00MB/s 0:00:29 (xfr#4, ir-chk=1016/1086)
tv/American Horror Story/Season 01/S01E08 - Rubber Man.nfo
3,436 100% 4.25kB/s 0:00:00 (xfr#5, ir-chk=1015/1086)
tv/American Horror Story/Season 01/S01E08 - Rubber Man.tbn
20,783 100% 25.69kB/s 0:00:00 (xfr#6, ir-chk=1014/1086)
tv/American Horror Story/Season 01/S01E09 - Spooky Little Girl.mkv
2,342,826,357 100% 82.00MB/s 0:00:27 (xfr#7, ir-chk=1013/1086)
tv/American Horror Story/Season 01/S01E09 - Spooky Little Girl.nfo
3,520 100% 13.64kB/s 0:00:00 (xfr#8, ir-chk=1012/1086)
tv/American Horror Story/Season 01/S01E09 - Spooky Little Girl.tbn
33,238 100% 128.81kB/s 0:00:00 (xfr#9, ir-chk=1011/1086)
tv/American Horror Story/Season 01/S01E10 - Smoldering Children.mkv
2,341,992,143 100% 6.63MB/s 0:05:36 (xfr#10, ir-chk=1010/1086)
tv/American Horror Story/Season 01/S01E10 - Smoldering Children.nfo
3,540 100% 6.35kB/s 0:00:00 (xfr#11, ir-chk=1009/1086)
tv/American Horror Story/Season 01/S01E10 - Smoldering Children.tbn
21,754 100% 39.05kB/s 0:00:00 (xfr#12, ir-chk=1008/1086)Notice how it is around 70 MB/sec for the first two or three big transfers and then drops ten-fold?
Offline