Arch Linux

DaMadOne

Member

Registered: 2014-07-30

Posts: 16

ntfs-3g speed up SSD writes?

First, I hope this is in the right place for this, mods feel free to scold me and move it if not.

I've not had the pleasure of having this particular issue(feature?) before. I'm a gamer so I have a Windows install(/me ducks the incoming tomatoes) solely for gaming that I can't do in Linux. I have 2x 250GG 840 EVO SSD's, one for Linux and one for Windows. You can't hold too many games these days on a 250GB drive so I've had a 1TB HDD that I would move games to I wasn't currently playing in both linux and windows plus various other things that weren't worthy of my then limited SSD space. Yesterday I got a 960GB Sandisk Ultra II SSD to replace that drive($200 for "1tb" of SSD? yes please!). I've been using that 1TB HDD formatted to NTFS so that I could easily use it with both OS's for a long time, but it's mechanical so I never hit the limits I'm seeing with ntfs-3g+SSD now.

Alll is fine with the drive in windows obviously but now for the problem... with ntfs-3g the reads are nearly as fast as in windows, but writing is not. With a normal copy operation using cp or with any of the file manager GUI's I've tried writing seems to top out at around 90-98MB/s. It's not processor limited as a normal copy operation doesn't even hardly bring the processor out of it's lowest power state and doesn't use more than a 1/4-1/3 of a core and it's certainly not memory limited with 32GB of DDR4. So what gives?!

I've been banging my head against the wall for hours now over this... I have big_writes set on its mount in fstab which allows for writing larger chunks at a time but cp and file managers don't seem to care. I have figured out that with big_writes set I can use DD or rsync to copy files from my Linux SSD to it nearly as fast or faster than Windows by specifying a 128k+ block size so ntfs-3g is perfectly capable of it. It seems cp and file managers just won't write chunks larger than 4k since I get comparable speeds to cp or file managers when using dd or rsync with bs set to 4k. The gnome-discs benchmark on the drive also shows it to be blistering fast.

It's obviously fast if I format it to EXT4, but having the space available to Windows a must. It's not the biggest of problems seeing as how reads are still fast and I can use dd or rsync to move larger files but if there's a fix I'm all for it! So... am I just going to have to live with this limitation or is there something I'm missing here? Thanks in advance!

Last edited by DaMadOne (2015-11-28 20:33:44)

frostschutz

Member

Registered: 2013-11-15

Posts: 1,647

Re: ntfs-3g speed up SSD writes?

NTFS-3G is slow. Be glad it works at all.

DaMadOne

Member

Registered: 2014-07-30

Posts: 16

Re: ntfs-3g speed up SSD writes?

NTFS-3G is slow. Be glad it works at all.

I understand there is overhead and it was once really slow.. but seeing as how I can use rsync with bs set to 128K and transfer at almost 400MB/s it certainly can be fast.

mich41

Member

Registered: 2012-06-22

Posts: 796

Re: ntfs-3g speed up SSD writes?

Not that it fixes your issues with NTFS, but UDF is another usable filesystem supported by Windows

EDIT: Just tested ntfs-3g on ramdisk - dd bs=4k is indeed about 90MB/s, but cp was going at 700MB/s. Not sure why it's slower for you.

Last edited by mich41 (2015-11-29 09:50:14)

DaMadOne

Member

Registered: 2014-07-30

Posts: 16

Re: ntfs-3g speed up SSD writes?

Not that it fixes your issues with NTFS, but UDF is another usable filesystem supported by Windows

I hadn't thought about that.. exFAT would also be an option, but I've never used either on an HDD/SSD. I suppose theoretically exFAT having been made with flash storage in mind should be the better option than one made with DVD's in mind I guess I need to research exFAT and see if it would be ok.

I've got a 120gb SSD I have OSX installed on.. I'm going to dd it and then format it with exFAT and see how that goes.

EDIT* exFAT certainly overcomes the speed problem. 450MB/s burst with 350+MB/s sustained copying a 40GB junk file I made with dd from my EXT4 SSD to the exFAT SSD. The SSD i'm using as a test is a really cheap PNY with mediocre speeds.. so that's really good.

I mostly worry about the lack of journaling in the long run...

EDIT Again* windows didn't recognize the exFAT partition I made in Linux, but Linux recognizes it having been formatted to exFAT from windows. Weird.. oh well. I have no need for OSX ATM so I guess I'm going to leave this SSD formatted as exFAT and use it for some things and see how it goes.. In the long run I may not even find the writing limitations of ntfs-3g on the big SSD to really bother me for the limited amount of writing it will get from Linux and I can still install Linux games on it and read speed is fine. It will just be another one of those Linux quirks I get use to breaking out the terminal to copy large files with rsync or dd should I need to.

Last edited by DaMadOne (2015-11-29 01:19:38)

ezacaria

Member

Registered: 2007-12-10

Posts: 113

Re: ntfs-3g speed up SSD writes?

In my case, the performance of plain "cp" is not limited at all. I found that big_writes makes a big difference in my system, though. Copying a 3.3GB chunk from tmpfs to the ntfs-formatted SSD goes only at 150 MB/s without big_writes, and jumps to 302 MB/s when adding the option to the mount command.

I am not sure if it will help, but is there a large difference in the amount of "Writeback" reported in /proc/meminfo, for the cases of "cp" and "rsync"? I noticed that this can vary depending on the filesystem in the target disk. If the writeback amount is seemingly too low when using ntfs-3g, maybe you could attempt to increase it by raising the max_sectors parameter.

I monitor the "Writeback" like this when benchmarking, to get a rough idea of whether the source file goes fully to the OS buffers in the very beginning, and how much data the system attempts to push to the disk.

watch grep -e Dirty: -e Writeback: /proc/meminfo

As for the max_sectors, it seems that one may simply raise the default 240 to something else by setting it in sys/block:

echo newValue > /sys/block/sdN/device/max_sectors

My experience with ntfs-3g has been good for many years. My only complaint is that for some reason writing to NTFS-formatted USB sticks has become very slow of late - but that is a different story