You are not logged in.

#1 2016-08-30 02:14:40

sxlijin
Member
Registered: 2016-08-30
Posts: 8

Choosing Filesystem for Cross-OS Usage

I'm looking to change my current machine configuration to a multi-boot environment (specifically, being able to boot from multiple OS's, including Windows, Hackintosh, and Arch) and to make this change easier (in addition to having a more sane backup system), I want to split my OS from my files.

As such, I expect that I'll use a separate physical hard drive (a 250GB SSD - my first one too!) for my files (I'll be setting up another SSD for the OS's), but my principal concern is what filesystem I should be choosing for said hard drive.

NTFS seems to be the best option, since I expect to use Windows a fair amount and Arch/OSX support it decently well (given ntfs-3g and UserMapping), and there aren't really any other good alternatives (FAT32 nowhere near sufficient for this, exFAT isn't robust enough, ext4 not supported well by Windows/OSX, and so on).

This is, however, the first time I'm going to be undertaking anything like this, so I'd really appreciate any feedback I could get from a wider community about my approach to this all.

Last edited by sxlijin (2016-08-30 03:02:49)

Offline

#2 2016-08-30 02:38:19

Alad
Wiki Admin/IRC Op
From: Bagelstan
Registered: 2014-05-04
Posts: 2,407
Website

Re: Choosing Filesystem for Cross-OS Usage

Typically you'd choose to put your files on a large HDD and keep the operating systems on the SSD, as you benefit more from the improved performance this way. If moving the OS to a new drive is a concern, you can use e.g rsync.

ntfs-3g works, though you should look at the FAQ/pitfalls if you haven't already. Also, it seems the OSX version is non-free.

Last edited by Alad (2016-08-30 02:38:41)


Mods are just community members who have the occasionally necessary option to move threads around and edit posts. -- Trilby

Offline

#3 2016-08-30 15:00:47

mrunion
Member
From: Jonesborough, TN
Registered: 2007-01-26
Posts: 1,938
Website

Re: Choosing Filesystem for Cross-OS Usage

Why is FAT32 a poorer choice than NTFS? It'd shy away from my "universal" drive being NTFS, and chose FAT32.


Matt

"It is very difficult to educate the educated."

Offline

#4 2016-08-30 15:38:43

sxlijin
Member
Registered: 2016-08-30
Posts: 8

Re: Choosing Filesystem for Cross-OS Usage

Alad wrote:

Typically you'd choose to put your files on a large HDD and keep the operating systems on the SSD, as you benefit more from the improved performance this way. If moving the OS to a new drive is a concern, you can use e.g rsync.

ntfs-3g works, though you should look at the FAQ/pitfalls if you haven't already. Also, it seems the OSX version is non-free.

Thanks for the heads-up about the OS/files thing - I forgot to mention that I'm installing two SSD's.

mrunion wrote:

Why is FAT32 a poorer choice than NTFS? It'd shy away from my "universal" drive being NTFS, and chose FAT32.

FAT32 caps out at 32GB, and I'm not chopping a 250GB drive into 32GB partitions if I can avoid it. If you meant exFAT, I'm not using a filesystem without journaling.

Offline

#5 2016-08-30 16:00:18

drcouzelis
Member
From: Connecticut, USA
Registered: 2009-11-09
Posts: 4,092
Website

Re: Choosing Filesystem for Cross-OS Usage

I do multi-boot, with Arch Linux / Haiku / OpenBSD / whatever else I'm in the mood for.

In my opinion, there's no good answer. sad The only guaranteed method of mounting any drive on any OS is by using FAT32, which is old and crappy.

Consider a different approach, such as using virtual machines in Windows, or using CloneZilla for backups to take bit-for-bit snapshots of entire drives.

I'm confused how you can be working in all three operating systems. For me, one is my primary operating system, and the others are just for quick testing or education. Are you really doing work in all three operating systems that needs to be backed up? Can all that work be done in a single operating system instead?

Offline

#6 2016-08-30 16:08:48

Scimmia
Fellow
Registered: 2012-09-01
Posts: 11,466

Re: Choosing Filesystem for Cross-OS Usage

mrunion wrote:

Why is FAT32 a poorer choice than NTFS? It'd shy away from my "universal" drive being NTFS, and chose FAT32.

And I'd very much go the other way, ntfs-3g works fine.

FAT32 is old and has a number of limitations, including file sizes, total files, file name character limitations, and virtually no fault tolerance what so ever. It's probably the worst choice from any perspective other than compatibility.

Offline

#7 2016-08-30 16:15:54

sxlijin
Member
Registered: 2016-08-30
Posts: 8

Re: Choosing Filesystem for Cross-OS Usage

drcouzelis wrote:

I do multi-boot, with Arch Linux / Haiku / OpenBSD / whatever else I'm in the mood for.

In my opinion, there's no good answer. sad The only guaranteed method of mounting any drive on any OS is by using FAT32, which is old and crappy.

Consider a different approach, such as using virtual machines in Windows, or using CloneZilla for backups to take bit-for-bit snapshots of entire drives.

I'm confused how you can be working in all three operating systems. For me, one is my primary operating system, and the others are just for quick testing or education. Are you really doing work in all three operating systems that needs to be backed up? Can all that work be done in a single operating system instead?

My current env is Windows with Ubuntu/Arch in vbox, but because Windows is a PITA so much of the time, I'd like to try the others out. I don't know yet which I expect to keep as a primary; all I know is that I'd like the ability to be able to boot all and work from the same set of common files without worrying about things like my filesystem.

Last edited by sxlijin (2016-08-30 18:42:04)

Offline

#8 2016-08-31 23:59:02

Lone_Wolf
Member
From: Netherlands, Europe
Registered: 2005-10-04
Posts: 11,868

Re: Choosing Filesystem for Cross-OS Usage

sxlijin wrote:

FAT32 caps out at 32GB

That is an artificial limitation microsoft put in their fat32 implementation , the real limit for fat32 volumes is 2 Tib .
(I once formatted a 500 GB external drive as fat32 from AL and windows had no porblems using it).

The filesize limit is smaller though, usually 2 Gib - 1 (can also be 4Gib-1 but that's not supported on all fat32 systems)


Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.


(A works at time B)  && (time C > time B ) ≠  (A works at time C)

Offline

#9 2016-09-01 00:24:02

TheChickenMan
Member
From: United States
Registered: 2015-07-25
Posts: 354

Re: Choosing Filesystem for Cross-OS Usage

I've had trouble getting Apple systems to write to NTFS. What about exfat?


If quantum mechanics hasn't profoundly shocked you, you haven't understood it yet.
Niels Bohr

Offline

#10 2016-09-01 08:27:24

kozaki
Member
From: London >. < Paris
Registered: 2005-06-13
Posts: 671
Website

Re: Choosing Filesystem for Cross-OS Usage

That is an artificial limitation microsoft put in their fat32 implementation , the real limit for fat32 volumes is 2 Tib .
(I once formatted a 500 GB external drive as fat32 from AL and windows had no porblems using it).

I can confirm that, having worked with a single partitionned 160 gig fat32 HDD for many months without trouble.


Seeded last month: Arch 50 gig, derivatives 1 gig
Desktop @3.3GHz 8 gig RAM, linux-ck
laptop #1 Atom 2 gig RAM, Arch linux stock i686 (6H w/ 6yrs old battery smile) #2: ARM Tegra K1, 4 gig RAM, ChrOS
Atom Z520 2 gig RAM, OMV (Debian 7) kernel 3.16 bpo on SDHC | PGP Key: 0xFF0157D9

Offline

#11 2016-09-01 10:00:30

Awebb
Member
Registered: 2010-05-06
Posts: 6,275

Re: Choosing Filesystem for Cross-OS Usage

TheChickenMan wrote:

I've had trouble getting Apple systems to write to NTFS. What about exfat?

The exfat support on everything but Windows is still questionable. The legal situation is as bad as with NTFS, but it has the least mature software.

Offline

#12 2016-09-01 19:51:15

drcouzelis
Member
From: Connecticut, USA
Registered: 2009-11-09
Posts: 4,092
Website

Re: Choosing Filesystem for Cross-OS Usage

Since there is no perfect filesystem, how about something else?

How about, using a little NAS that you can access as if it was locas by using NFS mounts? It would work in any operating system, even Haiku! big_smile

Offline

#13 2016-09-01 19:53:55

graysky
Wiki Maintainer
From: :wq
Registered: 2008-12-01
Posts: 10,595
Website

Re: Choosing Filesystem for Cross-OS Usage

drcouzelis wrote:

Since there is no perfect filesystem, how about something else?

How about, using a little NAS that you can access as if it was locas by using NFS mounts? It would work in any operating system, even Haiku! big_smile

+1 This is a great suggestion unless you're wanting to keep it simple (ie 1 box, 3 operating systems, no desire to build/maintain a NAS)... plus access time unless you're gigalan+ will suffer to/from a NAS.

Last edited by graysky (2016-09-01 19:55:51)


CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#14 2016-09-01 20:54:55

kokoko3k
Member
Registered: 2008-11-14
Posts: 2,390

Re: Choosing Filesystem for Cross-OS Usage


Help me to improve ssh-rdp !
Retroarch User? Try my koko-aio shader !

Offline

#15 2016-09-02 16:11:20

drcouzelis
Member
From: Connecticut, USA
Registered: 2009-11-09
Posts: 4,092
Website

Re: Choosing Filesystem for Cross-OS Usage

AND IT WORKS WITH HAIKU! big_smile

...Sadly, in practice it still might not work, at least not how we imagine. sad

http://askubuntu.com/questions/27936/ca … ive-format

Offline

#16 2016-09-02 19:06:01

Awebb
Member
Registered: 2010-05-06
Posts: 6,275

Re: Choosing Filesystem for Cross-OS Usage

Summary of what we know (not much):

FAT32 works as well as it can on Linux, OSX and Windows, ...
... but Windows cannot create very big partitions (arbitrary limit to promote NTFS).
... but "the boot sector uses a 32-bit field for the sector count, limiting the FAT32 volume size to 2 TiB for a sector size of 512 bytes and 16 TiB for a sector size of 4,096 bytes" -> Wikipedia.
... but file sizes are limited to 2 GiB/4 GiB (LFS limit on Windows).

NTFS works well on Windows ...
... and reasonably well on Linux with the occasional performance problem. Either fuse-only (ntfs-3g) or somewhat buggy (is the ntfs still in the kernel?), legal reasons, same as exFAT.
... but not OOTB on OSX, with either expensive commercial solutions.
... or for free, but with user interaction outside the usual OSX user's comfort zone (turning off System Integrity Protection is something you'd do on your own Mac, but not on, say, aunt Audelle's Macbook (if aunt Audelle has a say in it).

exFAT seems to work on Linux, OSX and Windows ...
... but while there indeed is a kernel module for Linux, it's basically nowhere because of legal reasons, so we only have a FUSE module to work with, which can or cannot mean performance issues.
... but fsck repair doesn't work at the moment.

EXT2/3/4 works as intended on Linux ...
... but only with so-so stable third party drivers on Windows. I haven't had data loss in over a year, but that's not sufficient evidence.
... but only with "you're on your own" type third party FUSE drivers on OSX.

HFS/HFS+ works on OSX (natively) ...
... but only through a FUSE system on Linux.
... but only through either unstable or commercial (as in you have to pay money).


What I think I know (not that much either):

- LSB for FAT32 is a clusterfuck between operating systems and requires research on all ends before it works flawlessly, so let's assume a safe filesize limit of 2 GiB.
- Getting a Mac user to install third party file system compatibility layers, so they can look at your vacation photos is - considering the invasive nature of the task - is probably not easy.
- We know the benefits and limits of FUSE, but we cannot predict how custom solutions on Windows behave.


What I would do, if I had to:

- Use a Windows native FS and pick the one with the least resistance on OSX, because if it's only about transport and not archive level storage, Linux has the diagnostic advantage, in case anything goes wrong.
- I'd install exFAT support on Linux and be done with the whole thing. It works with util-linux, it seems to be mountable via fstab and the performance doesn't seem to be worse than nfts-3g's. It's the least hassle to take your external drive and plug it into a friend's or client's computer.
- If I had to deal with different file systems on those three operating systems in a professional environment, I'd install all the usual Linux packages and contact Paragon for an offer regarding their Ext for Windows and OSX, their HFS/HFS+ for Windows and their NTFS for OSX. I have worked with all of them and they work reasonably well. When dealing with Windows and OSX, I assume you don't have a F(L)OSS only mindset anyway. A single license was around 10 bucks per system last time I checked and they offer the usual volume license discounts.


What I actually do:

- Since I don't own any apple devices and since I already have all the NAS/SAN/SOHO infrastructure, everything is done over the network.


Bonus:

- I will have to contact Paragon for an ad deal, because I'm going to mention them twice in the same post, but they have a free (donationware) HFS+/NTFS/exFAT driver for Android, that works reasonably well with both internal drives as well as USB OTG stuff.

Offline

#17 2016-09-10 18:02:35

sxlijin
Member
Registered: 2016-08-30
Posts: 8

Re: Choosing Filesystem for Cross-OS Usage

Thanks to everyone for the suggestions! I ended up going with NTFS, since the OSX/Linux stuff is reasonably far enough into the distant future that I'd rather not handicap my current use case (Win7 host with Arch VM, file sharing via vboxsf). It's not the best, but it's been a decently acceptable compromise.

The FAT32 stuff is great to learn, but the file size limit (although for most purposes insignificant) is something I'd much rather have to not worry about, particularly since I do have to deal with the occasional multi-GB file.

For the people suggesting NAS: I did actually consider that briefly, but as I'm a university student running around, the only way to do that would be to have FreeNAS or something running in a VM, with direct write access to the physical drive (which Windows already complicates immensely, as I've learned), and the virtualbox networking drivers would probably throttle r/w performance a fair amount (since I'm running on SSD's now), which I'd rather not deal with.

Last edited by sxlijin (2016-09-10 18:04:09)

Offline

Board footer

Powered by FluxBB