Removable storage and file ownership problem.

kokoko3k · 2018-01-31 12:45:39

Say i need to share big files (more than 4GB) between 2 Linux machines via an usb stick.
The first filesystem that comes to mind is exfat, of course.
But what if i've no root acces to the target system and exfat module is not installed on it?

As far as i know, ext is not an option either, because it supports permissions and file ownership and that can cause troubles on the target machine.
One can think that just chmod 777 would suffice, but unfortunately it is not always true, that happened just yesterday while trying to pass a steam game backup via an usb stick.
Steam just refused to read the game, because the ownership of the game files and the user account that started steam differs, even if the entire mounted filesystem was "chmodded" to 777; it took me a while to understand it (damn).

What have i considered so far:
fat: File size limit
exfat: needs to be installed as a FUSE module -> needs root access to install on every target machine.
UDF: it seems to support file ownership and permissions as well, and i don't know of a way to disable them vias
ext: see UDF (plus, tune2fs does not seem to offer anything useful).

It is not a big deal, i can always lower steam backups volume size; and of course there are workarounds, but i wonder if there is a way to just format that usb stick in a way i can place big files on it and read it on other linux systems i've not root access to without issues.

Thanks.

Slithery · 2018-01-31 12:50:01

NTFS would allow read access on any Arch installation - you only need to install ntfs-3g if you want write access as well.

kokoko3k · 2018-01-31 13:02:50

Doesn't it support ownership and so expose the problem i've had with some funky software (steam) refusing to access files if the ownership is "wrong?"

V1del · 2018-01-31 13:05:07

No, the ownership the NTFS drivers expose is simply to not break every unix tool that attempts read permission information. It will always "show" the ownership of the mounting user, no matter what they are actually mapped to.

Trilby · 2018-01-31 13:22:05

I don't understand your problem with ext. You can change the ownership of the files on the target machine. In fact, just `cp`ing them into place will do this by default.

kokoko3k · 2018-01-31 14:08:08

Thanks V1del,slithery.
Then the best bet seems to be NTFS, hoping the target machine will have not installed NTFS-3G.
Plus, it is slow.

Trillby,
for some reason i ignore, steam refuses to read the backup files directly from the stick if

kokoko3k wrote:

the ownership of the game files and the user account that started steam differs

.

Trillby wrote:

You can change the ownership of the files on the target machine. In fact, just `cp`ing them into place will do this by default.

What do you mean by "in place"?

[15:04:27]  cd /run/media/koko/PNY_30gB_ext2/civ6/Disk_1
koko@Gozer# id|cut -d " " -f 1
uid=1012(koko)

[15:04:27]  cd /run/media/koko/PNY_30gB_ext2/civ6/Disk_1
koko@Gozer# ls -la
total 1160788
drwxrwxrwx  2 1000 users       4096 gen 22 11:52 .
drwxrwxrwx 18 1000 users       4096 gen 22 11:52 ..
-rwxrwxrwx  1 1000 users  176400942 gen 22 11:29 533502_depotcache_1.csd
-rwxrwxrwx  1 1000 users      24932 gen 22 11:29 533502_depotcache_1.csm
-rwxrwxrwx  1 1000 users 1010938066 gen 22 11:30 533503_depotcache_1.csd
-rwxrwxrwx  1 1000 users      79292 gen 22 11:30 533503_depotcache_1.csm
-rwxrwxrwx  1 1000 users       1044 gen 22 11:52 sku.sis

15:04:36]  cd /run/media/koko/PNY_30gB_ext2/civ6/Disk_1
koko@Gozer# cp sku.sis sku.sis
cp: 'sku.sis' and 'sku.sis' are the same file

...yes, i can copy the files to the target machine and so change the owner, but i'd really like to avoid an extra copy.

Trilby · 2018-01-31 14:18:04

kokoko3k wrote:

...yes, i can copy the files to the target machine and so change the owner, but i'd really like to avoid an extra copy.

Ah, I didn't realize you wanted to just keep/use them on the removable media. Then yes, ext would be a problem.

V1del · 2018-01-31 14:43:41

If the explicit goal is running steam games off of the HDD, i'm not sure that would work, there used to be many complaints/bug reports about Steam games refusing to run on drives without an ext4 partition, specifically because Steam does try to check and set correct UIDs itself. However I'm not fully updated on the current situation (and especially it being essentially ro might also break...).

And regarding the hoping that NTFS3-G isn't installed, why not? That would be even better. Because it is better than the kernel ntfs driver even for reading and the same 'limitations' apply there.

R00KIE · 2018-01-31 19:45:54

Is the second machine where you don't have access to root a random machine or always the same machine? If the later you could always on the source machine chown the files to the user/group you are using on the second machine. You can chown to a numeric uid/gid which does not need to exist on the source machine. Not the easiest/cleanest solution but at least avoids having to rely on ntfs-3g being installed on the target machine and avoids the speed/cpu usage cost of using ntfs-3g.

kokoko3k · 2018-02-01 10:44:08

V1del:
The explicit goal is to have, ideally, an exfatfs like filesystem built into Linux.
My specific trouble was not to start the game from the removable device, but telling steam to restore a game backup from it.

R00KIE:
That's a good idea to workaround specific cases.

Apart from specific issues one can circumvent in a way or another, i feel that linux (kernel) is missing something comparable to exfat for removable devices.
Maybe i can even understand the usefulness of file permissions even on a usb stick,
but given the nature of the removable media, something you carry around and plug in different systems, with different users and different uids,what's the point of using a filesystem that restricts file access basing on uids? It makes no sense to me.

Last edited by kokoko3k (2018-02-01 10:52:05)

progandy · 2018-02-01 15:55:43

kokoko3k wrote:

but given the nature of the removable media, something you carry around and plug in different systems, with different users and different uids,what's the point of using a filesystem that restricts file access basing on uids? It makes no sense to me.

Maybe a patch would be accepted if you wrote one for a file system feature in ext2-4?

Edit: Is it necessary to be root in order to install exfat? It should be fuse filesystem, so can't you just copy the executable somewhere and then use it to mount?
Edit: I forgot you need access to the /dev node, so probably not.

Last edited by progandy (2018-02-01 16:04:23)

kokoko3k · 2018-02-02 12:08:29

progandy wrote:

Maybe a patch would be accepted if you wrote one for a file system feature in ext2-4?

Yes, maybe, if only i knew C...

Arch Linux

#1 2018-01-31 12:45:39

Removable storage and file ownership problem.

#2 2018-01-31 12:50:01

Re: Removable storage and file ownership problem.

#3 2018-01-31 13:02:50

Re: Removable storage and file ownership problem.

#4 2018-01-31 13:05:07

Re: Removable storage and file ownership problem.

#5 2018-01-31 13:22:05

Re: Removable storage and file ownership problem.

#6 2018-01-31 14:08:08

Re: Removable storage and file ownership problem.

#7 2018-01-31 14:18:04

Re: Removable storage and file ownership problem.

#8 2018-01-31 14:43:41

Re: Removable storage and file ownership problem.

#9 2018-01-31 19:45:54

Re: Removable storage and file ownership problem.

#10 2018-02-01 10:44:08

Re: Removable storage and file ownership problem.

#11 2018-02-01 15:55:43

Re: Removable storage and file ownership problem.

#12 2018-02-02 12:08:29

Re: Removable storage and file ownership problem.

Board footer