You are not logged in.

#1 2013-08-16 18:43:05

2ion
Member
Registered: 2013-04-19
Posts: 74

[Solved] Reflect all changes to a filesystem in another filesystem

I'm looking for a solution to the following problem:

I want to setup an external harddisk so that two different partitions on it behave in the following way:

Given an ext4 partition mounted rw at /mnt/ext and a NTFS partition mounted also rw at /mnt/ntfs, any changes I do under /mnt/ext should be reflected exactly under /mnt/ntfs minus everything ext4 specific. The NTFS partition should work as an exact mirror of the ext4 partition, reflecting all writes, deletes, and changes in the directory structure.

The purpose of this is having a Linux native FS with all the journaling and stuff for normal use, while being able to go around in a Windows/Mac infested  environment and pass on data from the NTFS partition to random users without having to maintain/sync everything to the NTFS partition beforehand.

I have looked into union mounts, especially aufs3 as a seemingly up-to-date and actively maintained project, but got a little bit lost in the documentation because of its complexity.

From the LWN series on unioning filesystems (a little old) at http://lwn.net/Articles/396020 (under the section "background") I learned that union FSs are mostly used to stack one/or multiple read-write FS onto a read-only FS.

At least for aufs, its design documentation https://github.com/sfjro/aufs3-linux/bl … 1intro.txt states that it supports "multiple writable branches", but "several policies to select one among multiple writable branches", indicating that a certain change can be written only to one of several branches at the same time, which, as explained above, is not my intention.

Does somebody know whether such a setup can be realised using aufs or any other working union fs solution, or could point me towards a more simple and better solution? (Or at least tell me that it is impossible.)

Thanks in advance.

Last edited by 2ion (2013-08-19 22:35:57)

Offline

#2 2013-08-16 18:50:16

WonderWoofy
Member
From: Los Gatos, CA
Registered: 2012-05-19
Posts: 8,412

Re: [Solved] Reflect all changes to a filesystem in another filesystem

Yeah, aufs won't actually mirror the filesystem like you want here.  Honestly, I'm not entirely sure why you don't just use the NTFS filesystem to store these files anyway.  I mean, unless you have some sort of performance requriements for the files stored on the ext4 side, this seems unnnecessary.

But if this setup is a requirement for whatever you want to do here (for example if the NTFS is on removable media and the ext4 is internal) then I would just set up an rsync on some kind of cronjob or systemd.timer.  This of course would mean that you would have a kind of delay for when the actual changes occurred, but with a sufficiently low interval, I think it would be fine (not too low or you might end up with multiple rsync processes occuring at the same time).

Maybe someone else might have a better idea...

Offline

#3 2013-08-16 18:58:49

alphaniner
Member
From: Ancapistan
Registered: 2010-07-12
Posts: 2,716

Re: [Solved] Reflect all changes to a filesystem in another filesystem

I did some searching, starting with "linux real-time drive mirroring". Most of what I found had to do with network synchronization. But there was some stuff about FAM and inotify (this for example). Maybe worthwhile to look into.

Last edited by alphaniner (2013-08-16 19:02:59)


But whether the Constitution really be one thing, or another, this much is certain - that it has either authorized such a government as we have had, or has been powerless to prevent it. In either case, it is unfit to exist.
-Lysander Spooner

Offline

#4 2013-08-16 20:16:21

2ion
Member
Registered: 2013-04-19
Posts: 74

Re: [Solved] Reflect all changes to a filesystem in another filesystem

WonderWoofy wrote:

Yeah, aufs won't actually mirror the filesystem like you want here.  Honestly, I'm not entirely sure why you don't just use the NTFS filesystem to store these files anyway.  I mean, unless you have some sort of performance requriements for the files stored on the ext4 side, this seems unnnecessary.

But if this setup is a requirement for whatever you want to do here (for example if the NTFS is on removable media and the ext4 is internal) then I would just set up an rsync on some kind of cronjob or systemd.timer.  This of course would mean that you would have a kind of delay for when the actual changes occurred, but with a sufficiently low interval, I think it would be fine (not too low or you might end up with multiple rsync processes occuring at the same time).

Maybe someone else might have a better idea...

Yes, the reason I'm uncomfortable with using NTFS is that in the past, I once ended up with a damaged NTFS file system (invalid data read, errors on write) which to repair I found impossible using just the NTFS tools available on Linux. The internet basically said to go and use Windows to fix it, but I at the time didn't have a Windows system, and possibly won't have one in the future – thus I can't rely on NTFS. And I'd rather not run a Windows system utility under WINE.

Offline

#5 2013-08-16 20:22:12

2ion
Member
Registered: 2013-04-19
Posts: 74

Re: [Solved] Reflect all changes to a filesystem in another filesystem

alphaniner wrote:

I did some searching, starting with "linux real-time drive mirroring". Most of what I found had to do with network synchronization. But there was some stuff about FAM and inotify (this for example). Maybe worthwhile to look into.

Thanks for the pointer. I indeed played with the thought of using inotify to watch the tree and executing rsync if needed, or running rsync in daemon mode to sync the folders, but this wouldn't have had the beauty of solving the problem at the file system level.

Offline

#6 2013-08-17 11:08:23

Lone_Wolf
Member
From: Netherlands, Europe
Registered: 2005-10-04
Posts: 4,299

Re: [Solved] Reflect all changes to a filesystem in another filesystem

This does remind me a bit of the famous x y problem : http://mywiki.wooledge.org/XyProblem

basically, you want to have some data on a ext4 partition, but the data must be easily accessible for windows / mac systems ?

IF providing access to the data over network is enough for your purposes, you could make the data available through a samba share.


Booting with apg Openrc, NOT systemd.
Automounting : not needed, i prefer pmount
Aur helpers : makepkg + my own local repo === rarely need them

Offline

#7 2013-08-17 11:46:24

2ion
Member
Registered: 2013-04-19
Posts: 74

Re: [Solved] Reflect all changes to a filesystem in another filesystem

I admit that "using 2 data-wise syncronized partitions" could be the Y of an X "sharing data with Mac and Windows users".

However, I  have specifically situations in mind where sharing data over a network is not a viable or the more complicated solution. It's really all about having an USB thumb drive/USB hard disk that can be plugged into just any Windows or Mac users computer who wants to spontaneously receive data from me, like any technically illiterate user (just as technically illiterate as I am when it comes to helping them setup their network config on their PC to connect to mine), when I don't have my hands on my computer.

So I wondered whether the idea above could have an elegant solution on the file system level. True, it's not the obvious way to go (just using one NTFS partition), but this is why I asked the residents of System Administration.

Offline

Board footer

Powered by FluxBB