decentralized distributed p2p filesystem driver - does it exist?

cmtptr · 2012-08-01 02:25:40

For over a year my home server's "backup" has been a "mount; cp -l; rsync; umount" script kicked off from cron to create a nightly incremental backup saved on an external drive. This is nice because the implementation is very simple and it gives me the feature of being able to roll back or peek at historical copies of files at very little cost to disk space.

The problem is that it sits two feet from my machine, so there's absolutely no protection whatsoever from fires, tornados, nuclear holocaust, Godzilla, or anything else that could potentially wipe out everything in my apartment all at once. I started Googling for free online storage solutions when I found SymForm and Tahoe-LAFS. This is the first exposure I've had to the concept of distributed peer-to-peer filesystems (but SymForm is not yet available for Linux, and Tahoe-LAFS is not decentralized).

So then I started thinking: From what I've read about these, I'm not totally happy with how they're implemented. What would be AWESOME is if they were implemented as a filesystem driver. I could simply create a partition, mount it with the right options (put it in fstab!), and when I access the filesystem I'm actually accessing my distributed/clouded data. Meanwhile, what's actually written to the disk on that partition is the encrypted, fragmented data uploaded by everyone else. Then my backup script doesn't change; I just point it to the directory where my distributed filesystem is mounted, and it would "cp -l; rsync" as usual.

So, does this exist? Is there something at least close to what I described? The closest I've found is Tahoe-LAFS, using its ssh interface to mount it using FUSE sshfs. However, I don't like that Tahoe-LAFS is not decentrialized. The website talks a lot about creating a grid with my friends. I don't have any friends, and even if I did I would still prefer to have my data strewn across thousands of machines all over the planet, so that's not an option for me!

joshdmiller · 2012-08-01 04:33:53

You could do this with Amazon S3. There is a FUSE-based driver called s3fs that is available in the AUR. With S3, your data is stored redundantly. S3 is not a typical filesystem, so there are some limitations, but you can rsync to it using s3fs.

In theory, you can also sync an rsync'd folder to both Dropbox and Ubuntu One, but I can't vouch for these.

TedP · 2013-01-31 20:03:12

Update: SymForm now has a Linux beta. 32-bit and 64-bit RPM's and DEB's are available for download. Maybe someone should create an AUR for them...

chris_l · 2013-01-31 20:14:45

So, if SymForm had a fuse driver, that would complish your requirements?

Arch Linux

#1 2012-08-01 02:25:40

decentralized distributed p2p filesystem driver - does it exist?

#2 2012-08-01 04:33:53

Re: decentralized distributed p2p filesystem driver - does it exist?

#3 2013-01-31 20:03:12

Re: decentralized distributed p2p filesystem driver - does it exist?

#4 2013-01-31 20:14:45

Re: decentralized distributed p2p filesystem driver - does it exist?

Board footer