Arch Linux

Mox

Member

Registered: 2012-02-07

Posts: 32

Using Unionfs to move heavy packages to another drive

My main drive is quite full. And there are some big and huge packages that I rarely use and don't want to allocate expensive SSD storage space to. Especially, whenever I need these packages it would be acceptable to use another drive (external drive or another internal spinning disk).
So now I am considering using unionfs (or overlayfs or aufs) to install such packages to a second drive:

1. Setup:

   1. Wipe the external drive (just to be sure)

   2. Use pacman --root to install these big packages to the external drive

   Or:

   1. Wipe the external drive (just to be sure)

   2. Create /mnt/external/etc, /mnt/external/usr, /mnt/external/opt and /mnt/external/opt by hand

   3. Use regular pacman (or the wrapper described below) to install my packages for the external drive

      

2. Operation:
   Write a script that will:

   1. Use unionfs (or one of the others) to map /mnt/external/etc --> /etc, /mnt/external/usr --> /usr, /mnt/external/opt --> /opt and /mnt/external/var --> /var

   2. Create some canary that is visible when the external drives are mapped. For example an immutable file like /mnt/external/usr/EXTERNAL_DRIVE_IS_MAPPED

   3. Not run while pacman is running (making some parts of an important package go to external drive but overlaying in mid-upgrade seems like a surefire way to b0rk my system)

   And write a wrapper around pacman that will:

   1. not allow it to run while the canary is visible

   2. Have an option like --intall_to_external_drive which will only run pacman when the canariy is visible

      

3. Updates:
   For updates I would go for the following procedure:

   1. Unmap the external drive if mapped

   2. Do a regular system update

   3. Map the external drive

   4. Do another system upgrade. This time it should only contain those packages that are installed to the external drive

      

My use case is different from this thread and that thread in that I have removed unneccessary packages and cleaned the pacman cache and in that I know perfectly well that there is a host of other files that I could reasonably move off my main drive (even arguably more reasonably than trying this stunt). But ultimately this is my computer whose sole purpose is to serve my needs (or wants) and to support my usage. And that is that I would rather move some packages than (admittedly unneccessarily big amounts of) data from my main drive*. It is also different in that I intend not to insult people

Now, I have a bunch of questions:

1. Do you know of somebody who has already tried or even implemented such a thing? I don't want to reinvent the wheel

2. Can you suggest a search terms that yield more results for this? My search has turned up the mentioned threads but overall results were quite meager.

3. Are there any obvious flaws, security- or stability problems with my approach? Beside the obvious annoyance of having to map/unmap the external drive and having to do the update twice I think this should be safe, right?

4. How should I handle pacmans database of installed packages? Could I do something like this on mapping:

   1. move $database $internal_drive_database the orignal file out of the way

   2. cat $internal_drive_database $external_drive_database > $database

   and that on unmapping:

   1. Replace each package/line in $internal_drive_database with the corresponding package/line in $database and remove that line in $database

   2. Replace the $external_drive_database with the remaining lines from $database

   ?
      

5. I've read that overlayfs is faster and it would seem to me that I would not need the additional features of aufs. And I got the impression that unionfs is older, so probably has some problems prompting to write aufs and overlayfs, but I have yet to find a proper comparison of the three. So, which flavour of unionfs/overlayfs/aufs would you recommend?

Best regards,
mox

* I tried moving photos to my external drive only to discover that I had no access to them when I spontaneously wanted to show some photo when visiting a friend. For work-related programs it is somewhat easier to plan ahead when I will need them and worst case I can ssh to the work machines and use them remotely.

p.S.> Second attempt at writing this. Last time the forum logged me out while writing this and my text was lost on trying to post it