Script to compress /usr with squashfs

brando56894 · 2010-02-21 00:11:20

I found an article in the wiki that points to a how-to on the gentoo forums on how to compress your /usr directory into a squashfs to increase the read speed of programs. It was slightly confusing since it was obviously meant for a gentoo system and some of the directions were slightly confusing since they didn't mirror what should be done in an arch system.

I wrote a bash script to do all this for the user and figured I'd post it here first for any suggestions before I added it to the wiki.

Heres the main script

#!/bin/bash

#Created by Brando56894 from the Arch Linux Community
#Based off of a tutorial found on the Gentoo forums
#This script comes with ABSOLUTELY NO WARRANTY use it at your risk!

#checks to see if the user is root
if [[ $UID != "0" ]];then
     echo "This script MUST be run as root"
     echo "Please login to the root account"
     echo "And re-execute this script"
     exit
fi

echo "This script will place /usr into a squashfs for faster loading"
read -p "Press enter to continue"

#installs aufs2 and squashfs tools
pacman -Sy --needed aufs2 aufs2-util squashfs-tools

#creates the directories that /usr will be squashed into
echo
mkdir -p /squashed/usr/{ro,rw}

#compress /usr into a squashfile with a 64k blocksize
if [[ -e /var/lib/pacman/db.lck ]]; then
  echo "Pacman seems to be in use, please wait for it to finish"
  echo "before you create the squashfs image or else nasty things"
  echo "will most likely happen to your system"
else
  mksquashfs /usr /squashed/usr/usr.sfs -b 65536
fi

#adds the filesystems to fstab
echo "Please add the following lines to your /etc/fstab"
echo
echo "/squashed/usr/usr.sfs    /squashed/usr/ro squashfs     loop,ro 0 0" 
echo "usr /usr  aufs  br:/squashed/usr/rw:/squashed/usr/ro 0 0"
echo
read -p "Press any key to continue after you finished editing the file"
echo

#probably not needed since the umount -a option should do it but left it in case it is needed
#unmounts the squashfs during shutdown
#echo "Please add the following lines to /etc/rc.shutdown"
#echo "under the Unmounting Filesystems section"
#echo
#echo "umount -l /usr"
#echo "umount -l /squashed/usr/ro"
#echo
#read -p "Press any key to continue after you finished editing the file"
#echo

#move the /usr folder instead of deleting it
mv /usr /usr.old
mkdir /usr

echo "Would you like to set up a cron job to remake the image"
echo "Every three weeks? (y or n)"
read choice
if [[ choice == "y" ]];then
    #sets up a cron job to remake the image every three weeks
    echo "Please add the following to your crontab"
    echo "and place remake-squash.job in /etc/cron.monthly"
    echo 
    echo "It will remake the sqashfs image every 21 days at noon"
    echo
    echo "0 12 21 * * bash /etc/cron.monthly/remake-squashfs.job"
fi

and heres the remake-squashfs.job script:

#!/bin/bash

#This script comes with ABSOLUTELY NO WARRANTY use it at your own risk!

#cron job to rebuild the squashfs image of /usr
if [[ -e /var/lib/pacman/db.lck ]]; then
  exit
fi

mksquashfs /usr /squashed/usr/usr_tmp.sfs -b 65536
umount -l /usr
umount -l /squashed/usr/ro
rm /squashed/usr/usr.sfs
mv /squashed/usr/usr_tmp.sfs /squashed/usr/usr.sfs
rm -rf /squashed/usr/rw/*
mount /squashed/usr/ro
mount /usr

Last edited by brando56894 (2010-03-09 16:59:30)

brando56894 · 2010-02-21 01:51:55

edit: found my error

Last edited by brando56894 (2010-02-21 01:57:16)

thoughtcrime · 2010-02-21 13:12:28

Nice, how about writing a full wiki article about that and linking there in the "Maximum performance" article?

rttommy · 2010-02-21 22:22:45

Hi. I'm the one who posted the link and I was going to make a arch-specific script and a standalone wiki page when I had the time, so its great you did it before me. I'll definitely help if I can. The only comment I have is that you should mention that the user shoudn't install/uninstall packages while the image is re/generated, or some weird shit (I'm short on technical terms) will happen.

brando56894 · 2010-02-22 03:13:41

@thoughtcrime: there's not really much to write a wiki article about and also I don't really have a concrete understanding of how it works I just know it should I just followed all the commands on the gentoo version and tried to adapt them as best as possible to arch.

@rttommy: Cool maybe you can write the wiki since I already have the code done? That is if you understand the actual workings of it better than I do, I kind of understand what you mean but I'm not really following you Do you mean don't (un)install packages while the image is in the process of being re-generated?

I haven't really gotten it to work yet, for some reason the squash file doesn't load and all the programs aren't usable. I'm at work in a computer lab at my college so I'm going to try and get it to work within the next few hours.

edit: apparently my problem is with the ubda=reval option for aufs, its mentioned in the gentoo how-to but I cant seem to find anywhere what it does since aufs doesn't have any man pages and i cant seem to find it anywhere. anyone have any idea?

edit 2: I finally got it to work apparently that option doesn't seem to affect anything. My major problem was (another) dumb mistake: once I moved /usr to /usr.old I never re-created /usr so it had nothing to mount to! ::face palm:: I don't seem to notice a speed increase so far, maybe thats because I'm using an SSD and everything is already really fast. I'm sure it'd be a lot more noticeable with a regular HDD.

@rttommy: I added in the warning that you suggested and also added an if statement in the cron job script that will exit if /var/lib/pacman/db.lck exists, it's not the best way to go about it but it was the only thing I could think of ATM, can anyone think of something better? I was thinking of something maybe along the lines of a pop-up box that would say that the job wont run because pacman is in use but IDK how to do that. Also I know that there's a way to send an email when a job fails but IDK how to do that either.

edit 3: since it works perfectly (dare I say so?) I linked to it in the wiki, if one of you guys decides to make an individual wiki for this feel free to use my code, just give me credit for it

Last edited by brando56894 (2010-02-22 06:49:01)

jwwolf · 2010-02-22 11:18:28

#move the /usr folder instead of deleting it
sudo mv /usr /usr.old
sudo mkdir /usr

A little problem.
You can't sudo after you moved sudo.
Got it working though

Last edited by jwwolf (2010-02-22 11:40:26)

draugdel · 2010-02-22 13:20:53

You could touch /var/lib/pacman/db.lck before you start doing something if it doesn´t exist. Don´t forget to remove it after you are done. That way pacman should not be able to run, while you are doing stuff.

brando56894 · 2010-02-22 18:41:55

Good idea, I'll think about that.

jwwolf wrote:

#move the /usr folder instead of deleting it
sudo mv /usr /usr.old
sudo mkdir /usr
A little problem.
You can't sudo after you moved sudo.
Got it working though

Good catch I didn't think of that. I added an if statement that will only continue running the program if the user is root.

edit: I just started using this and I don't think I'm going to continue using it since I tend to install and uninstall stuff a lot and that would mean a lot of image recreations :-/

Last edited by brando56894 (2010-02-22 22:51:27)

icarus-c · 2010-02-23 10:24:11

i think:

#checks to see if the user is root
if [[ $UID != "0" ]];then
    exit

#next

would be more the Arch way rather than covering the whole script with if-fi clause.
Also, in the rebuild script it is not necessary to have an else-clause since it wouldn't be executed once it exits

Last edited by icarus-c (2010-02-23 10:26:22)

brando56894 · 2010-02-23 18:09:35

Thanks I'll change that around right now

icarus-c · 2010-02-24 09:11:05

Besides, is the following really necessary? i seen umount failure during shutdown.

echo "umount -l /usr"
echo "umount -l /squashed/usr/ro"

shouldn't

/bin/umount -a -r -t noramfs,notmpfs,nosysfs,noproc -O no_netdev

do the job already?

brando56894 · 2010-02-24 23:34:44

I guess that would do the job, I just stuck it in because it was in the how-to and I was trying to follow it to the T since I wasn't entirely sure what was necessary and what wasn't. I'll comment it out so if there are problems it can easily be re-enabled if need be.

NuclearGorilla · 2010-03-07 22:19:55

It looks like there's a typo in the remake script--one line refers to usr_tmp.sfs and another to usr_temp.sfs:

mksquashfs /usr /squashed/usr/usr_tmp.sfs -b 65536
...
mv /squashed/usr/usr_temp.sfs /squashed/usr/usr.sfs

I think there might be some use for more error checking, although I'm not sure where yet. I don't know what happened exactly, but when I woke up I didn't have any usr.sfs or usr_tmp.sfs (so nothing in /usr, and nothing to put there). I'm assuming it ran as a cron job, since I don't what else would mess with it.

Once I've finished getting /usr back to normal I'll try this again and see if I can figure out what I did. Cool script, though.

brando56894 · 2010-03-09 17:01:26

thanks, i fixed the typo. Most of this stuff was just copied from the gentoo guide so i didn't actually type most of it. The script itself should work, ive tested it. as for the cron im unsure if it works correctly, im assuming it does.

BlackLotus89 · 2011-04-03 23:25:28

brando56894 wrote:

#!/bin/bash

#This script comes with ABSOLUTELY NO WARRANTY use it at your own risk!

#cron job to rebuild the squashfs image of /usr
if [[ -e /var/lib/pacman/db.lck ]]; then
  exit
fi

mksquashfs /usr /squashed/usr/usr_tmp.sfs -b 65536
umount -l /usr
umount -l /squashed/usr/ro
rm /squashed/usr/usr.sfs
mv /squashed/usr/usr_tmp.sfs /squashed/usr/usr.sfs
rm -rf /squashed/usr/rw/*
mount /squashed/usr/ro
mount /usr

I would add touch /var/lib/pacman/db.lck and rm /var/lib/pacman/db.lck before mksquashfs and after mount to prevent the user from doing anything stupid

mokasin · 2011-05-18 09:40:23

Maybe one should add "|| exit 1" after the mksqushfs command to prevent, that if something went wrong /usr isn't deleted

#!/bin/bash

#This script comes with ABSOLUTELY NO WARRANTY use it at your own risk!

#cron job to rebuild the squashfs image of /usr
if [[ -e /var/lib/pacman/db.lck ]]; then
  exit
fi

mksquashfs /usr /squashed/usr/usr_tmp.sfs -b 65536 || exit 1
umount -l /usr
umount -l /squashed/usr/ro
rm /squashed/usr/usr.sfs
mv /squashed/usr/usr_tmp.sfs /squashed/usr/usr.sfs
rm -rf /squashed/usr/rw/*
mount /squashed/usr/ro
mount /usr

BlackLotus89 · 2011-05-22 14:53:17

#!/bin/bash

#This script comes with ABSOLUTELY NO WARRANTY use it at your own risk!

#cron job to rebuild the squashfs image of /usr
if [[ -e /var/lib/pacman/db.lck ]]; then
  exit
fi

touch /var/lib/pacman/db.lck
mksquashfs /usr /squashed/usr/usr_tmp.sfs -b 65536 || return 1
umount -l /usr
umount -l /squashed/usr/ro
rm /squashed/usr/usr.sfs
mv /squashed/usr/usr_tmp.sfs /squashed/usr/usr.sfs
rm -rf /squashed/usr/rw/*
mount /squashed/usr/ro
mount /usr
rm /var/lib/pacman/db.lck

Is the script I now use.

linux-ka · 2011-05-23 13:53:49

Thats really nice, but I have got an issue with file access. Regarding to the wiki article, there is no need to specify certain mount options to make this squashfs mount accessable to normal users, but I can only use it when being logged in as root. Else I have serious trouble with a jailed user. I tried users and user as fstab mount option, but this leads to even more troubles.

Is there any clue to solve this problem?

EDIT:

-=solved=-

Last edited by linux-ka (2011-05-27 20:44:13)

pogeymanz · 2011-05-25 20:48:37

Has anyone been clever enough to figure out a way to do a diff of some kind so that you don't have to recompress ALL of /usr everytime?

pogeymanz · 2011-05-28 14:33:01

I have a question about how this works.

What happens when we do an upgrade while the compressed /usr is mounted?

Are the upgraded packages going to be there when I reboot? Do I have to recompress /usr after every upgrade? I realize that they at least wont be compressed, but I just don't understand what is happening.

my0pic · 2011-05-29 01:29:36

pogeymanz wrote:

Are the upgraded packages going to be there when I reboot? Do I have to recompress /usr after every upgrade? I realize that they at least wont be compressed, but I just don't understand what is happening.

The directory /squashed/usr/rw is overlayed over /usr so any packages added/removed use this instead of /usr (read only).

Data written is uncompressed and is persistent. For example, you upgrade pacman on a system with a newly compressed /usr and files will need to go into
/usr/bin, /usr/include, /usr/lib and /usr/share
but instead those directories will now be in
/squashed/usr/rw/bin, /squashed/usr/rw/include, /squashed/usr/rw/lib and /squashed/usr/rw/share
When you recompress /usr, these changes are included into the new squashed image. This includes any packages removed from the system as well.

How often you recompress /usr is up to you. After doing pacman -Syu a few times you'll find /squashed/usr/rw starts to fill in size.

kihanos · 2012-08-06 15:54:36

Hello every one,

As you may know, the /lib directory is now a sym-link pointing to /usr/lib (see http://www.archlinux.org/news/the-lib-d … -symlink/). Squashing the /usr directory the way described here causes problems when update the kernel for example (because /usr/lib/modules/ is not mounted during early userspace, the modules for the new kernel are missing even if one had kept the former /usr/lib on the root partition).

Here is a hook for mkinitcpio to mount the squashed /usr directory soon enough to avoid those issues. It has been inspired by the "usr" and the "rootaufs" hooks. Here is the /usr/lib/initcpio/hooks/aufsusr file:

#!/usr/bin/ash
                                                                                       
run_latehook () {                                                                      
    #check if wanted                                                                   
    if [ "x${aufsusr}" = "x" ]; then                                                   
        return                                                                         
    fi                                                                                 
                                                                                       
    #defaults                                                                          
    if [ "${aufsusr}" == "y" ]; then                                                   
        aufsusr="/squashed/usr"                                                        
    fi                                                                                 
    if [ "x${squashedusr}" = "x" ]; then                                               
        squashedusr="$aufsusr/usr.sfs"                                                 
    fi                                                                                 
                                                                                       
    #load modules                                                                      
    /sbin/modprobe aufs                                                                
    if [ $? != 0 ]; then                                                               
        msg ":: aufsusr: Could NOT load aufs module, bailing out"
        launch_interactive_shell
        return
    fi
    /sbin/modprobe loop
    /sbin/modprobe squashfs

    #fs tree
    squashedusr_nr="/new_root/$squashedusr"
    aufs_ro="/new_root/$aufsusr/ro"
    aufs_rw="/new_root/$aufsusr/rw"
    
    #mount "ro":
    msg ":: Mounting squashed ro usr directory..."
    if [ ! -f "$squashedusr_nr" ]; then
        msg ":: aufsusr: $squashedusr does not exist, bailing out"
        launch_interactive_shell
        return
    fi
    mkdir -p "$aufs_ro"
    /bin/mount -t squashfs -o loop,ro "$squashedusr_nr" "$aufs_ro"

    #mount "aufs" 
    msg ":: Mounting aufs usr..."

    mkdir -p "$aufs_rw"
    /bin/mount -t aufs -o br:"$aufs_rw"=ro:"$aufs_ro"=rr usr /new_root/usr

}

# vim: set ft=sh ts=4 sw=4 et:

together with an installation file /usr/lib/initcpio/install/aufsusr

# vim:set ft=sh:

build ()
{
    add_module aufs
    add_module squashfs
    add_module loop
    add_runscript
}

In order to use it, one need to re-mkinitcpio the kernel after having added "aufsusr" and "shutdown" at the end of the hook list in /etc/mkinitcpio.conf
One moreover need to have an aufs-friendly kernel and aufs3 installed, both of which can be found on the AUR.

Now assuming the /squashed/usr/usr.sfs have been build as before and that /squashed/usr/rw and /squashed/usr/ro do exist, one would need to adapt the corresponding menuentry in /boot/grub/grub.cfg by adding the aufsusr option to the linux command.

linux /vmlinuz-linux-aufs_friendly root=[...] ro quiet aufsusr

The hook also allows you to override the /squashed/usr/ by setting aufsusr=/path/to/where/you/wanna/be and to override the path to your squashfs file by adding squashedusr=/path/to/the/squashfs/file.sfs

Finally, remember to add the proper lines to your /etc/fstab file. As explained in the above posts, we are to add two lines to this file:

/squashed/usr/usr.sfs /squashed/usr/ro squashfs loop,ro 0 0
usr /usr aufs br:/squashed/usr/rw=rw:/squashed/usr/ro=rr 0 0

There is though a little something to change. Because the aufs /usr is already mounted by the hook in READ-ONLY mode, we need to change this mode to READ-WRITE. The br option is thus replaced by a mod option (see man aufs for details) and the squashfs line is not needed anymore:

#/squashed/usr/usr.sfs /squashed/usr/ro squashfs loop,ro 0 0
usr /usr aufs mod:/squashed/usr/rw=rw 0 0

I still have a few remarks about it:

When running mkinitcpio, you can have some error looking like "fsck.aufs does not exist". This comes from the fact that aufs is not a real file system. To avoid it, I added a symlink /sbin/fsck.aufs pointing to /bin/true This is a bit coarse and there must be a better way.

Another thing is the udba=reval mount option which we can see in the gentoo script. This option is setting some security against aufs problems when modifying files directly in /squashed/usr/rw/. Keeping the option causes some error, I don't really understand why because this option works when mounting something after boot. The result is: DO NOT MODIFY THE /squashed/usr/rw/ CONTENT.

A last remark will of course be: USING ALL THIS IS YOUR RESPONSABILITY...

I would be thankful if someone has some comments and/or improvements.

EDIT: Added the part about /etc/fstab.

EDIT2: Some weird bug happened at shutdown after a while. Even though having it working back is possible, this whole thing may deserve being deleted...

Last edited by kihanos (2012-08-11 19:36:21)

Arch Linux

#1 2010-02-21 00:11:20

Script to compress /usr with squashfs

#2 2010-02-21 01:51:55

Re: Script to compress /usr with squashfs

#3 2010-02-21 13:12:28

Re: Script to compress /usr with squashfs

#4 2010-02-21 22:22:45

Re: Script to compress /usr with squashfs

#5 2010-02-22 03:13:41

Re: Script to compress /usr with squashfs

#6 2010-02-22 11:18:28

Re: Script to compress /usr with squashfs

#7 2010-02-22 13:20:53

Re: Script to compress /usr with squashfs

#8 2010-02-22 18:41:55

Re: Script to compress /usr with squashfs

#9 2010-02-23 10:24:11

Re: Script to compress /usr with squashfs

#10 2010-02-23 18:09:35

Re: Script to compress /usr with squashfs

#11 2010-02-24 09:11:05

Re: Script to compress /usr with squashfs

#12 2010-02-24 23:34:44

Re: Script to compress /usr with squashfs

#13 2010-03-07 22:19:55

Re: Script to compress /usr with squashfs

#14 2010-03-09 17:01:26

Re: Script to compress /usr with squashfs

#15 2011-04-03 23:25:28

Re: Script to compress /usr with squashfs

#16 2011-05-18 09:40:23

Re: Script to compress /usr with squashfs

#17 2011-05-22 14:53:17

Re: Script to compress /usr with squashfs

#18 2011-05-23 13:53:49

Re: Script to compress /usr with squashfs

#19 2011-05-25 20:48:37

Re: Script to compress /usr with squashfs

#20 2011-05-28 14:33:01

Re: Script to compress /usr with squashfs

#21 2011-05-29 01:29:36

Re: Script to compress /usr with squashfs

#22 2012-08-06 15:54:36

Re: Script to compress /usr with squashfs

Board footer