You are not logged in.

#1 2010-10-20 15:29:45

gradgrind
Member
From: Germany
Registered: 2005-10-06
Posts: 921

larch-8 - live medium construction kit, new version

The aim here was to simplify, so several features which I don't use (and don't want to support) have been stripped out, and data persistence is handled in a completely different way, using the medium itself (if it is writeable) as the aufs persistence overlay - so no more 'session-saving',
persistence is now handled in realtime.
If anyone is interested, please test it - feedback welcome (see the website for details and downloads).

Offline

#2 2010-10-24 00:53:26

kwirkie
Member
Registered: 2010-01-09
Posts: 15

Re: larch-8 - live medium construction kit, new version

I've created several Live CDs and have had no issues. (Apart from shebang python/python2 issues when creating CDs on other distros, nothing a quick 'sed' cant fix)

Using live CDs for bespoke, secure, volatile storage only configurations at work. Utilising the root-overlay feature allows creation of CDs with consistent configurations and the latest packages. Having created live CDs before, I can highly recommend larch to anyone needing to create live CDs quickly and VERY easily.

Kudos to gradgrind.

Offline

#3 2010-10-24 15:36:10

gradgrind
Member
From: Germany
Registered: 2005-10-06
Posts: 921

Re: larch-8 - live medium construction kit, new version

Thanks for the positive feedback.

I have made some quite extensive changes to the final stages now, to incorporate larch medium copying facilities. I hope I haven't broken too much in doing this ... it all still needs testing.

The 3 medium-building scripts have now been coalesced into just 1, medium.py - I guess it's easier to use the gui though, anyway. Apart from bugfixes and possible feature requests I'm not planning on changing much now.

Offline

#4 2010-10-26 04:52:25

Akuma
Member
From: France
Registered: 2010-10-26
Posts: 35

Re: larch-8 - live medium construction kit, new version

Hello,

I'm quite new in the Arch world, I totally felt in love with Archlinux and switched to it from (K)ubuntu a month ago, still learning from it everyday big_smile.

Since some days I'm trying to build an installable customised Arch liveCD using larch-8. It's basically Arch + kdebase + some other stuff I need.
I managed to generate the ISO using the GUI starting from the "default" profile, but I encountered some problems because pacman.conf repos of this profile were still set on the previous version of larch, so I had to fix it manually replacing the repo with ftp://ftp.berlios.de/pub/larch/larch8/*platform*, but except this, all was fine, and the LiveCD works perfectly (testing with qemu-kvm).

However, I have a problem concerning the installation of the LiveCD on the hard disk.
First, the GUI installer seems really incomplete, I can only view my partitions but nothing more, all seem to be in read only mode, and no Install button anywhere...
Well no problem, I can use the cli, but it fails after formatting partitions, and I really don't know why nor how to fix it.
Here's the output or larchin-install, note that I also tried with different partition layouts (with only / and swap for example) :

[root@astana user]# fdisk -l
Périphérique Amorce  Début        Fin      Blocs     Id  Système
/dev/sda1   *          63      208844      104391   83  Linux
/dev/sda2          208845      738989      265072+  82  Linux swap / Solaris
/dev/sda3          738990    16113194     7687102+  83  Linux
/dev/sda4        16113195    20964824     2425815   83  Linux
[root@astana user]# larchin-install -l /:/dev/sda3:ext4:,/boot:/dev/sda1:ext2:,/home:/dev/sda4:ext4,swap:/dev/sda2::
#+get-mounts
##/dev/sr0 /.livesys/medium
##/dev/loop0 /tfs/system
##/dev/loop1 /tfs/mods
##/dev/loop0 /.livesys/system                                                                                  
##/dev/loop1 /.livesys/mods
##/dev/sr0 /boot
#-get-mounts
#>Formatting /dev/sda3
#+part-format /dev/sda3 ext4 
[...]
#-part-format
#>Formatting /dev/sda1
#+part-format /dev/sda1 ext2  
[...]
#-part-format
#>Formatting /dev/sda4
#+part-format /dev/sda4 ext4
[...]
#-part-format
#>Mounting /dev/sda3 at /
#>Mounting /dev/sda1 at /boot
#>Mounting /dev/sda4 at /home
#+do-imount /tmp/larchin/install /dev/sda3 /
#-do-imount
#+do-imount /tmp/larchin/install /dev/sda1 /boot
#-do-imount
#+do-imount /tmp/larchin/install /dev/sda4 /home
#-do-imount
#+copy-init
##cat: /.livesys/utype: Aucun fichier ou dossier de ce type (=File not Found)
##ERROR: Couldn't determine type of union file-system
#-copy-init
!>Couldn't initialize file copy process
#+do-unmount /tmp/larchin/install/home
#-do-unmount
#+do-unmount /tmp/larchin/install/boot
#-do-unmount
#+do-unmount /tmp/larchin/install
#-do-unmount

I would like to get some help concerning this issue, thanks in advance.

Anyways thanks for this awesome tool.

(Sorry for the lame english, I'm french smile)

Last edited by Akuma (2010-10-26 04:53:38)


AMD Ryzen 3700X - ASUS ROG STRIX B550 - Sapphire RX 580 - KDE Plasma

Offline

#5 2010-10-26 08:42:02

gradgrind
Member
From: Germany
Registered: 2005-10-06
Posts: 921

Re: larch-8 - live medium construction kit, new version

I'm afraid I haven't updated larchin for larch version 8 yet ...

The gui is, as you say, still in the very early stages, but the command line scripts should still be usable with larch-7 media. larch-7 should itself still work so long as you change the python link /usr/bin/python to point to /usr/bin/python2.

Offline

#6 2010-11-01 17:01:06

firecat53
Member
From: Lake Stevens, WA, USA
Registered: 2007-05-14
Posts: 1,542
Website

Re: larch-8 - live medium construction kit, new version

Hey, I've created the larch installation, larchified and copied to the USB drive (using the larch GUI). On booting, I'm getting a kernel panic after the bootloader screen within just a couple of seconds after starting to boot.

No filesystem could mount root, tried:
Kernel panic - not syncing: VFS Unable to moutn root fs on unknown-block(9,3)

Any idea where I should look for the issue? I didn't mess with mkinitcpio.conf: mkinitcpio.conf.larch and .larch0 both exist. What else should I check?

Thanks!
Scott

Offline

#7 2010-11-02 12:41:26

gradgrind
Member
From: Germany
Registered: 2005-10-06
Posts: 921

Re: larch-8 - live medium construction kit, new version

firecat53 wrote:

Hey, I've created the larch installation, larchified and copied to the USB drive (using the larch GUI). On booting, I'm getting a kernel panic after the bootloader screen within just a couple of seconds after starting to boot.

No filesystem could mount root, tried:
Kernel panic - not syncing: VFS Unable to moutn root fs on unknown-block(9,3)

Any idea where I should look for the issue? I didn't mess with mkinitcpio.conf: mkinitcpio.conf.larch and .larch0 both exist. What else should I check?

Thanks!
Scott

I'm not really an expert on these problems, but I think this sort of thing normally happens after the initramfs code has run, when the root file-system is about to be switched to. In that case there may be some useful information on the screen before this final failure. You can also try booting with the additional 'verbose' parameter. And by the sound of it you are using larch-7 (larch-8 has no mkinitcpio.conf.larch0), so this would be the wrong thread ...

Offline

#8 2010-11-03 15:00:39

hongy19
Member
From: China
Registered: 2005-04-08
Posts: 85
Website

Re: larch-8 - live medium construction kit, new version

I get a aufs2-utils error, but actually I have installed aufs2-util

[Archlinux ~]$ pacman -Q |grep aufs
aufs2-util 20100802-1


##Initializing larchify process
>>rm -f /home/hongy19/larch_build/.larch/system.sqf
>?0
>>rm -rf /home/hongy19/larch_build/.larch/larchify
>?0
>>mkdir -p /home/hongy19/larch_build/.larch/larchify/overlay
>?0
##Seeking kernel information
##Kernel: vmlinuz26   -   version: 2.6.35-ARCH
>>chroot /home/hongy19/larch_build depmod 2.6.35-ARCH
>?0
##Testing for necessary packages and kernel modules
!>ERROR:
Package 'aufs2-util' is needed by larch systems

Offline

#9 2010-11-04 07:02:36

gradgrind
Member
From: Germany
Registered: 2005-10-06
Posts: 921

Re: larch-8 - live medium construction kit, new version

aufs2-util needs to be in the installation which is being larchified, not in the host system. Actually it should be automatically included, I think, but I'll check on that.

Offline

#10 2010-11-05 12:44:10

hongy19
Member
From: China
Registered: 2005-04-08
Posts: 85
Website

Re: larch-8 - live medium construction kit, new version

I change installation dir to /larch_build  and it is solved.

gradgrind wrote:

aufs2-util needs to be in the installation which is being larchified, not in the host system. Actually it should be automatically included, I think, but I'll check on that.

Offline

#11 2010-11-30 20:58:07

birdo
Member
Registered: 2010-11-30
Posts: 7

Re: larch-8 - live medium construction kit, new version

First of all, larch is awesome like arch smile.

Things which still could be improved:
I had a few problems building an i686 arch while running a x86_64 distro (had to copy pacman.conf.options and pacman.conf.repos to the profile path and hardcode "Architecture" and the larch-repo url to i686).
I think the rootpw file is not mentioned in the documentation.

Now I would like to create some "bundles". I thought about installing bigger apps (like texlive), which I do not need regularly to some fake root directory and squash it. Then I could add and remove them easily from the bundles direcory.
I tried to accomplish this with aufs2 and chroot, but aufs does not seem to work as usual on e.g /usr (dmesg writes "unsupported filesystem"). Would this work somehow or what is a good way to create these bundles otherwise?

Thank you
Birdo

Offline

#12 2010-12-01 12:19:22

gradgrind
Member
From: Germany
Registered: 2005-10-06
Posts: 921

Re: larch-8 - live medium construction kit, new version

birdo wrote:

I had a few problems building an i686 arch while running a x86_64 distro...

That doesn't surprise me. I can't test - and therefore not really support - this as I don't have any 64-bit systems.

birdo wrote:

... had to copy pacman.conf.options and pacman.conf.repos to the profile path and hardcode "Architecture" and the larch-repo url to i686.

I would have expected that, but the larch-repo url is a bit of a surprise as there is a symlink from x86_64 on the server.

birdo wrote:

I think the rootpw file is not mentioned in the documentation.

Indeed you're right. I'll put it on my todo list.

birdo wrote:

Now I would like to create some "bundles". I thought about installing bigger apps (like texlive), which I do not need regularly to some fake root directory and squash it. Then I could add and remove them easily from the bundles direcory.
I tried to accomplish this with aufs2 and chroot, but aufs does not seem to work as usual on e.g /usr (dmesg writes "unsupported filesystem"). Would this work somehow or what is a good way to create these bundles otherwise?

Strange, I would expect something like that to work - unless you are trying it on the live system (I believe aufs doen't support aufs branches). It should really work if you use the installation from which the live system was built as the lower layer of the aufs mount.

Offline

#13 2010-12-01 19:13:41

birdo
Member
Registered: 2010-11-30
Posts: 7

Re: larch-8 - live medium construction kit, new version

Hi, thanks for your reply!

gradgrind wrote:
birdo wrote:

... had to copy pacman.conf.options and pacman.conf.repos to the profile path and hardcode "Architecture" and the larch-repo url to i686.

I would have expected that, but the larch-repo url is a bit of a surprise as there is a symlink from x86_64 on the server.

Ok, so I guess it would have also worked if I did not replace the *platform* variable.

gradgrind wrote:

unless you are trying it on the live system

Yes, I tried that. But I finally got it working:
Now I am layering write/mods/system directories from /.livesys over a new directory on /dev/shm.
and so on... I attached the script below.

I think it works quite nice but I have not created too many bundles yet.


#!/bin/sh

if [ $UID != "0" ]; then
  echo "run as root"; exit 1
fi
if [ "$#" != "1" ]; then
  echo "pass name of bundle"; exit 1
fi

# setup fake root 
mkdir /dev/shm/fake_root
mkdir /dev/shm/fake_root_delta
mount -t aufs -o br=/dev/shm/fake_root_delta=rw:/.livesys/write=rr+wh:/.livesys/mods=rr+wh:/.livesys/system=ro none /dev/shm/fake_root

# chroot to fake root
chroot /dev/shm/fake_root

# ...root installs the software, calls exit when done...

# remove noise from delta directory
rm -f /dev/shm/fake_root_delta/etc/ld.so.cache
rm -fR /dev/shm/fake_root_delta/var/cache
rm -fR /dev/shm/fake_root_delta/var/lib/pacman
rm -fR /dev/shm/fake_root_delta/var/log
rm -fR /dev/shm/fake_root_delta/root

# create the bundle and change owner to sudo-ing user
rm -f $1.sqb
mksquashfs /dev/shm/fake_root_delta $1.sqb
chown "${SUDO_UID}:${SUDO_GID}" $1.sqb 

# clean up
umount /dev/shm/fake_root
rm -R /dev/shm/fake_root
rm -R /dev/shm/fake_root_delta

# list result
unsquashfs -l $1.sqb

Offline

#14 2010-12-02 22:12:11

Amnon82
Member
From: Munich
Registered: 2009-05-01
Posts: 46
Website

Re: larch-8 - live medium construction kit, new version

We from the Chakra Project using larch8 now too.All went fine except the last page where I larchify the iso. It comes down to labeling the image.

I got errors to retype my root-pw all the time when I use CL 0.2.85pre x32. I can run the cmd in the terminal just fine:

sudo /opt/apps/larch/cli/medium.py -s "-l" "CL 0.2.85pre x32" "-o" "/home/phil/chakra-0.2.85pre-i686.iso" "-p" "/home/phil/.config/larch/myprofiles/chakra-minimal-testing-i686" "-i" "/home/phil/larch_build"

With CL-0.2.85pre-x32 all is fine. It's a little quirk. Only that you know wink


visit chakra-project.org to get latest Chakra GNU/Linux

Offline

#15 2010-12-03 08:59:47

gradgrind
Member
From: Germany
Registered: 2005-10-06
Posts: 921

Re: larch-8 - live medium construction kit, new version

Amnon82 wrote:

We from the Chakra Project using larch8 now too.All went fine except the last page where I larchify the iso. It comes down to labeling the image.

I got errors to retype my root-pw all the time when I use CL 0.2.85pre x32. I can run the cmd in the terminal just fine:

sudo /opt/apps/larch/cli/medium.py -s "-l" "CL 0.2.85pre x32" "-o" "/home/phil/chakra-0.2.85pre-i686.iso" "-p" "/home/phil/.config/larch/myprofiles/chakra-minimal-testing-i686" "-i" "/home/phil/larch_build"

With CL-0.2.85pre-x32 all is fine. It's a little quirk. Only that you know wink

Thanks for the report, I probably didn't really consider labels with spaces. I'll put it on my todo list.

Offline

#16 2010-12-05 11:25:31

gradgrind
Member
From: Germany
Registered: 2005-10-06
Posts: 921

Re: larch-8 - live medium construction kit, new version

I've looked into that label-with-spaces problem and found a quoting bug, but I decided that labels with spaces are in general not a good thing - consider using them as boot or mount targets (which I do) - so they are now banned (spaces are now converted to underlines) ...

(I hope that doesn't upset chakra too much).

Offline

#17 2010-12-30 01:09:17

manutortosa
Member
From: Reus - Spain
Registered: 2009-03-28
Posts: 25
Website

Re: larch-8 - live medium construction kit, new version

That's more than perfect for us, thanks smile


Manuel Tortosa
manutortosa@chakra-project.org
The Chakra Project

Offline

#18 2010-12-31 12:33:07

farvardin
Member
Registered: 2008-09-03
Posts: 120
Website

Re: larch-8 - live medium construction kit, new version

hello,

I'm trying to use Larch8 for "larchifying" an existing system, but I got an error during the mkinitcpio process:

>::parsing hook [larch]
>generating image '/.larch/larchify/medium/boot/larch.img'...lzma: (stdin): cannot allocate memory
> writing filebug failed
> line 9
etc

before it was working with 500 Mb ram, now I've increased the ram in my virtual machine to 680 Mb, but I still get this error.

Offline

#19 2011-01-01 10:10:50

gradgrind
Member
From: Germany
Registered: 2005-10-06
Posts: 921

Re: larch-8 - live medium construction kit, new version

Rather check the virtual disk space, isn't that where mkinitcpio is trying to do its work? I'm afraid the error message doesn't mean much to me.

Offline

#20 2011-01-02 11:39:42

farvardin
Member
Registered: 2008-09-03
Posts: 120
Website

Re: larch-8 - live medium construction kit, new version

I've increased again the memory (almost 1 Gb), and it's working now.
I don't have swap space, maybe I could configure that too (but I had enough virtual disk space, both on the virtual disk, and on my physical drive)

Offline

#21 2011-01-02 12:51:13

gradgrind
Member
From: Germany
Registered: 2005-10-06
Posts: 921

Re: larch-8 - live medium construction kit, new version

farvardin wrote:

I've increased again the memory (almost 1 Gb), and it's working now.
I don't have swap space, maybe I could configure that too (but I had enough virtual disk space, both on the virtual disk, and on my physical drive)

That seems like an awful lot to me, does mkinitcpio really need so much memory?

Offline

#22 2011-02-27 17:52:47

systemshq
Member
Registered: 2011-01-06
Posts: 37

Re: larch-8 - live medium construction kit, new version

I'm a new larch user and I want to make a live usb stick from my current installation. The documentation says:-

"Making a live system from an existing Arch installation

By setting the installation path (with the '-i' option if using the command line) to an existing Arch installation, a live medium can be made from it. One convenient way of doing this is to run the build process from a live system. A larch live system is of course particularly well suited, but most others should work too. "

So is:-

./larch-archin -i /
./larch-larchify -i /
./larch-medium -i /
./larch-medium -i /

Within an existing installation which is currently running correct?
Many thanks

Offline

#23 2011-02-28 08:19:34

gradgrind
Member
From: Germany
Registered: 2005-10-06
Posts: 921

Re: larch-8 - live medium construction kit, new version

Basically yes, but do check the various options (by first running each script with the -h option) and be aware that this approach hasn't been tested recently, as I don't recommend it. Ideally one should tidy up the installation first (and if there are many data files move these away - you probably don't want these in the compressed file-system) and then larchify from a live medium.

Offline

#24 2011-03-02 18:55:07

systemshq
Member
Registered: 2011-01-06
Posts: 37

Re: larch-8 - live medium construction kit, new version

Many thanks for your answer above. Basically what I want to do is create a VirtualBox arch system and when I'm happy with it, use larch to make a live dvd/usb system form it. So the procedure would be:-

Create virtualbox arch linux guest
Shutdown guest and use vdfuse to mount the virtualbox guest root partition within the host
Use larch to make a live dvd/usb system from my virtualbox guest system on the mounted virtualbox disk image

Many thanks for your advice again

Offline

#25 2011-03-03 04:23:12

kyle_white
Member
Registered: 2009-09-15
Posts: 13

Re: larch-8 - live medium construction kit, new version

Thanks for all the work on larch, it's become my portable operating system of choice (as arch is by far my favorite distro) and is what I use to install new systems. Before larch, I used puppy linux on occasion (also a great distro) and really liked the option to run the system with a tmpfs layer on top and the option to copy down the top layer to a non-volatile overlay at runtime or shutdown.

I've tactlessly hacked the script they use to work with larch and have been successfully using it for several days. By default I boot with 'nw' to force the top layer to be a tmpfs and then run the script if I make changes I want to keep. Currently the script assumes that the aufs layers are (bottom to top) /.livesys/system /.livesys/overlay /.livesys/write. It should also work with bundles, as it would simply merge down the new files and the whiteouts.

Most people probably wouldn't find a use for it; but it's small and simple and could easily be added to one of the larch packages when it's more mature.

When I have some more free time I'll clean it up and I'll fix bugs as I hit them. I'll post the current code if anyone would like to use or expand on it.

Creds to Barry Kauler, shinobar, Dougal, Jemimah for pupsnapmerge

Edit 3/9/11 - I've updated the script, largely cleaning up and fixing small issues with filenames with spaces (the bane of bash scripts). I've been running it frequently over the past several days and only had a few problems. I wouldn't use it on a system with important files you want to keep, but

#!/bin/bash
#2007 Lesser GPL licence v2 (http://www.fsf.org/licensing/licenses/lgpl.html)
#based on 110208 snapmergepuppy - credit Barry Kauler (www.puppylinux.com), Shinobar, Dougal, Jemimah
#110301 Kyle: modified, refactored for larch 8.2, code cleanup

VERBOSE="yes"

RESERVESPACE=$(( 5 * 1024 ))                                        #Kyle: how much space to leave free in kbytes
FREESPACETHRESHOLD=$(( 50 * 1024 ))                                 #Kyle: threshold of free space in kbytes before cutting out less necessary directories from merge down

SKIPDIRS='^mnt|^initrd|^proc|^sys|^tmp|^root/tmp|^dev/\.|^dev/fd|^dev/pts|^dev/shm|^var/run'
EXTRASKIPDIRS='mozilla|printjobs|archive|cache|^var/log|^var/cache|/\.thumbnails'  #Kyle: large directories which can be excluded to save space if needed
SKIPFILES='\.thumbnails|trash|\.cache$|\.part$|\.crdownload$'

ERRORDIRS='gvfs-metadata'                                           #Kyle: directories or files which cause hangs or bugs, will be excluded
ERRORFILES='\.config/dconf/user'                                    #Kyle: files which cause hangs or bugs, will be excluded

FLUSH="yes"                                                         #Kyle: move (delete after copy)

PATH="/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin"
WD="`pwd`"
SRC="/.livesys/write"
DST="/.livesys/overlay"
BASE="/.livesys/system"

[ -s /etc/merge-down.conf ] && . /etc/merge-down.conf

SRCISNFS="no"; [ "$(stat -f -c %T "${SRC}")" = "nfs" ] && SRCISNFS="yes"

SHUTDOWN="no"; case `ps` in *rc.shutdown*) SHUTDOWN="yes" ;; esac
XRUNNING="no"; [ -n "`pgrep Xorg`" ] && XRUNNING="yes"
 
#v3.02 first time, do not use '-u' option when saving, save everything... Kyle: Unlikely to happen in larch, but couldn't hurt to leave it
NEXTPASSTHRU=""; [ -e /tmp/flagnextpassthru ] && NEXTPASSTHRU="yes"

[ "${SHUTDOWN}" = "no" -a "${XRUNNING}" = "yes" ] && notify-send 'START' " Beginning merge." -t 5000 -u normal -i dialog-ok

echo "Remount aufs / udba=notify..."
mount -o remount,udba=notify / #remount aufs with best evaluation mode.
[ $? -ne 0 ] && echo "Failed to remount aufs / with udba=notify"

echo "Remount $DST rw..."
mount -o remount,rw ${DST}
[ $? -ne 0 ] && echo "Could not remount ${DST} rw" && exit

cd ${SRC} || exit 1

echo "Merging ${SRC} onto $DST..."



#Handle Whiteouts...
find . -mount \( -regex '.*/\.wh\.[^/]*' -type f \) | sed -e 's/\.\///;s/\.wh\.//' | grep -v -E "${SKIPDIRS}|${ERRORDIRS}" | grep -v -E -i "${SKIPFILES}|${ERRORFILES}" | while read filename; do
 basename=${filename##*/}
 dirname=${filename%/*}

 if [ "${dirname}" = "." ] || [ "${basename}" = ".wh.aufs" ] ; then
  #aufs has file .wh..wh.aufs in /initrd/pup_rw.
  continue
 elif [ "${basename}" = ".wh..opq" ] ; then
  #.wh..opq marks ignore all contents in lower layers (actually .wh..wh..opq before stripping .wh.)...
  rm -rf "${DST}/${dirname}" 2>/dev/null  #wipe anything in save layer.
     [ "${VERBOSE}" == "yes" ] && echo rm -rf \"${DST}/${dirname}\"
  mkdir "${DST}/${dirname}" #jemimah: files sometimes mysteriously reappear if you don't delete and recreate the directory, aufs bug?
     [ "${VERBOSE}" == "yes" ] && echo mkdir \"${DST}/${dirname}\"
  #also need to save the whiteout file to block all lower layers (may be readonly)...
  touch "${DST}/${dirname}/.wh..wh..opq" 2>/dev/null 
     [ "${VERBOSE}" == "yes" ] && echo touch \"${DST}/${dirname}/.wh..wh..opq\"
  rm -f "${SRC}/${dirname}/.wh..wh..opq"  #should force aufs layer "reval".
     [ "${VERBOSE}" == "yes" ] && echo rm -f \"${SRC}/${dirname}/.wh..wh..opq\"
  continue
 else
  #Kyle: if file doesn't exist in lower levels, remove whiteout file. should force aufs layer "reval".
  [ ! -e "${BASE}/`echo "${filename}" | sed -e 's/\.\///;s/\.wh\.//'`" ] && rm -f "${SRC}/${dirname}/.wh.${basename}" && 
     [ "${VERBOSE}" == "yes" ] && echo rm -f \"${SRC}/${dirname}/.wh.${basename}\"
  #comes in here with the '.wh.' prefix stripped off, leaving actual filename...
  rm -rf "${DST}/${filename}"
     [ "${VERBOSE}" == "yes" ] && echo rm -rf \"${DST}/${filename}\"
 fi

done



#calculate free space in overlay after removing whited out files
freespacek=`df -k | grep "$DST" | tr -s ' ' | cut -f 4 -d ' '`

###skip more directories if space is low
[ ${freespacek} -lt ${FREESPACETHRESHOLD} ] && SKIPDIRS="${SKIPDIRS}|${EXTRASKIPDIRS}" &&
   [ "${SHUTDOWN}" = "no" -a "${XRUNNING}" = "yes" ] && notify-send 'WARNING!' " Free space below ${FREESPACETHRESHOLD}k. Excluding nonessential directories." -t 10000 -u normal -i dialog-warning

#Build directory tree
find . -mount -type d | tail -n +2 | sed -e 's/\.\///' | grep -v -E "${SKIPDIRS}|${ERRORDIRS}" | while read filename; do
 [ ! -d "${DST}/${filename}" ] && mkdir -p "${DST}/${filename}" &&
    [ "${VERBOSE}" == "yes" ] && echo mkdir -p \"${DST}/${filename}\"
 chmod "${DST}/${filename}" --reference="${filename}"
    [ "${VERBOSE}" == "yes" ] && echo chmod \"${DST}/${filename}\" --reference=\"${filename}\"
 Command=$(stat --format="chown %U \"${DST}/${filename}\" ; chgrp %G \"${DST}/${filename}\"" "${filename}")
 eval "$Command"
    [ "${VERBOSE}" == "yes" ] && echo eval \"$Command\"
 touch "${DST}/${filename}" --reference="${filename}"
    [ "${VERBOSE}" == "yes" ] && echo touch \"${DST}/${filename}\" --reference=\"${filename}\"
done

##v2.21 -u causes trouble if timezone malconfigured...
##[ -L "${DST}/${filename}" ] && rm -f "${DST}/${filename}"
##cp -a -u -f "${filename}" "${DST}/${filename}"
#cp -a --remove-destination "${filename}" "${DST}/${filename}"
CPOPTS="-a -f"
[ "$NEXTPASSTHRU" = "yes" ] && CPOPTS="${CPOPTS} -u"  #link, or first run since reboot should not include '-u' (update) option.

#Copy Files
find . -mount -not -type d |  sed -e 's/\.\///' | grep -v -E "${SKIPDIRS}|${ERRORDIRS}" | grep -v -E -i "${SKIPFILES}|${ERRORFILES}|$0" | while read filename; do

 basename=${filename##*/}
 dirname=${filename%/*}
 
 #stop saving if not enough room left in ${DISTRO_FILE_PREFIX}save file...
 filesizeb=`eval "stat --format=%s \"${filename}\""`
 [ -z "${filesizeb}" ] && break
 filesizek=$(((filesizeb/1024)+1)) #Kyle: round up
 if [ $((${filesizek}+${RESERVESPACE})) -gt $freespacek ];then
   freespacek=`df -k | grep "${DST}" | tr -s ' ' | cut -f 4 -d ' '` #reevaluate free space only if close
   if [ $((filesizek+${RESERVESPACE})) -gt $freespacek ];then
    [ "${SHUTDOWN}" = "no" -a "${XRUNNING}" = "yes" ] && notify-send 'ERROR!' " Unable to save all files. You need to delete some." -t 10000 -u critical -i dialog-error
    break
   fi
 else
  freespacek=$((freespacek-filesizek)) #jemimah: keep track of the worst case
 fi
    [ "${VERBOSE}" == "yes" ] && echo freespacek=${freespacek}


 [ -L "$DST/${filename}" ] && rm -f "$DST/${filename}" &&
    [ "${VERBOSE}" == "yes" ] && echo rm -f \"${DST}/${filename}\"
 cp ${CPOPTS} "${filename}" "${DST}/${filename}"
    [ "${VERBOSE}" == "yes" ] && echo cp ${CPOPTS} \"${filename}\" \"${DST}/${filename}\"
 
 [ -e "${DST}/${dirname}/.wh.${basename}" ] && [ ! -e "${dirname}/.wh.${basename}" ] && rm "${DST}/${dirname}/.wh.${basename}" #jemimah bugfix - I/O errors if you don't do this
 
 #if set to flush, remove the file being copied from the source
 NOFLUSH="\.wh\.\.wh\.aufs"
 [ "${FLUSH}" == "yes" ] && [ ! "$( echo "${filename}" | grep -v -E -i "${NOFLUSH}" )" == "" ] && rm "${filename}" &&
    [ "${VERBOSE}" == "yes" ] && echo rm \"${filename}\"
 
done



##Kyle: clean up whiteouts - go into destination directory, check whiteouts and delete those which whiteout non-existant files. Shouldn't be needed if merging down is clean
cd $DST
find . -mount \( -regex '.*/\.wh\.[^/]*' -type f \) | grep -v -E "\.wh\.\.opq|\.wh\.aufs"  | while read filename; do
 [ ! -e "${BASE}/`echo "${filename}" | sed -e 's/\.\///;s/\.wh\.//'`" ] && rm -f "${filename}" &&
    [ "${VERBOSE}" == "yes" ] && echo rm -f \"${filename}\"
done



cd "$WD"

sync

echo "Remount ${DST} ro..."
mount -o remount,ro ${DST}
[ $? -ne 0 ] && echo "WARNING! Could not remount ${DST} ro"

echo "Remount aufs / udba=reval..."
mount -o remount,udba=reval / #remount with faster evaluation mode.
[ $? -ne 0 ] && echo "Failed to remount aufs / with udba=reval"


touch /tmp/flagnextpassthru

[ "${SHUTDOWN}" = "no" -a "${XRUNNING}" = "yes" ] && notify-send 'SUCCESS!' " Files merged into overlay." -t 5000 -u normal -i dialog-ok

exit 0

Last edited by kyle_white (2011-04-21 06:25:22)

Offline

Board footer

Powered by FluxBB