Mounting the Root File System into RAM

AGT · 2014-05-20 16:39:23

I also found this, meanwhile: https://bbs.archlinux.org/viewtopic.php?id=150544.

Rexilion · 2014-05-20 17:50:20

Union stores it cache in ~/.union however you might be right about that it's not the case. But that would be in contradiction with the fast sync time you are observing.

AGT · 2014-05-20 23:06:48

There is something going on when synchronzing back to disk.
Whatever files I sync, they seem to get corrupted, but not really.
For example, I updated the cpio image from the RAM disk, or I modified /boot/grub/grub.cfg, and when trying to reboot, in each case, the cpio image was either giving errors about some missing files from /run and from other places before running my custom hook, or it was not found, by grub, at all; and with grub.cfg, grub did not find it at startup, displaying a grub> prompt.
But, these files are present on the disk, and do not appear to be corrupted: for example, opening the synced grub.cfg from the disk is the same as the one modified in its original RAM disk location.
I tested synchronizing by manally running /root/Documents/rootfs/unmount-root-fs.sh.

I updated the hook and the unmount script, and the original post. I added the parameters ramdisk_cache_use= "0" or "1" and ramdisk_cache_update= "0" or "1". If /root/Documents/rootfs/unmount-root-fs.sh is run with a 1st argument of "0", it does not sync the cache from the RAM disk to the source root.
Using the cache with unison that was created from a previous boot sped up the bootup, in my case (in a VM), by 30s-60s. The quickest that unison was able to copy was, I think, 3:15s. Maybe even 3:00s.

Last edited by AGT (2014-05-20 23:14:08)

Rexilion · 2014-05-21 06:02:41

Are you properly unmounting afterwards?

If you are getting errors from '/run' then there is something wrong with your cmd as you want to ignore it according to your flags.

Furthermore, you have disabled fastcheck, why?

AGT · 2014-05-21 13:13:46

I tried with and without unmounting, but it seems that the problem is with unison. If I use "cp" for the initramfs, it works (so far), but if I let unison update it, it fails in various ways. If the image is updated (not completely copied), then it is not found; if it is absent from the destination (completely copied), it is found, but it fails to boot -- it starts loading, but then, the screen blanks out, and nothing happens anymore; a hard reset is required.

I'm not sure what you mean by your seconds sentence. /run was just an example. Those errors happened before my hook was run, and it was only happening when the initramfs was corrupt. My flags are not related to ignoring errors.

I left fast checking to Unison's default, as I thought that it may be better to let its developers decide when to use it or not. Currently, it's on for Unix, and off for Windows:

When fastcheck is set to default [...], Unison will use fast checks on Unix replicas and slow checks on Windows replicas.

Rexilion · 2014-05-21 14:01:16

I'm going to post the relevant lines now. This makes the conversation easier:

Initial copy with cp:

elements=$(ls --almost-all --literal --show-control-chars --quote-name --quoting-style=escape -1 --ignore={dev,proc,run,sys,tmp,swap.sys,media,mnt,misc,net,{new,old}[_-]root,"${source#/}","${temporary#/}"} "${source}")

while read element
do
	cp --sparse=always --no-dereference --preserve=all --recursive --one-file-system --update --no-clobber --verbose --target-directory="${temporary}""/" """${source}""/""${element}"""
done

Initial copy with cpio:

find -P -O3 . -xdev -path "./dev" -prune -o -path "./proc" -prune -o -path "./run" -prune -o -path "./sys" -prune -o -path "./tmp" -prune -o -path "./swap.sys" -prune -o -path "./media" -prune -o -path "./mnt" -prune -o -path "./misc" -prune -o -path "./net" -prune -o -path "./var/log/journal" -prune -o -path ".""${source}" -prune -o -path ".""${temporary}" -prune -o -print0 | cpio --pass-through --null --dot --make-directories --unconditional --preserve-modification-time --sparse "${destination}"

Initial copy with rsync:

rsync --archive --inplace --hard-links --acls --xattrs --devices --specials --one-file-system --8-bit-output --human-readable --progress --update --exclude-from="/root/.rsync/exclude.txt" "${source}/" "${temporary}"

Initial copy with unison:

unison "${source}""${unison_cfg_d}" "${unison_cfg_d}" -times -backuploc local -xferbycopying -fastcheck true -batch -force "${source}""${unison_cfg_d}" -confirmbigdel=false -confirmmerge=false -log=false

UNISON="${unison_cfg_d}"

unison "${source}" "${temporary}" -times -backuploc local -xferbycopying -fastcheck "${ram_unison_fastcheck}" -batch -ignore "Path {dev,proc,run,sys,tmp,swap.sys,media,mnt,misc,net,{old,new}[_-]root,var/log/journal,""${source#/}"",""${temporary#/}""}" -force "${source}" -confirmbigdel=false -confirmmerge=false -log=false

unset UNISON

unison "${temporary}""${unison_cfg_d}" "${unison_cfg_d}" -times -backuploc local -xferbycopying -fastcheck true -batch -force "${unison_cfg_d}" -confirmbigdel=false -confirmmerge=false -log=false

Command used to synchronize back:

unison "${source}""${unison}" "${temporary}""${unison}" -times -backuploc local -xferbycopying -fastcheck false -batch -force "${temporary}""${unison}" -confirmbigdel=false -confirmmerge=false -log=false

I'm not surprised the unison method fails. You first sync the metadata to the main root. Then use this metadata to speed up the initial copy of the rest of the system. Then you ditch this data and sync this back to the original location of the first metadata sync. This is way too complicated. Maybe unison get's confused when you use it to sync back.

Furthermore, unison has no perm parameter. Maybe set it to '-1' to make sure it copies all the permissions.

Last edited by Rexilion (2014-05-21 14:01:50)

AGT · 2014-05-23 01:43:45

I think I have found the problem.
Cache was not used, so it should not be considered.
The HDD shows some activity about 25 seconds after syncing back to disk, regardless of the fact if I use unison or rsync (I also tried with rsync).
I have to wait at least 2 minutes after the sync has finished to "allow the environment to settle", so that files do not get corrupted. Maybe it's just my hardware, but the VM seemed the same.

Arch Linux

#26 2014-05-20 16:39:23

Re: Mounting the Root File System into RAM

#27 2014-05-20 17:50:20

Re: Mounting the Root File System into RAM

#28 2014-05-20 23:06:48

Re: Mounting the Root File System into RAM

#29 2014-05-21 06:02:41

Re: Mounting the Root File System into RAM

#30 2014-05-21 13:13:46

Re: Mounting the Root File System into RAM

#31 2014-05-21 14:01:16

Re: Mounting the Root File System into RAM

#32 2014-05-23 01:43:45

Re: Mounting the Root File System into RAM

Board footer