You are not logged in.
UPDATE (6/10/10): http://bbs.archlinux.org/viewtopic.php? … 51#p772651
THIS POST DEPRECATED UNTIL I UPDATE IT, USE THE UPDATED POST ABOVE
available:
http://aur.archlinux.org/packages.php?ID=33376
this package will install a mkinitcpio hook intended for those who wish to have a single or multi-drive BTRFS file system as their / (root). the hook will ensure that the chosen root device from the kernel command line is intact and safe to boot. if root is not a BTRFS device, the hook is quietly skipped.
it requires btrfs-progs from here:
http://aur.archlinux.org/packages.php?ID=15635
hook supports both UUID or device based root names. UUID is safer; the entire BTRFS array only has one UUID, so referring to it means no problems. use /dev/device/by-uuid on kernel boot.
hook supports hot spares by using a dummy, spare BTRFS file system to steal devices from as needed. the spare pool can be referenced by label or UUID.**
hook supports automatic repairing of the array if possible, automatic dropping of missing/failed devices if possible, and booting the array to a degraded state (one or more devices missing) if possible.**
**NOTE: there are some bugs that may be in the hook/btrfs module/btrfs-progs that prevent raid1 (maybe others) arrays from being repaired (see http://pastebin.com/m5706f47f) however, this hook will work fine as is for single/multi-device arrays that are intact. enable experimental options manually by editing options in the hook itself (/lib/initcpio/hooks/btrfs). USE WITH CAUTION ON MULTI-DEVICE SETUPS.
the initramfs script can be seen here:
http://aur.archlinux.org/packages/mkini … btrfs_hook
its fairly commented, although there are a few things i'll point out for others who want to edit:
1) i plan to rework _btrfs_scandev() to be more flexible in the variables it produces. that function simply parses "btrfs-show" and gleans valuable information from it. it will be used to verify array states with certainty.
2) seems that btrfsctl does not have meaningful exit codes right now, do not rely, see _btrfs_register().
3) seems that after legitimately removing a device from an array (NOT using "missing" special keyword), the mount needs a "mount -o remout [mnt]" for the ***missing flag to be removed from btrfs-show. not sure the ramification of this or if its necessary.
4) some or all of this functionality may someday be provided by module/kernel
needs work + comments
add "btrfs" to HOOKS in mkinitcpio.conf
Last edited by extofme (2010-06-10 06:09:10)
what am i but an extension of you?
Offline
You have a lot of stuff going on in your script compared to mine. I don't use UUID, but yours definitely looks more professional:
hooks/btrfs:
# vim: set ft=sh:
run_hook ()
{
# A rather unimpressive hack for making striped btrfs available in the initrd. By gonX - gonX@overclocked.net
# Version 0.25 beta - changelog: Removed most debug hacks.
/sbin/modprobe -q btrfs
/sbin/btrfsctl -a
}
install/btrfs:
# vim: set ft=sh:
install ()
{
MODULES="btrfs libcrc32c crc32c zlib_deflate"
BINARIES="/sbin/btrfsctl /sbin/btrfsck"
FILES=""
SCRIPT="btrfs"
}
help ()
{
cat<<HELPEOF
No setup should be needed here. This just runs btrfsctl -a on boot.
This is very hacky - if you run a custom kernel with the modules this script wants integrated please disregard the error messages mkinitcpio pops up.
btrfsck is included for your convenience. To actually use it, you need to break initrd first.
HELPEOF
}
since 2009
Offline
UPDATE:
- complete rewrite with newer btrfs tools. REQUIRES GIT PROGS UNTIL FURTHER NOTICE
- added rollback support
- added snapshot utility for use with rollback support
QUICK:
1) install the hook
2) create a snapshot:
$ sudo rollback_snapshot
3) do some crazy dangerous stuff
4) reboot
5) press a key when it says "PRESS ANY KEY TO ROLLBACK..."
6) choose a snapshot
7) enjoy...
NOTES:
1) until 2.6.34, rollbacks are VOLATILE (unless someone rewrites it). this means that when you're in "rollback" mode, anything you do/save/create/whatever may/will potentially be lost. it's meant to give you a working environment so you can mount your real system and fix it (chroot into it/whatever). once 2.6.34 rolls around this can be changed (we need the ability to list subvolumes and set a new default, and we need to discuss the right way to implement this permanently)
HOW IT WORKS:
rollback_snapshot creates a snapshot (named by time created) of / and puts it in /snapshots. on boot, if you enter rollback mode, you are given the list of snapshots from /snapshots. after one is chosen, a copy of it is made to /__rollback__, IT IS THIS SNAPSHOT THAT YOU ARE BOOTED INTO, NOT THE SNAPSHOT YOU CHOSE. this protects your snapshots. once you boot to a good environment you can repair you're real installation. do not save work to this "rollback" snapshot; it is removed before every rollback and replaced with the new selection.
once 2.6.34 hits the official repos, i will update the hook to use the real subvolume listing (btrfs subvolume list), this will allow us to have USABLE snapshots (easier to implement, and much cleaner, that why i'm waiting), the ioctl needed is not in the 2.6.33 kernel.
i will be adding automatic repair to the hook again soon, but actually operational and safe this time
Last edited by extofme (2010-04-28 22:53:03)
what am i but an extension of you?
Offline
Hi,
I followed your discussion on aur-general and was motivated by your experience with btrfs.
I already waited so long, I immediatly went ahead and dropped (well, put aside) lvm2 and moved my root partition to btrfs.
After minor problems with the mkinitcpio (lvm hook seems to conflict with the btrfs one) image and fstab (why do I have to specify the root fs there? its already mounted when fstab is examined) I am now running btrfs with your hooks, everything seems fine.
But I'm not sure (newbie to btrfs) whether I understand all the stuff happening there. Creating snapshots is a piece of cake, using your script (rollback_snapshot) or directly using btrfs/btrfsctl. Booting into a snapshot, your hooks do it all.
But what happens if I really screw it up? I'm on 2.6.33 (.2-1) and read about missing ioctl()s regarding btrfs, meaning I can't replace the "default" subvolume (whats the difference between a subvolume and a snapshot anyway?). Its also not possible to list available subvolumes/snpshots, meaning your script (or organizing snapshots in similar ways) is the only chance to keep track of them, right?
I guess a plain "mount -t btrfs /dev/xyz" will mount the "default" subvolume then. Once I really screw up I will have to boot into a snapshot, mount the default subvolume somewhere and restore it using a full copy from the snapshot (a la rsync --delete), then reboot and I'm back at the start again.
Is this correct? Is there some way to permanently change the subvolume mounted by your hook (i.e. not running in a volatile snapshot, but actually drop the default one and always boot into a given snapshot until further notice - a new snapshot)?
Btw, there are 2 reasons I can't wait for btrfs (seems stable enough for me):
1.) I have an archlinux installation at work where I have some (more or less) unsupported hardware (udlfb graphics card). No lvm2 there, I forgot about it when installing arch. Before every -Syu I have to check for kernel/xorg updates, read about them, check the mailing list, wait ... wait ... wait ... loose patience, give it a shot, recompile/reinstall the necessary stuff (at least I've got PKGBUILDs prepared) and hope for the best. With btrfs, this is just a non-issue. Or at least will be once the integration is better.
Oh, and I just need a running system there, none of my work is actually stored locally.
2.) It just "feels right" compared to lvm2 (and I know this is highly subjective). No need to care about PVs, LVs and VGs. One tool to rule them all.
Well, a 3rd reason popped up when learning about btrfs. It seems the ENOSPC / full disk situations are not handled well yet. There seem to be several reasons (mostly RAID configurations). But I probably don't have to care about that. Most of my systems have like 5 to 10 times the diskspace I usually need, so thats not a big deal right now. And the others can wait.
Offline
Nice
This is just amazing! Pleas keep up the good work. I think I'll wait the month or so until 2.6.34 is out and then I'll be using Btrfs until a better file-system is made. These "Copy on Write" B-Tree file-systems are so amazing. I've been waiting for Btrfs to be done for years now.
This ability to roll-back to a older snapshot will solve 90% of the problems with using Linux. (Though Arch Linux is SUPER Stable in my experience. Way more stable then an Ubuntu that is always kept up-to-date. I am sure if one were to stick to long-term releases there would be far less problems.) Most of the problems my clients or I have with Linux is updates braking things. Sure, I can just reinstall the old package, but that doesn't always fix the problem, and this is harder to do on remote client systems. However, if I can just tell my clients to reboot into a snapshot "problem solved." Then I can spend the time to solve the problem, without the client siting around with a broken system.
Did you know Fedora 13 supports Btrfs Snapshotting and Roll-Back. Super cool. This is really going to put Linux over the top and make it far more attractive, and easier to use for people just switching from say.. Windoz. I know when I first switched I had loads of problems and was reinstalling like every week. Though partly just for fun
Go Oracle for starting this GPL project.
Boy and not to mention the SSD support and zlib compression and software RAID options and HOT file-system Grow/Shrink and Hot file-system fixing and Hot file-system moving... A new era for Linux for sure.
Last edited by hunterthomson (2010-05-12 12:36:19)
OpenBSD-current Thinkpad X230, i7-3520M, 16GB CL9 Kingston, Samsung 830 256GB
Contributor: linux-grsec
Offline
bleh, sorry about the late response, i usually subscribe to topics i create/join, i must have missed this one.
After minor problems with the mkinitcpio (lvm hook seems to conflict with the btrfs one) image and fstab (why do I have to specify the root fs there? its already mounted when fstab is examined)
i could be mistaken here, but initially your / is mounted read-only. during the init process, one of the primary scripts issues a "mount -o remount,rw /" to remount the root device as read-write. i _think_ using the mount command in this way (against a mount point and not a device) requires the mount point to exist in the fstab.
But what happens if I really screw it up? I'm on 2.6.33 (.2-1) and read about missing ioctl()s regarding btrfs, meaning I can't replace the "default" subvolume (whats the difference between a subvolume and a snapshot anyway?). Its also not possible to list available subvolumes/snpshots, meaning your script (or organizing snapshots in similar ways) is the only chance to keep track of them, right?
the inability to set the default subvolume is kind of a non-issue after i thought about it more, and some of the discussion on arch-general. i plan to update the hook soon to allow the "promotion" of the rollback to become the new active subvolume. ill do this by snapshotting your current / to a new subvolume called /__active, and then permanently booting into that [__active] instead of / or "." (a period is the default subvolume, like current directory).
there is no difference internally, whatsoever (AFIAK), between a subvolume and a snapshot. its just a matter of context: a subvolume starts out empty (explicitly created via a command/etc.), and a snapshot starts out with content (COW operation against an existing subvolume, i.e. "copy")
yeah without the ability to list available subvolumes, we have to make up some structure, and assume everything goes well. my script simply assumes anything within /snapshots is indeed a snapshot that can be copied (to /__rollback) and booted. even if we can list subvolumes, we'll still need some kind of predefined structure (so the script knows which snapshots are bootable, and which are not, i.e. snapshots of /home are not bootable).
I guess a plain "mount -t btrfs /dev/xyz" will mount the "default" subvolume then.
correct.
Once I really screw up I will have to boot into a snapshot, mount the default subvolume somewhere and restore it using a full copy from the snapshot (a la rsync --delete), then reboot and I'm back at the start again.
Is this correct?
unfortunately, without modifying the script, a manual copy is necessary :-( totally defeating the advantages of btrfs. so, yes. the problem is that the default subvolume itself cannot be replace by another... note this is different from SETTING the default subvolume. i'm not sure if the real default subvolume ("." the period) will ever be replaceable/removable, i'd have to ask the btrfs list. this is why i advocate installing your system into a subvolume FROM THE BEGINNING. this gives us more flexibility.
Is there some way to permanently change the subvolume mounted by your hook (i.e. not running in a volatile snapshot, but actually drop the default one and always boot into a given snapshot until further notice - a new snapshot)?
this is actually how i intend to implement non-volatile rollback abilities in the hook. instead of booting the default "." subvolume, my hook will ALWAYS boot the subvolume called "__active", located at /__active. this way you can put whatever you want there, and it will be booted at the next reboot. therefore, you can boot into rollback mode (essentially booting into /__rollback), update/add/remove whatever you want, and when your satisfied, you simply snapshot /__rollback to /__active and your done. reboot, and your new __active will be in the same state as the point it was created in rollback mode.
does all this make sense? i'm told i tend to over explain/complicate things :-).
Before every -Syu I have to check for kernel/xorg updates, read about them, check the mailing list, wait ... wait ... wait ... loose patience, give it a shot, recompile/reinstall the necessary stuff (at least I've got PKGBUILDs prepared) and hope for the best. With btrfs, this is just a non-issue. Or at least will be once the integration is better.
i hear ya. i like the idea of riding the edge without the fear of being cut by it. unfortunately, until btrfs is supported by grub2, kernel rollbacks aren't possible (you'd have to manually backup your old kernel+initrd). something to keep in mind.
However, if I can just tell my clients to reboot into a snapshot "problem solved." Then I can spend the time to solve the problem, without the client siting around with a broken system.
bingo. that is pretty much my primary motivation here... less time freaking out about down time/whatever; everybody wins.
Did you know Fedora 13 supports Btrfs Snapshotting and Roll-Back. Super cool.
i know they have some support but i haven't tried it; from what i've read, the support is pretty weak... you can get yum to create the snapshot by installing a package, but there is no utility to actually use them later on. i could be mistaken as i don't use fedora, this is only what i've read.
what sucks is pacman doesnt have any ability to use hooks... i dont think we will see automatic snapshots by the package manager without a wrapper like clyde/powerpill/packer/etc.
C Anthony
what am i but an extension of you?
Offline
what sucks is pacman doesnt have any ability to use hooks... i dont think we will see automatic snapshots by the package manager without a wrapper like clyde/powerpill/packer/etc.
pacman hooks are on the roadmap. work on it has already been done, i'm not sure about the current status.
note that clyde is not just a wrapper. it is a frontend which implements libalpm so I think it would be trivial to implement a hooks system in it.
< Daenyth> and he works prolifically
4 8 15 16 23 42
Offline
pacman hooks are on the roadmap. work on it has already been done, i'm not sure about the current status.
The work done so far is limited to some sort of design document: http://wiki.archlinux.org/index.php/Use … cman_Hooks
note that clyde is not just a wrapper. it is a frontend which implements libalpm so I think it would be trivial to implement a hooks system in it.
Given it is a true libalpm frontend, it would be much more useful to just implement it in libalpm. That way we can all have it!
Offline
Dieter@be wrote:note that clyde is not just a wrapper. it is a frontend which implements libalpm so I think it would be trivial to implement a hooks system in it.
Given it is a true libalpm frontend, it would be much more useful to just implement it in libalpm. That way we can all have it!
that's true. my point was just "it's not impossible"
< Daenyth> and he works prolifically
4 8 15 16 23 42
Offline
Hay hay, 2.6.34 is out on kernel.org.
I can't wait until Arch gets it in the repo and this mkinitcpio hook implements the new snapshotting support stuff.
Ya, for sure Fedora13 can't have that fantastic of a snapshotting implementation because they are not building it off of the 2.6.34 kernel, as far as I know (They did put a roll-back feature integrated into the GRUB menu though). I also remember something about a Btrfs roll-back "Slider" the guys on the LinuxActionShow were talking about...?... In any case I bet the Fedora guys will come up will some cool ways of doing things that will be worth checking out in the future. Probably by Fedora14 they will really have things rocking.
Mostly, I just think it is cool that big distros are already using it. Btrfs is really looking like it is going to be the new defacto Linux file-system. Heck even Meego is using it. + the Btrfs dev's are saying that is is stable enough to use now and they are making every effort to keep it stable form now on. I figure sens I'm not going to be doing any advanced stuff like RAID or multi drive implementations, I should have little problem. As always though, I'll keep my tar backup up-to date and I have all my doc's on a SD card.
Last edited by hunterthomson (2010-05-17 08:32:09)
OpenBSD-current Thinkpad X230, i7-3520M, 16GB CL9 Kingston, Samsung 830 256GB
Contributor: linux-grsec
Offline
edit: shit sorry. never mind. i thought this was the testing builds topic
keep on talking about btrfs here! nothing happened
Last edited by Dieter@be (2010-05-17 20:29:23)
< Daenyth> and he works prolifically
4 8 15 16 23 42
Offline
UPDATE!!!
i finally got a break from coding all the other nonsense that people actually pay me for :-P, so i decided to work on this...
non-volatile rollback is now supported!!! hooray!!!
/snapshots renamed to /__snapshot for consistency, MOVE YOUR STUFF MANUALLY
i did a significant amount of cleaning/adding this round. report any bugs!
after installation, reboot and read to enable true rollback support! yay!
NOTE: the rollback_snapshot utility has been removed. i am working on a new utility based off my vps-lxc-git tools (nice template for bash apps). after enabling non-volatile rollbacks, the /__snapshot folder is NOT IN YOUR ROOT... it is technically "under" your root. if you want to play with it before i release the next update (soon), you can manually mount the your btrfs device somewhere, and snapshot __active:
btrfs subvolume snapshot <mnt>/__active <mnt>/__snapshot/<name_of_snapshot>
i have some updates coming to take advantage of 2.6.34 features, and speed up the hook. see FIXME's in the hook for details.
more info to come... so tired/late right now, and it's my turn to get up w/the baby....
ENJOY!!!!
C Anthony
what am i but an extension of you?
Offline
awesome. btw, you are subscribed to arch-releng right? I think i will discuss a roadmap (which includes btrfs) there.
< Daenyth> and he works prolifically
4 8 15 16 23 42
Offline
awesome. btw, you are subscribed to arch-releng right? I think i will discuss a roadmap (which includes btrfs) there.
I was only subscribed to arch-general, but I just added arch-releng. I'll be sure to keep an eye out for your discussion there and chime in with details/answers. Good stuff!
I forgot to mention how to actually rollback. I'm on a mobile right now so I'll update in a bit (I might also need to make a small change to the hook, I just thought of a potential problem... just a small tweak tho :-)
C Anthony
what am i but an extension of you?
Offline
ok, no patching was needed!
to truly rollback, all you need to do is mount your btrfs device somewhere, rename/move __active to __active_old (you can't remove it because your system is running within it, any changes made to __active are immediately visible in your /, try it out), and snapshot either __rollback (not useful unless your current system is running in rollback mode) or a saved snapshot from the __snapshot folder:
# mount <btrfs_device> /mnt
# mv /mnt/__active /mnt/__active_old
# btrfs subvolume snapshot /mnt/__snapshot/<snapshot_x_y_z> /mnt/__active
on the next reboot, your primary system will be in whatever state <snapshot_x_y_z> was in when it was taken! that's all there is to it.
i will soon include a utility to perform these operations for you.
what am i but an extension of you?
Offline
Hi all
i just try out the new hook and got a message at boot time. the problem is, my input device isn't loaded at this time.
I posted a message in the installation forum
http://bbs.archlinux.org/viewtopic.php?id=98936
Hope you can help out. How to deactivate this message at boot time without using the rollback feature.
Offline
hmm i didnt think about that... i was going to put a timeout on the read, but i want to make sure poeple get a chance to see it, and then i'd have to change the default to no instead of yes.
you could manually edit the hook (function btrfs_ask_volatile, line 140), and set a timeout (add -t3 for 3 seconds).
however, you should probably get input working by adding the necessary modules to your /etc/mkinitcpio.conf; if you don't have working input, you will not be able to trigger the rollback selection process, or choose a snapshot to boot.
im not really sure how to handle this gracefully... ill think on it, maybe someone has an idea.
what am i but an extension of you?
Offline
I've added usbinput to modules for my laptop keyboard and rebuild the init. I also try a ps/2 keyboard without success.
One good way for handle new features, is the install notice or a link to a readme in this install notice. So that a user can set the rollback feature manual.
Offline
i will add the notice to the install as well, however i'd still like to keep the ability to automatically enable the feature from the ramfs...
did you get it working then?
maybe ill just add small timeout read that says "PRESS KEY TO ENABLE NON_VOLATILE ROLLBACK...", then if the user presses a key ill display the warning
what am i but an extension of you?
Offline
have no idea where to add the -t3 param. do i need to replace the -p?
the exact line is
read -p "${prompt}" ans
but i use the workaround with disablehook=btrfs, that works perfekt
Offline
have no idea where to add the -t3 param. do i need to replace the -p?
the exact line is
read -p "${prompt}" ans
but i use the workaround with disablehook=btrfs, that works perfekt
the line would then become:
read -t3 -p "${prompt}" ans
you must not have a multidevice setup. the kernel + default initramfs can actually handle btrfs on its own (without this hook) as long as you are not doing anything fancy like raid. the hook ATM is mainly for rollback support, but i do intend on adding more features like hotswap/spare pool support and automatic array repair in the future, in addition to a config file and anything else i can come up with :-)
any particular reason you dont want to enable the rollback support? if you do not want this, and your system boots fine with "disablehook=btrfs" (meaning you have a simple setup), i would just uninstall this hook because its not doing anything but slowing down your boot time.
C Anthony
what am i but an extension of you?
Offline
any particular reason you dont want to enable the rollback support? if you do not want this, and your system boots fine with "disablehook=btrfs" (meaning you have a simple setup), i would just uninstall this hook because its not doing anything but slowing down your boot time.
C Anthony
Got me, simple reason, i don't understanding the usage and benefit of rollback.
if this is the same as the recovery mode under windows, i didn't need it. i use a 128 gb ssd and would save as much space i can. also i don't experiment with my system.
but the main reason is still the knowledge gap.
thx for the hint, that i didn't need the hook.
Offline
extofme wrote:any particular reason you dont want to enable the rollback support? if you do not want this, and your system boots fine with "disablehook=btrfs" (meaning you have a simple setup), i would just uninstall this hook because its not doing anything but slowing down your boot time.
C AnthonyGot me, simple reason, i don't understanding the usage and benefit of rollback.
if this is the same as the recovery mode under windows, i didn't need it. i use a 128 gb ssd and would save as much space i can. also i don't experiment with my system.but the main reason is still the knowledge gap.
thx for the hint, that i didn't need the hook.
well the hook in the future may require a specific "subroot" structure, to provide advanced btrfs features. full system rollback its just an obvious case... you can use subvolumes and snapshots for other interesting purposes. if i get a chance to work on AIF, i will probably setup /var, /home, /etc, and friends ALL inside different subvolumes by default. this allows you to mount each with different mount options, such as compression for var to gain speed benefits, and encryption on /home (when it's available). you could enforce quotas on any of them. you could auto-snapshot your /home each day and never worry about accidentally removing anything. snapshot are very lightweight, getting heavier the farther back you go (maybe you only keep the 5 most recent around or something)
i dont know about windows recovery. this however rolls back your system to the day the snapshot was taken; as if you shut down your computer after the snapshot was taken and never turned it on until today.
maybe you don't experiment, but you never know when someone is experimenting on you :-). by that i mean linux distros are a giant, never-ending work-in-progress, and things go wrong. better to get back up and running in 5 min than fudging around wondering how your going to undo the your borked upgrade. if this hasn't happened to you, then it's only a matter of time ;-)
the operation is safe. it is only putting your system inside a dedicated subvolume, instead of letting it live in the root, where it cannot be easily controlled/manipulated.
C Anthony
Last edited by extofme (2010-06-12 22:26:17)
what am i but an extension of you?
Offline
i just try out the new hook and got a message at boot time. the problem is, my input device isn't loaded at this time.
just pushed a minor update that addresses this issue:
addresses possible "lock" (requiring input although no input modules loaded)
lots of updates to .install file
minor cosmetic changes
what am i but an extension of you?
Offline
extofme wrote:any particular reason you dont want to enable the rollback support? if you do not want this, and your system boots fine with "disablehook=btrfs" (meaning you have a simple setup), i would just uninstall this hook because its not doing anything but slowing down your boot time.
C AnthonyGot me, simple reason, i don't understanding the usage and benefit of rollback.
if this is the same as the recovery mode under windows, i didn't need it. i use a 128 gb ssd and would save as much space i can. also i don't experiment with my system.but the main reason is still the knowledge gap.
thx for the hint, that i didn't need the hook.
Windows recovery only saves system files. Also, Snapshotting in a Copy-on-Write file system is effortless to the system i.e. unlike Windows recovery it is not (reading, analyzing, copy, compressing, and indexing), it is also far lighter on disk space.
Basically though. the cool thing that I will use it for is to run before every update. Then if something brakes I can just reboot and all better. Heck, you could even get analytical with it and mount the snapshot and the _active then run diff.
OpenBSD-current Thinkpad X230, i7-3520M, 16GB CL9 Kingston, Samsung 830 256GB
Contributor: linux-grsec
Offline