e4rat - reduce boot time (into X) by some 50% for ext4

Cdh · 2011-04-17 14:46:32

Works good for me but I think it first was irritated by my /swapfile (I have it only for hibernation). I think there should be a note to disable swapfiles for the collection.

toad · 2011-04-17 16:15:26

@ misc
1. Your /var is a showstopper I'm afraid as some files from that directory are included in (my) startup.log (see below)
2. see reply #117 - for further questions check with fsss

toad@archtop 510\11 ~ > cat /var/lib/e4rat/startup.log|grep /var
2054 151236 /var/lib/alsa/asound.state
2054 130888 /var/lib/misc/random-seed
2054 134919 /var/lib/dbus/machine-id
2054 143067 /var/spool/cron/root
2054 136117 /var/spool/cron/toad
2054 135058 /var/spool/cronstamps/root.sys-hourly
2054 135059 /var/spool/cronstamps/root.sys-daily
2054 135060 /var/spool/cronstamps/root.sys-weekly
2054 135061 /var/spool/cronstamps/root.sys-monthly
2054 548040 /var/state/ntp.drift
2054 130484 /var/cache/fontconfig/3830d5c3ddfd5cd38a049b759396e72e-le64.cache-3
2054 134673 /var/cache/fontconfig/df311e82a1a24c41a75c2c930223552e-le64.cache-3
2054 134674 /var/cache/fontconfig/17090aa38d5c6f09fb8c5c354938f1d7-le64.cache-3
2054 134675 /var/cache/fontconfig/f6b893a7224233d96cb72fd88691c0b4-le64.cache-3
2054 134676 /var/cache/fontconfig/d62e99ef547d1d24cdb1bd22ec1a2976-le64.cache-3
2054 134677 /var/cache/fontconfig/2d31a572ce6667f6a0da9c8dc611898b-le64.cache-3
2054 134753 /var/cache/fontconfig/a1c95d6dfc9a7b34f44445cf81166004-le64.cache-3
2054 134680 /var/cache/fontconfig/8d4af663993b81a124ee82e610bb31f9-le64.cache-3
2054 134678 /var/cache/fontconfig/4b172ca7f111e3cffadc3636415fead9-le64.cache-3
2054 134672 /var/cache/fontconfig/5ca8086aeacc9c68e81a71e7ef846b3b-le64.cache-3
2054 534484 /var/tmp/kdecache-toad/ksycoca4
2054 554679 /var/tmp/kdecache-toad/ksycoca4stamp
2054 275672 /var/lib/upower/history-rate-42T4504-56-1319.dat
2054 282942 /var/lib/upower/history-charge-42T4504-56-1319.dat
2054 284429 /var/lib/upower/history-time-full-42T4504-56-1319.dat
2054 282034 /var/lib/upower/history-time-empty-42T4504-56-1319.dat
2054 549210 /var/tmp/kdecache-toad/plasma-svgelements-oxygen
2054 788980 /usr/share/qalculate/variables.xml
2054 791981 /var/tmp/kdecache-toad/plasma-wallpapers/usr/share/archlinux/wallpaper/archlinux-simplyblack.png_#000000_0_1680x1050.png

ChoK · 2011-04-17 18:21:27

I'm not sure it's a showstopper, e4rat can't optimize my /var partition (maybe there is too many big files due to pacman cache) but it optimize my root partition.

Also e4rat-preload is optional, as I understand it, the main gain from e4rat is by minimizing seeking times by reordering the physical location of files accessed at boot. preload is just another readahead implementation.

I use e4rat with systemd (without the preload though), it works, but I didn't bootchart it yet, as I said in the systemd thread, bootchart 0.9 doesn't load the alternative init system even though I set bootchar_init=/bin/systemd

toad · 2011-04-17 18:44:17

Good point. Probably no harm in creating the startup.log file to see which files are involved.

...the main gain from e4rat is by minimizing seeking times by reordering the physical location of files accessed at boot.

Exactly, and reallocation on non-ext4 partitions appears to be a show stopper - at least for some in this thread.

ChoK · 2011-04-17 19:46:20

Ah yeah of course, e4rat uses a special feature of ext4 (related to online defragmenting iirc) to reallocate the files. So on ntfs and ext2 and others it will not work. Still it would be a bug if just having one partition as such prevent the others from being optimized.

Last edited by ChoK (2011-04-17 19:46:42)

toad · 2011-04-17 20:07:37

I wonder if one could just delete files from startup.log which happen to be "offending" partitions.

sonay · 2011-04-17 21:45:53

graysky wrote:

sonay wrote:
I don't know where to report this bug, but I should warn you. It's not playing nice with filenames containing spaces. Don't worry though, it just complains about syntax when reallocating. No data lost, at least not in my hard drive.
Why do you have spaces in your files names

Networkmanager names the interfaces such as "Auto eth0" or "Auto Default" and Chromium has some subdirectories with spaces in their names such as "Local Storage", etc.

sonay · 2011-04-17 21:52:36

fsckd wrote:

What kind of spaces are we talking about?

Well, I am not skilled enough to answer that but it's just regular filenames like "~/.config/chromium/Local Storage" or some suffix "chromium %u". I just deleted those lines from start.log and it's working just fine, 6 or 7 lines don't add any difference to boot.

misc · 2011-04-18 15:02:19

toad wrote:

I wonder if one could just delete files from startup.log which happen to be "offending" partitions.

Sounds like a viable option for me, particularly if preload is really optional as ChoK said. Still, I need this computer and thus can't risk a loss of my data, so I'll pass until there's a better official documentation.

toad · 2011-04-18 15:08:35

preload is only for reading files - I doubt the file system is of great import there. It is the reallocation procedure which is at the heart of e4rat. But I think is is worth a shot. Of course, I'd back everything up first Muahahahaha

Sorry, I shouldn't be encouraging you and I'm sure you won't give a monkey's toss about what I say - and quite rightly so.

EDIT:
readability

Last edited by toad (2011-04-19 07:25:13)

ethail · 2011-04-19 06:53:29

Just two question about e4rat, not sure if this is the correct thread for that.

Does e4rat have any kind of incompatibility with readahead (list, fedora, uber-readahead)? I think had read that somewhere but don't remember where.

Also, if I'm not mistaken e4rat and readahead both do the same but is e4rat focused on ext4 filesystems, right? So, there is no use using both at the same time.

Could anyone clarify this? Thank you

OrionFyre · 2011-04-19 07:58:54

Just thought I'd post my experiences here as a cautionary warning (mainly since I didn't pay close enough attention)

My filesystem was ext4. So I go along merrily on my way and get to the e4rat-realloc and a whole slew of errors about extents fly up. I had forgotten that I had converted to ext4 relatively recently.... About 2/3rds of the files in the log were vanquished in one fell swoop

Everything from fstab, user accounts and passwords even /etc/rc.conf.... So much for not reading the fine print close enough HAHA!

convertee's beware (or be cautious) I had absolutely no problems with startup scripts or config files in my home directory even though that was upgraded from ext3 as well. However on my home I had used chattr +e on everything, just not my root. So apparently adding the extents flag will get you by. How would that effect links? (i think that might have been why I didn't do it on root)

ChoK · 2011-04-20 20:34:31

@ethail :

e4rat has 2 components :
- first one is e4rat reallocation process, it changes the physical location of files accessed during the first minutes after booting so the OS doesn't have to read everything all other the place. This is the main part of e4rat and the key difference. And it uses ext4-only features.
- second one is preload. e4rat preloads the libraries accessed the first minutes after booting (for exemple kwin/compiz). It is similar to readahead / sreadahead and preload. Obviously you should only use one have those. But you can use the realloc part of e4rat with the readahead of systemd/fedora for exemple. Also this part shouldn't depends on ext4.

Last edited by ChoK (2011-04-20 20:36:09)

ANOKNUSA · 2011-04-21 16:54:44

So I've been tinkering with this but so far can't render a judgement, because changing init=/sbin/e4rat-collect--either replacing it with e4rat-preload or leaving it blank--screws up my touchpad.

┌╼[anoknusa]+[Welcome To SKYNET]
└╼[~]╼╼╼> synclient
Couldn't find synaptics properties. No synaptics driver loaded?

I can use my touchpad, but my custom settings don't get loaded and I can't change them myself. I followed the setup instructions to the letter multiple times, but haven't seen any change. One other person mentioned this in an unrelated thread, with no mention of trying to find a solution. Any thoughts?

OrionFyre · 2011-04-21 21:45:38

ANOKNUSA wrote:

So I've been tinkering with this but so far can't render a judgement, because changing init=/sbin/e4rat-collect--either replacing it with e4rat-preload or leaving it blank--screws up my touchpad.
┌╼[anoknusa]+[Welcome To SKYNET]
└╼[~]╼╼╼> synclient
Couldn't find synaptics properties. No synaptics driver loaded?
I can use my touchpad, but my custom settings don't get loaded and I can't change them myself. I followed the setup instructions to the letter multiple times, but haven't seen any change. One other person mentioned this in an unrelated thread, with no mention of trying to find a solution. Any thoughts?

I can confirm this as well. Once I got my crashed laptop up and running I installed and ran the e4rat suit on my second laptop. Everything went smooth and my boot time into gnome with firefox,pidgin, a terminal and all the gnome widgets went from 59-63 seconds to 23 seconds. Touchpad didn't load my settings. Circular scrolling, horizontal scrolling on the bottom and middle click via the upper left corner were all hosed.

Two strikes and I'm out. It was impressive to see the GUI just spring up on the screen like it did though!

ngoonee · 2011-04-21 23:12:44

Is there any way to check whether a current ext4 partition was converted from ext3 or created as ext4? Can't remember what I did to my partitions before this....

simunic · 2011-04-22 05:01:33

ANOKNUSA wrote:

So I've been tinkering with this but so far can't render a judgement, because changing init=/sbin/e4rat-collect--either replacing it with e4rat-preload or leaving it blank--screws up my touchpad.
┌╼[anoknusa]+[Welcome To SKYNET]
└╼[~]╼╼╼> synclient
Couldn't find synaptics properties. No synaptics driver loaded?
I can use my touchpad, but my custom settings don't get loaded and I can't change them myself. I followed the setup instructions to the letter multiple times, but haven't seen any change. One other person mentioned this in an unrelated thread, with no mention of trying to find a solution. Any thoughts?

Same problem here. When I used e4rat-collect the touchpad and the two buttons on it were working fine. Then I changed to init 1, used e4rat-realloc and now, when I start using e4rat-preload, I can't use my touchpad.

I can't boot without preload because grub tells me that "alloc magic is broken at 0x7f420820" so I can't test this situation.

I don't know if it is important, but I'm using x86_64 and KDE. Any ideas of what I could try?

EDIT: Now I can disable preload from grub but touchpad is still dead.

EDIT 2: Using the Fallback entry on grub solves the problem once but it seems that doesn't solve it anymore. So any ideas of what is going on?

Last edited by simunic (2011-04-22 15:59:01)

X-dark · 2011-04-22 09:12:39

Just tested e4rat. The gain for loading X is not that big (~10s). Where I sense the best gain is after login (ie loading of Xmonad, Firefox, Thunderbird, Pidgin).

fsss · 2011-04-22 10:01:45

Another warning!
After a few uses of e4rat-realloc my harddrive began to utter strange and loud sounds... I've checked it with smartctl and since last test have appeared many reallocated sectors (bad blocks). When I switch to my an old Arch, disk works fine. So it means that only this part of disk where e4rat mixed up is damaged.
I believe that this is just a coincidence, but you should pay more attention to the SMART reports.

Andyvec · 2011-04-23 17:03:28

- Erased -

Last edited by Andyvec (2011-04-23 21:16:20)

smartass · 2011-04-24 13:36:34

Hi!
I tested the e4rat suite on a fresh "sandbox" install this weekend, so let me share my experience:
tested on this configuration: KDE 4.6.2, 64 bit

results:
after clean install : 41 s
after e4rat-realloc : 33 s
with e4rat-preload: 32 s

Conslusions:
The 10 s difference may not seem like much, but apps pop-up very quickly. Without preload it boots faster, but KDE loads slower.
With preload booting takes extra time (loading files into cache) but KDE loads faster and so do other apps, so using preload is IMHO more beneficial in the long-run, not just for booting. e4rat-preload also doesn't slow down boot as the preload daemon does.

Problems:
after using preolad, synaptics really screws up, driver can't load, tried removing it from startup.log, tried reinstalling it, tried reallocating without it.....and then, tried e4rat-collect AND IT WORKED!

so here's a very ugly hotfix for those who don't want to reinstall everything and can live without e4rat preload:
add permanently e4rat-collect to boot parameters and in /etc/e4rat.conf set timeout to 1. That means that e4rat-collect will run, which makes synaptics work (God knows why) and yet it terminates after 1 sec, so it can't rewrite startup.log, because it will be mounted read-only at that time.

Will report this odd fix upstream, let's hope it will help them to find the real bug.

flamelab · 2011-04-24 16:25:36

A maybe silly question: should we re-do the e4rat steps once in e.g. one month ? Or it is just once ?

kyla · 2011-04-24 17:48:54

Under what circumstances would I need to have it "re-scan" (...there's a better word, isn't there?) my start-up? Like, if I changed from gnome to a different desktop environment? If I changed themes within gnome?

kokoko3k · 2011-04-24 18:03:50

e4rat is not a "set and forget it" tool, probably you have to 'defrag' every time the application you want to speed up at boot and/or it's dependancies are updated.

The more you update them without rescan, the lower the benefits you get from this tool;

Isn't spending time to reallocate or finding apps that needs to be reallocated because they were updated just to gain back that time over the boot process a little nonsense?
After all we're using a rolling release distro which (by design) can't play nicely with e4rat over time.

So i keep my idea: +1 for hibernation.

Last edited by kokoko3k (2011-04-24 18:12:28)

Cdh · 2011-04-24 18:12:09

Since there was some discussion I want to give my bit too.
My synaptics touchpad works just fine with the default kernel.
BUT: For trying if I could fix the wakeup from resume issues I compiled kernel26-mainline and booting it without ever reallocating something, but still preloading the previous files it first worked but after a reboot the touchpad stopped working and it was not fixed by rebooting. Booting the old kernel again - still with preloading - the touchpad has worked every time for me.

So definitely worth investigating but it's not that it is not working. It seems more like a race condition. Something starting too fast?

Arch Linux

#126 2011-04-17 14:46:32

Re: e4rat - reduce boot time (into X) by some 50% for ext4

#127 2011-04-17 16:15:26

Re: e4rat - reduce boot time (into X) by some 50% for ext4

#128 2011-04-17 18:21:27

Re: e4rat - reduce boot time (into X) by some 50% for ext4

#129 2011-04-17 18:44:17

Re: e4rat - reduce boot time (into X) by some 50% for ext4

#130 2011-04-17 19:46:20

Re: e4rat - reduce boot time (into X) by some 50% for ext4

#131 2011-04-17 20:07:37

Re: e4rat - reduce boot time (into X) by some 50% for ext4

#132 2011-04-17 21:45:53

Re: e4rat - reduce boot time (into X) by some 50% for ext4

#133 2011-04-17 21:52:36

Re: e4rat - reduce boot time (into X) by some 50% for ext4

#134 2011-04-18 15:02:19

Re: e4rat - reduce boot time (into X) by some 50% for ext4

#135 2011-04-18 15:08:35

Re: e4rat - reduce boot time (into X) by some 50% for ext4

#136 2011-04-19 06:53:29

Re: e4rat - reduce boot time (into X) by some 50% for ext4

#137 2011-04-19 07:58:54

Re: e4rat - reduce boot time (into X) by some 50% for ext4

#138 2011-04-20 20:34:31

Re: e4rat - reduce boot time (into X) by some 50% for ext4

#139 2011-04-21 16:54:44

Re: e4rat - reduce boot time (into X) by some 50% for ext4

#140 2011-04-21 21:45:38

Re: e4rat - reduce boot time (into X) by some 50% for ext4

#141 2011-04-21 23:12:44

Re: e4rat - reduce boot time (into X) by some 50% for ext4

#142 2011-04-22 05:01:33

Re: e4rat - reduce boot time (into X) by some 50% for ext4

#143 2011-04-22 09:12:39

Re: e4rat - reduce boot time (into X) by some 50% for ext4

#144 2011-04-22 10:01:45

Re: e4rat - reduce boot time (into X) by some 50% for ext4

#145 2011-04-23 17:03:28

Re: e4rat - reduce boot time (into X) by some 50% for ext4

#146 2011-04-24 13:36:34

Re: e4rat - reduce boot time (into X) by some 50% for ext4

#147 2011-04-24 16:25:36

Re: e4rat - reduce boot time (into X) by some 50% for ext4

#148 2011-04-24 17:48:54

Re: e4rat - reduce boot time (into X) by some 50% for ext4

#149 2011-04-24 18:03:50

Re: e4rat - reduce boot time (into X) by some 50% for ext4

#150 2011-04-24 18:12:09

Re: e4rat - reduce boot time (into X) by some 50% for ext4

Board footer