YAHTBLFA (Yet Another How To Boot Linux Faster Article)

Jariko · 2005-07-28 14:47:51

YAHTBLFA (Yet Another How To Boot Linux Faster Article)
=======================================================

Since bootchart (www.bootchart.org) was released, boot-time optimization has been very popular in Linux world.
I recently tried some boot-time reducing tricks on my Arch Linux box. Arch already boots really fast but there is always room for improvement ;-)

My PC is an Athlon 64 3200+, 1GB RAM, 120 GB SATA hard drive and Nvidia FX5900XT video card, running a pretty much stock
Arch 0.7 install with 2.6.12-ARCH kernel & udev.

Original boot speed
-------------------
Phase Stopwatch time (seconds)
------ -------------------------
Bootloader starts kernel 0
Kernel starts init 11
Hotplug starts 15
Hotplug ends, Kdm starts 25
Kdm login screen visible 36

As we can see, there are 3 big tasks taking almost all time:
A. Kernel initialization is slow, takes 11 seconds
B. Hotplug is slow, takes 10 seconds
C. Starting X+kdm is slow, takes 11 seconds

Optimizations:
--------------
1. Increased boot parallelism & init script changes
---------------------------------------------------
-I changed /etc/inittab so that it loads X in parallel with rc.multi (I loaned this tip from discussion on archlinux mailing list):
>I dont know how useful is this, but its simple to parallelize X and daemons
>-- change the rm:wait:rc.multi line in /etc/inittab to rc:once:...
>It works only if (k,g,x)dm is called through runlevel 5 and not as a daemon
>in runlevel 3.
This runs slow tasks B (Hotplug) and C (X+kdm) in parallel.

-Moved stuff from rc.sysinit (runs alone) to rc.multi (runs in parallel with X startup), after point "#now mount all the local filesystems" I have only "Removing leftover files" and "Load sysctl variables" in rc.sysinit, everything else is moved to rc.multi

-I do not start agettys in /etc/inittab anymore

-Added a sleep of 10 seconds in start of rc.multi (these initializations now run in background when I'm typing in my username/password on login screen)

2. Kernel optimizations
-----------------------
-I used "ShortIDEDelays" and "RTCNoSync" patches from CE Linux Forum (http://www.celinuxforum.org/CelfPubWiki/ShortIDEDelays). Because these patches are not against 2.6.12, I just looked inside patches with Kompare and made changes manually.

-"ShortIdeDelays": I set ide_delay = 5 ms (original is 50 ms). Reduces Task A (Kernel initialization) time about 3 seconds in my box.

-"RTCNoSync": This has a smaller effect (up to 1 second).

-To reduce hotplug time I compiled the following drivers into kernel: ohci_hcd, ehci_hcd, uhci_hcd, parport, realtime_clock, acpi, r8169, pc_speaker, joystick, irda, bluetooth.

-I took away "Load USB support" from rc.sysinit (because it is already in kernel).

3. Changed from Kdm to Gdm
--------------------------
-At the moment Gdm seems to start about 4-5 seconds faster than Kdm. I don't know why.

Boot speed now
--------------
Phase Stopwatch time (seconds)
------ -------------------------
Bootloader starts kernel 0
Kernel starts init 8
rc.multi starts / Gdm starts 11
Gdm login screen visible 20

cmp · 2005-07-28 15:07:18

nice, could you provide a patch for the changed init files?

Dusty · 2005-07-28 15:11:04

Might be interesting to flesh this out and post it to the wiki.

Did you disable hotplug altogether?

Dusty

Jariko · 2005-07-28 16:47:40

cmp wrote:

nice, could you provide a patch for the changed init files?

Yes, here comes (sorry if the format is incorrect, I haven't read patch/diff howto ;-)

1. /etc/inittab:
==============

14c14
< id:3:initdefault:
---
> id:5:initdefault:
18c18
< rm:2345:wait:/etc/rc.multi
---
> rm:2345:once:/etc/rc.multi
22,27c22,27
< c1:2345:respawn:/sbin/agetty 38400 vc/1 linux
< c2:2345:respawn:/sbin/agetty 38400 vc/2 linux
< c3:2345:respawn:/sbin/agetty 38400 vc/3 linux
< c4:2345:respawn:/sbin/agetty 38400 vc/4 linux
< c5:2345:respawn:/sbin/agetty 38400 vc/5 linux
< c6:2345:respawn:/sbin/agetty 38400 vc/6 linux
---
> #c1:2345:respawn:/sbin/agetty 38400 vc/1 linux
> #c2:2345:respawn:/sbin/agetty 38400 vc/2 linux
> #c3:2345:respawn:/sbin/agetty 38400 vc/3 linux
> #c4:2345:respawn:/sbin/agetty 38400 vc/4 linux
> #c5:2345:respawn:/sbin/agetty 38400 vc/5 linux
> #c6:2345:respawn:/sbin/agetty 38400 vc/6 linux
31c31
< x:5:respawn:/usr/X11R6/bin/xdm -nodaemon
---
> x:5:respawn:/opt/gnome/bin/gdm -nodaemon

2. /etc/rc.sysinit:
=================

26,29c26
< if [ -e /dev/.devfsd -a -x /sbin/devfsd ]; then
< # Looks like devfs is running, use it
< status "Starting DevFS Daemon" /sbin/devfsd /dev
< elif [ -x /etc/start_udev -a -d /sys/block ]; then
---
> if [ -x /etc/start_udev -a -d /sys/block ]; then
37,50d33
< if [ "$USELVM" = "yes" -o "$USELVM" = "YES" ]; then
< if [ -f /etc/lvmtab -a -x /sbin/vgchange ]; then
< # Kernel 2.4.x, LVM1 groups
< stat_busy "Activating LVM1 groups"
< /sbin/vgchange -a y
< stat_done
< elif [ -x /sbin/lvm -a -d /sys/block ]; then
< # Kernel 2.6.x, LVM2 groups
< stat_busy "Activating LVM2 groups"
< /sbin/lvm vgchange --ignorelockingfailure -a y
< stat_done
< fi
< fi
<
96,109d78
< stat_busy "Configuring System Clock"
< if [ "$HARDWARECLOCK" = "UTC" ]; then
< /sbin/hwclock --utc --hctosys
< else
< /sbin/hwclock --localtime --hctosys
< fi
< if [ ! -f /var/lib/hwclock/adjtime ]; then
< echo "0.0 0 0.0" > /var/lib/hwclock/adjtime
< fi
< if [ "$TIMEZONE" != "" ]; then
< /bin/ln -sf /usr/share/zoneinfo/$TIMEZONE /etc/localtime
< fi
< stat_done
<
124,174d92
< #status "Updating Shared Library Links" /sbin/ldconfig
<
< if [ "$HOSTNAME" != "" ]; then
< status "Setting Hostname: $HOSTNAME" /bin/hostname $HOSTNAME
< fi
<
< # Set the NIS domain name, if necessary
< [ -f /etc/conf.d/nisdomainname ] && . /etc/conf.d/nisdomainname
< if [ "$NISDOMAINNAME" != "" ]; then
< status "Setting NIS Domain Name: $NISDOMAINNAME" /bin/nisdomainname $NISDOMAINNAME
< fi
<
< status "Updating Module Dependencies" /sbin/depmod -A
<
< if [ -f /var/run/random-seed ]; then
< stat_busy "Initializing Random Seed"
< /bin/cat /var/run/random-seed >/dev/urandom
< stat_done
< fi
<
< if [ "$KEYMAP" != "" ]; then
< status "Loading Keyboard Map: $KEYMAP" /bin/loadkeys -q $KEYMAP
< fi
<
< if [ "$CONSOLEFONT" != "" ]; then
< stat_busy "Loading Console Font: $CONSOLEFONT"
< for i in `seq 1 12`; do
< if [ "$CONSOLEMAP" != "" ]; then
< /usr/bin/setfont -m $CONSOLEMAP $CONSOLEFONT -C /dev/vc/${i};
< else
< /usr/bin/setfont $CONSOLEFONT -C /dev/vc/${i};
< fi
< done
< stat_done
< fi
<
< # Load USB support
< /sbin/modprobe usbcore >/dev/null 2>&1
< [ "`grep usbfs /proc/filesystems`" ] && mount -t usbfs none /proc/bus/usb
<
< # Load modules from the MODULES array defined in rc.conf
< if [ -f /proc/modules ]; then
< stat_busy "Loading Modules"
< for mod in "${MODULES[@]}"; do
< if [[ `echo $mod | grep '^[^!]' | wc -l` -eq 1 ]]; then
< /sbin/modprobe $mod
< fi
< done
< stat_done
< fi
<
178,180d95
< # Screen blanks after 15 minutes idle time
< /usr/bin/setterm -blank 15
<

BTW, I have removed LVM needlessly here, so beware if you happen to use LVM.

3. /etc/rc.multi:
===========

8a9,78
> #Move these things forward (in time) after kdm is visible
> sleep 10
>
> # Screen blanks after 15 minutes idle time
> /usr/bin/setterm -blank 15
>
> stat_busy "Configuring System Clock"
> if [ "$HARDWARECLOCK" = "UTC" ]; then
> /sbin/hwclock --utc --hctosys
> else
> /sbin/hwclock --localtime --hctosys
> fi
> if [ ! -f /var/lib/hwclock/adjtime ]; then
> echo "0.0 0 0.0" > /var/lib/hwclock/adjtime
> fi
> if [ "$TIMEZONE" != "" ]; then
> /bin/ln -sf /usr/share/zoneinfo/$TIMEZONE /etc/localtime
> fi
> stat_done
>
> if [ "$HOSTNAME" != "" ]; then
> status "Setting Hostname: $HOSTNAME" /bin/hostname $HOSTNAME
> fi
>
>
> # Set the NIS domain name, if necessary
> [ -f /etc/conf.d/nisdomainname ] && . /etc/conf.d/nisdomainname
> if [ "$NISDOMAINNAME" != "" ]; then
> status "Setting NIS Domain Name: $NISDOMAINNAME" /bin/nisdomainname $NISDOMAINNAME
> fi
>
> #status "Updating Shared Library Links" /sbin/ldconfig
>
> status "Updating Module Dependencies" /sbin/depmod -A
>
> if [ -f /var/run/random-seed ]; then
> stat_busy "Initializing Random Seed"
> /bin/cat /var/run/random-seed >/dev/urandom
> stat_done
> fi
>
> if [ "$KEYMAP" != "" ]; then
> status "Loading Keyboard Map: $KEYMAP" /bin/loadkeys -q $KEYMAP
> fi
>
> if [ "$CONSOLEFONT" != "" ]; then
> stat_busy "Loading Console Font: $CONSOLEFONT"
> for i in `seq 1 12`; do
> if [ "$CONSOLEMAP" != "" ]; then
> /usr/bin/setfont -m $CONSOLEMAP $CONSOLEFONT -C /dev/vc/${i};
> else
> /usr/bin/setfont $CONSOLEFONT -C /dev/vc/${i};
> fi
> done
> stat_done
> fi
>
>
> # Load modules from the MODULES array defined in rc.conf
> if [ -f /proc/modules ]; then
> stat_busy "Loading Modules"
> for mod in "${MODULES[@]}"; do
> if [[ `echo $mod | grep '^[^!]' | wc -l` -eq 1 ]]; then
> /sbin/modprobe $mod
> fi
> done
> stat_done
> fi
>
>

Regards,

Jari

Jariko · 2005-07-28 17:05:25

Dusty wrote:

Did you disable hotplug altogether?
Dusty

No, it is just delayed so that user gets to see login screen earlier (things that X needs, like mouse, need to be loaded manually / compiled into kernel).

Regards,

Jari

Dusty · 2005-07-28 17:08:42

Ok. You might be able to speed it up a bit more by removing hotplug. Just load the modules you need in the MODULES array in /etc/rc.conf and you won't need hotplug at all.

Dusty

dtw · 2005-07-28 17:18:58

I think you should clarify that this is only useful for people who use a login manager! It's hardly a catch all solution It should be called how to get X to start faster at boot!

Jariko · 2005-07-28 17:38:52

dibblethewrecker wrote:

I think you should clarify that this is only useful for people who use a login manager! It's hardly a catch all solution It should be called how to get X to start faster at boot!

Yes (except kernel initialization part, which helps everything :-). Kernel initialization does not show in bootcharts, so it has not got much attention thus far.

Many people these days use a login manager, so this gives a better impression for them (similarly to what WinXP does). On my system with these changes, Arch is now 12 seconds faster than XP :-)

Jari

Dusty · 2005-07-28 17:39:56

Yeah, I think its good advice for those that use login managers (I don't). Feel free to add a wiki page for it.

Dusty

dtw · 2005-07-28 17:43:43

Jariko wrote:

On my system with these changes, Arch is now 12 seconds faster than XP :-)

on my system, without those changes, Arch boots about a minute faster than XP

Jariko · 2005-07-28 17:46:12

Dusty wrote:

Ok. You might be able to speed it up a bit more by removing hotplug. Just load the modules you need in the MODULES array in /etc/rc.conf and you won't need hotplug at all.
Dusty

Yes, hotplug *is* very slow, but I need it for my USB flash. I try to emulate what big guys (Fedora/Ubuntu) are currently doing (have full functionality) but at the same time stay away from their horrible SysV scripts

Jari

Dusty · 2005-07-28 17:47:02

size matters...

on my system, Arch boots infinitely faster than XP.

Dusty

kakabaratruskia · 2005-07-28 17:53:48

On my system XP doesn't boot anymore... (I'm no linux guru... my XP just broke).

Jariko · 2005-07-28 17:54:42

dibblethewrecker wrote:

on my system, without those changes, Arch boots about a minute faster than XP

Seriously ? Arch is the first Linux that has actually booted faster than XP on my box (I mean to login screen, Xp:32 secs, Arch:20 secs).

Jariko · 2005-07-28 18:14:33

Dusty wrote:

Yeah, I think its good advice for those that use login managers (I don't). Feel free to add a wiki page for it.
Dusty

These changes just feel so...hastily put together. I was thinking about porting Arch initscripts to init-ng instead. Has anyone done that yet ? Was it any faster ? *That* would be cool Wiki-material.

Jari

Dusty · 2005-07-28 19:15:23

If you wiki it, it will be edited and improved. Sometimes I am truly amazed at how this process works...

Dusty

Shofs · 2005-07-28 19:45:46

Is there something that you can pass at boot-time to start in a certain runlevel?

I am asking specifiaclly for grub. Mabye something like this:

kernel /boot/my_kernel root=/dev/hda3 init=[5,3]

phrakture · 2005-07-28 20:20:26

rmbalfa wrote:

Is there something that you can pass at boot-time to start in a certain runlevel?
I am asking specifiaclly for grub. Mabye something like this:
kernel /boot/my_kernel root=/dev/hda3 init=[5,3]

it's just the number... init=blah specifies the program to run (for instance, init=/sbin/bootchartd is used for logging bootchart info)

kernel /boot/my_kernel root=/dev/hda3 3
or
kernel /boot/my_kernel root=/dev/hda3 5

Jariko · 2005-07-28 20:20:35

rmbalfa wrote:

Is there something that you can pass at boot-time to start in a certain runlevel?

I tried to, but to no avail. I must have broken something

Jariko · 2005-07-28 20:25:30

phrakture wrote:

it's just the number... init=blah specifies the program to run (for instance, init=/sbin/bootchartd is used for logging bootchart info)
kernel /boot/my_kernel root=/dev/hda3 3
or
kernel /boot/my_kernel root=/dev/hda3 5

Aha, I tried to

init=3

and

append="init=3"

I should have known that, it is just a command-line paramter from kernel to init, so initscripts can not harm it. :oops:

iBertus · 2005-07-28 20:27:21

Why do you need hotplug for your usb flash? I'm not using it here and everything work fine.

Jariko · 2005-07-28 20:40:26

iBertus wrote:

Why do you need hotplug for your usb flash? I'm not using it here and everything work fine.

Yes, I know. Basically, I try to learn, to figure out what happens in the system when all these weird-but-wonderful-things (kernel2.6, hotplug, udev, hal, dbus) run.

I have high hopes that somebody somewhere is already coding / testing a much faster version of hotplug

JK

eWoud · 2005-07-29 22:58:06

another tip that might speed-up boot time a little: mount local filesystems simultaniously

edit /etc/rc.sysinit, line 93

change

/bin/mount -a -t nonfs,nosmbfs,noncpfs,nosysfs,nousbfs

to

/bin/mount -a -F -t nonfs,nosmbfs,noncpfs,nosysfs,nousbfs

iphitus · 2005-07-30 02:12:03

If you must have hotplug... dont use it at all.

Use lshwd. pacman -S lshwd hwd

and add hwd to daemons array.

iphitus

jnengland77 · 2005-07-30 02:20:39

:shock: Booted way faster but one i had no sound and two no network.
and i get a bunch of errors about status, stat_busy and stat_done not being commands.

O well back to normal. But i did remove devfs, lvm, and usbcore things since i have them built-in to the kernel. I made it mount local filesystems simultaniously like eWoud said.

Thanks for some tips tho.

Arch Linux

#1 2005-07-28 14:47:51

YAHTBLFA (Yet Another How To Boot Linux Faster Article)

#2 2005-07-28 15:07:18

Re: YAHTBLFA (Yet Another How To Boot Linux Faster Article)

#3 2005-07-28 15:11:04

Re: YAHTBLFA (Yet Another How To Boot Linux Faster Article)

#4 2005-07-28 16:47:40

Re: YAHTBLFA (Yet Another How To Boot Linux Faster Article)

#5 2005-07-28 17:05:25

Re: YAHTBLFA (Yet Another How To Boot Linux Faster Article)

#6 2005-07-28 17:08:42

Re: YAHTBLFA (Yet Another How To Boot Linux Faster Article)

#7 2005-07-28 17:18:58

Re: YAHTBLFA (Yet Another How To Boot Linux Faster Article)

#8 2005-07-28 17:38:52

Re: YAHTBLFA (Yet Another How To Boot Linux Faster Article)

#9 2005-07-28 17:39:56

Re: YAHTBLFA (Yet Another How To Boot Linux Faster Article)

#10 2005-07-28 17:43:43

Re: YAHTBLFA (Yet Another How To Boot Linux Faster Article)

#11 2005-07-28 17:46:12

Re: YAHTBLFA (Yet Another How To Boot Linux Faster Article)

#12 2005-07-28 17:47:02

Re: YAHTBLFA (Yet Another How To Boot Linux Faster Article)

#13 2005-07-28 17:53:48

Re: YAHTBLFA (Yet Another How To Boot Linux Faster Article)

#14 2005-07-28 17:54:42

Re: YAHTBLFA (Yet Another How To Boot Linux Faster Article)

#15 2005-07-28 18:14:33

Re: YAHTBLFA (Yet Another How To Boot Linux Faster Article)

#16 2005-07-28 19:15:23

Re: YAHTBLFA (Yet Another How To Boot Linux Faster Article)

#17 2005-07-28 19:45:46

Re: YAHTBLFA (Yet Another How To Boot Linux Faster Article)

#18 2005-07-28 20:20:26

Re: YAHTBLFA (Yet Another How To Boot Linux Faster Article)

#19 2005-07-28 20:20:35

Re: YAHTBLFA (Yet Another How To Boot Linux Faster Article)

#20 2005-07-28 20:25:30

Re: YAHTBLFA (Yet Another How To Boot Linux Faster Article)

#21 2005-07-28 20:27:21

Re: YAHTBLFA (Yet Another How To Boot Linux Faster Article)

#22 2005-07-28 20:40:26

Re: YAHTBLFA (Yet Another How To Boot Linux Faster Article)

#23 2005-07-29 22:58:06

Re: YAHTBLFA (Yet Another How To Boot Linux Faster Article)

#24 2005-07-30 02:12:03

Re: YAHTBLFA (Yet Another How To Boot Linux Faster Article)

#25 2005-07-30 02:20:39

Re: YAHTBLFA (Yet Another How To Boot Linux Faster Article)

Board footer