general optimization suggestions

drakosha · 2006-03-15 08:03:09

Hi

I love Arch. It's fast, convenient and easy to understand. However i sometimes feel the system should be faster.
So maybe i'm missing something... i'd like to hear system optimization techniques in this thread.

Let me start:
* compile your own kernel
* make sure DMA is turned on for hard drives
* use light desktop env/wm

JGC · 2006-03-15 08:27:25

drakosha wrote:

Hi
I love Arch. It's fast, convenient and easy to understand. However i sometimes feel the system should be faster.
So maybe i'm missing something... i'd like to hear system optimization techniques in this thread.
Let me start:
* compile your own kernel
* make sure DMA is turned on for hard drives
* use light desktop env/wm

I switched from self-compiled kernel to the standard kernel26 kernel using initrd. I don't see any speed degrades switching to it, so compiling your own kernel is bullshit nowadays. Also, when compiling own kernels, you lose all the patches we apply to it, causing things to break. Examples of these are udev rules that don't work or DRI drivers that hang up.

Making sure DMA is on: kernel should turn it on by default, within the last year or so, I haven't had to enable DMA on one single disk anymore. The only thing that isn't enabled by default all the time is 32bit I/O, which is good to enable.

About the WM: Only if you're low on memory. A light WM doesn't make your desktop faster if you have enough RAM to run a heavy one. KDE is quite fast, GNOME 2.12 is a bit slow, but 2.14 has many improvements in speed that makes it as fast as any other desktop.

drakosha · 2006-03-15 08:36:56

Thanks, i'd like to hear other suggestions (which in fact work... not like mine :oops: )

zbled · 2006-03-15 13:14:59

I switched from self-compiled kernel to the standard kernel26 kernel using initrd. I don't see any speed degrades switching to it, so compiling your own kernel is bullshit nowadays.

Not really. I've also tried the standard-kernel for a few weeks, but I've switched back to vanilla with the ck patchset. The standard-kernel was very slow here. Believe it or not, even pacman was loading slow as hell, at least at the first time. Next problem was that the standard-kernel didn't free any memory. After some hours it was using swap even if there was no need for swaping. and after 3 days without rebooting i had to reboot, because the entire system only hardly responded.

I'll post a "time pacman -Syu" - if you want me to ;-)

About the WM: Only if you're low on memory. A light WM doesn't make your desktop faster if you have enough RAM to run a heavy one. KDE is quite fast, GNOME 2.12 is a bit slow, but 2.14 has many improvements in speed that makes it as fast as any other desktop.

What is "low on memory"? I was running gnome 2.12 on my laptop (Pentium M 740 / 512 MB Ram) and it was slow. I switched back to fluxbox and everything's fine again.

iphitus · 2006-03-15 13:39:36

Use ArchCK

JGC: degrading was caused in distro kernels, by having practically every driver loaded, so in our ide kernel, that was every ide and sata driver. Excessive things in kernel = wasted cache. With initrd, assuming you have even somewhat cut it back, less cache is being wasted than a full kernel, less ram is being wasted.

As for WM/DE. Whether that assists the system in a noticable way, depends entirely on the system. Logic says that if you're runnin Gnome on a 486, it's gonna be ... slow. But when you're running it on even a moderate computer, >1.4ish ghz, there is a much much less nonticable impact, as gnome doesnt use enough resources, or in this case, CPU, to deprive any other process of it.
Same applies with ram, less than 512mb, and things will suck, more than it, it ought to not matter what WM/DE.

The other side of the coin, is by using twm, you're cutting down the usability of your system. Things you might have done in gnome using Gnome VFS, desktop icons, or even panels, may well take a bit longer with twm or whatever wm you choose. So while it may feel responsive, tasks could take longer.

Statements above are pretty much true for KDE or XFCE too. Things about cpu/ram usage, may not apply for all workloads.

IMHO, pick the WM/DE you like using the most, not the one you reckon will make your system run uber fast, chances are you can do things a heap quicker and better in it.

Other performance tips, make it easy to open applications, so you dont have to dig deep through menus, keyboard shortcuts work well. Learning keyboard shortcuts themselves is another very good performance tip. You can save a helluva lot of time easily by changin how you use your system.

On the performance hacks side, i've got a nice hack for pacman, that makes it much much quicker and more responsive. I'll upload details tommorow.

iphitus

FUBAR · 2006-03-15 15:03:15

Back in the days when I was running Gentoo, prelinking seriously boosted the startup times for many programs. Maybe you should give that a try.

Gentoo Linux Prelink Guide

EAD · 2006-03-15 15:49:51

I think the most speedup is if the people who make ARCH will compile it all from zero with the newest GCC and librarys, and so on.
and if there could be more version of ARCH, for exmaple there is a lot of different between AMD and INTEL when it come to max optimization.
sure it will take a lot of time to do all this, and sure it won't give much better performance, but it will give some more :twisted:

kensai · 2006-03-15 16:21:02

JGC wrote:

I switched from self-compiled kernel to the standard kernel26 kernel using initrd. I don't see any speed degrades switching to it, so compiling your own kernel is bullshit nowadays.

Couldn't disagree more. Using stock kernel memory usage in a fresh boot starts at 16% with my custom compiled kernel memory usage starts at 8% (Also make sure to use archck or ck patchset. And memory recuperation is better with a custom compiled kernel than with stock. Conclusion Custom compiled kernel with ck or archck patches handles memory significantly better and handles CPU slightly better,

EAD · 2006-03-15 16:31:00

kensai wrote:

JGC wrote:
I switched from self-compiled kernel to the standard kernel26 kernel using initrd. I don't see any speed degrades switching to it, so compiling your own kernel is bullshit nowadays.
Couldn't disagree more. Using stock kernel memory usage in a fresh boot starts at 16% with my custom compiled kernel memory usage starts at 8% (Also make sure to use archck or ck patchset. And memory recuperation is better with a custom compiled kernel than with stock. Conclusion Custom compiled kernel with ck or archck patches handles memory significantly better and handles CPU slightly better,

amm I guess you are right, but that is becasue all the moudles you havn't load at start, wich is nice, I think that maybe doing a more light wight kernel with ARCH will be very good.

drakosha · 2006-03-15 20:35:41

So currently the suggestions are:
1. compile your own kernel
2. complie your own everything
3. try prelink

Any other suggestions?

Dusty · 2006-03-15 20:45:41

I can guarantee you won't notice anything from 1 and 2. You may have a measurable performance increase but it won't be noticeable. I did have a noticeable increase when I used the archck kernel. Then something changed and I had a massively noticeable decrease in performance and went back to stock; its not hard to try, so try it.

Prelinking has been discussed, do a forum search. Most people were surprised by the lack of effect, but it *sounds* like a good idea. :-D

With processing speeds and memory available these days, I'd be so bold as to say that most user-noticeable performance issues are IO-bound. This is to say that speeding up your disk access will provide the most noticeable boost in productivity. You might do this with hdparm, and also with filesystem tuning. Selecting the right filesystem (good luck defining "right", its a lovely flamewar) and optimizing it may have some effect. Having programs loaded in memory before you open them will also have an effect. In my case, I tend to keep a lot of apps open even when I'm not using them (pekwm lets me hide them on other desktops). That way they're in memory or at worst, swap-space when I need them, reducing startup time.

There are kernel options to make your system more responsive, but I suspect either the stock kernel or arch-ck kernel will have optimized these as well as you can.

Dusty

phrakture · 2006-03-15 21:10:35

As for all you people disagreeing with the custom/stock kernel statement made by JGC, you are talking about a patchset, not a custom compile of the same kernel. The point is that a custom vanilla kernel vs the stock kernel will have no noticable difference. If you want to use the ck patchset, try the archck kernel, which is now officially supported.

jondkent · 2006-03-15 21:48:01

I just wanna add my 2c worth here :

Compiling is usually a waste of time. The only exception are apps like mplayer which do perform better when complied for your cpu type. But if you have a powerful system the difference will not be noticable

Roll your own kernel is again, mostly pointless these days. I've done it for years, but stopped recently as there is very little difference in performance. CK patch set is very good if your want resposiveness, but for a server it not really want you want to use.

I'd agree with the comment re desktop, find one you like, you'll be more productive.

Lastly, if you do wanna play, benchmark you system first otherwise you'll may think you've improve things when there is no change, it could even be worse. An improvement in performance can be all in the mind you know. So model your usage, and then see what can be done.

The best route I've found to increase performance is to make sure you have enough memory to avoid swapping (not paging, that'll always happen), and think about your disk speed, maybe even you filesystem.

Jon

jondkent · 2006-03-15 21:51:39

I should add that baselining your system to identify the bottlenecks, if any, will aid you in the process of getting more performance. sar is probably you best tool here, but there are others.

Of course you may wanna just increase the performance of one app, if which case maybe renicing lower than 0 may help. It'll increase the timeslice the kernel provides the app thereby, maybe, helpping it out.

Jon

Gullible Jones · 2006-03-15 22:24:30

JGC wrote:

Making sure DMA is on: kernel should turn it on by default, within the last year or so, I haven't had to enable DMA on one single disk anymore. The only thing that isn't enabled by default all the time is 32bit I/O, which is good to enable.

It isn't? Why not? :shock:

kensai · 2006-03-15 22:35:20

phrakture wrote:

As for all you people disagreeing with the custom/stock kernel statement made by JGC, you are talking about a patchset, not a custom compile of the same kernel. The point is that a custom vanilla kernel vs the stock kernel will have no noticable difference. If you want to use the ck patchset, try the archck kernel, which is now officially supported.

Still a custom compiled archck kernel uses less memory and is more cpu efficient than archck from repo.

Gullible Jones · 2006-03-15 22:50:24

Not really, initramfs remember?

And FWIW here's how you optimize your initrd:

1. Using mkinitrd.conf, disable subsystems that you don't use. If, for example, you do not use SCSI, set REMOVE_SCSI to "1".

2. Set AUTODETECT ("0" by default) to "1".

3. Specify your filesystem modules via FILESYSTEMS, e.g. a user of ext3 would have FILESYSTEMS="jbd ext3" and one using JFS would use FILESYSTEMS="jfs".

4. Now save mkinitrd.conf and remake your initrd via 'mkinitrd auto'.

The next time you boot up, you'll notice that the boot process is faster and there are no more modules floating around that you don't use.

Dusty · 2006-03-15 22:50:32

kensai wrote:

Still a custom compiled archck kernel uses less memory and is more cpu efficient than archck from repo.

The point is, the time it takes to compile that kernel (7-20 minutes, I've heard) will never be recovered in terms of *noticeable* responsiveness to the user. You can quote numbers saying its more efficient or uses less memory, but these resources are certainly not scarce in modern systems.

Dusty

JGC · 2006-03-15 23:34:49

kensai wrote:

JGC wrote:
I switched from self-compiled kernel to the standard kernel26 kernel using initrd. I don't see any speed degrades switching to it, so compiling your own kernel is bullshit nowadays.
Couldn't disagree more. Using stock kernel memory usage in a fresh boot starts at 16% with my custom compiled kernel memory usage starts at 8% (Also make sure to use archck or ck patchset. And memory recuperation is better with a custom compiled kernel than with stock. Conclusion Custom compiled kernel with ck or archck patches handles memory significantly better and handles CPU slightly better,

Next optimization: run pacman -Syu to get your kernel upgraded twice a week, or let your workhorse rebuild your entire kernel every 3 days... I choose for the first part. I was getting sick of all the compilation stuff, the needed patches for new udev versions, etc. I'll either use kernel26 or kernel26archck, nothing else.

Romashka · 2006-03-16 09:18:16

Gullible Jones wrote:

And FWIW here's how you optimize your initrd:
1. Using mkinitrd.conf, disable subsystems that you don't use. If, for example, you do not use SCSI, set REMOVE_SCSI to "1".

I rephrase this:
set all REMOVE_* to 1 except subsystem from which your kernel boots.
For example, if your system is on IDE - disable all subsystems except IDE; if your system is LiveCD - remove all except CDROM etc.
If your system boots from IDE HDD you can add REMOVE_MODULE=ide_floppy to remove yet another unneeded module.

iphitus · 2006-03-16 10:24:13

Dusty wrote:

kensai wrote:
Still a custom compiled archck kernel uses less memory and is more cpu efficient than archck from repo.
The point is, the time it takes to compile that kernel (7-20 minutes, I've heard) will never be recovered in terms of *noticeable* responsiveness to the user. You can quote numbers saying its more efficient or uses less memory, but these resources are certainly not scarce in modern systems.
Dusty

Tis why I finally switched to using a distro kernel. I end up compiling them anyway..... but that's beside the point .

kensai: A custom ArchCK, shouldnt use much more ram at all than an ArchCK from extra, with a properly configured and slim initramfs.

James

kozaki · 2006-03-16 11:21:34

Dusty wrote:

kensai wrote:
Still a custom compiled archck kernel uses less memory and is more cpu efficient than archck from repo.
The point is, the time it takes to compile that kernel (7-20 minutes, I've heard) will never be recovered in terms of *noticeable* responsiveness to the user. You can quote numbers saying its more efficient or uses less memory, but these resources are certainly not scarce in modern systems.
Dusty

Well, that's what I thought so i've been using standard kernel with adapted mkinitrd.conf for almost a year with great success. Also when I bought a new computer, I made sure I got plenty of (unexpansive) DDR400 SDRAM + the fastest rather than the biggest SATA(II) HDD along an AMD64 & a cool Mobo.

Now I succesfully compiled my 1st Iphitus' Archck kernel without initrd/~ramfs on my old laptop last month, & I found it eating as low as 29MB RAM with Founy's e17-cvs fully launched, which I liked (less RAM for system is allways good & better still on laptop in terms on usability & autonomy )

It seems that Archck kernel improved or maybe my knowledge on compiling an appropriate kernel, cauz' last week I compiled it on my Dell Latitude L400 along with Suspend & Speedstep support. Rebooted & :GASP:! Memory usage before starting X was 9 MB !!! I just didn't even dreamed about such a result.

And yesterday on my AMD64 main PC (1024 MB RAM with lots of FS & accessories like TV TUner or external HDD) --> 19MB without X, and an amazing 60MB with f@h, rTorrent, e16 (X) & aterm

Thoses would have been postponned without Wain's great kernel26-archck PKGBUILD (FR) which has improved _a lot_ recently, & allow any AL user to take care only of the modules (s)he needs

zbled · 2006-03-16 16:43:24

I'm sorry. it was my fault. I was using initrd26-full.img instead of initrd26.img. it's running smooth.

jaboua · 2006-03-16 22:06:11

phrakture wrote:

As for all you people disagreeing with the custom/stock kernel statement made by JGC, you are talking about a patchset, not a custom compile of the same kernel. The point is that a custom vanilla kernel vs the stock kernel will have no noticable difference. If you want to use the ck patchset, try the archck kernel, which is now officially supported.

A vanilla custom kernel boots faster than the stock kernel in my experience, and IMO it's easiest to put all my modules right in the kernel instead of hunting down modules to automaticly load at boot

drakosha · 2006-03-17 07:19:27

So a summary again. This time number of '*' specifies popularity of a suggestion (it's subjective, 1 to 5):

Compile your own kernel ****
Compile your own everything *
Use sysstat to analyse the system and then decide what to do **
Ensure your HDs are configured properly (hdparam) **
Using mkinitrd.conf, disable everything you don't use. Make sure you remove everything you don't need ****

Am i missing something?

Arch Linux

#1 2006-03-15 08:03:09

general optimization suggestions

#2 2006-03-15 08:27:25

Re: general optimization suggestions

#3 2006-03-15 08:36:56

Re: general optimization suggestions

#4 2006-03-15 13:14:59

Re: general optimization suggestions

#5 2006-03-15 13:39:36

Re: general optimization suggestions

#6 2006-03-15 15:03:15

Re: general optimization suggestions

#7 2006-03-15 15:49:51

Re: general optimization suggestions

#8 2006-03-15 16:21:02

Re: general optimization suggestions

#9 2006-03-15 16:31:00

Re: general optimization suggestions

#10 2006-03-15 20:35:41

Re: general optimization suggestions

#11 2006-03-15 20:45:41

Re: general optimization suggestions

#12 2006-03-15 21:10:35

Re: general optimization suggestions

#13 2006-03-15 21:48:01

Re: general optimization suggestions

#14 2006-03-15 21:51:39

Re: general optimization suggestions

#15 2006-03-15 22:24:30

Re: general optimization suggestions

#16 2006-03-15 22:35:20

Re: general optimization suggestions

#17 2006-03-15 22:50:24

Re: general optimization suggestions

#18 2006-03-15 22:50:32

Re: general optimization suggestions

#19 2006-03-15 23:34:49

Re: general optimization suggestions

#20 2006-03-16 09:18:16

Re: general optimization suggestions

#21 2006-03-16 10:24:13

Re: general optimization suggestions

#22 2006-03-16 11:21:34

Re: general optimization suggestions

#23 2006-03-16 16:43:24

Re: general optimization suggestions

#24 2006-03-16 22:06:11

Re: general optimization suggestions

#25 2006-03-17 07:19:27

Re: general optimization suggestions

Board footer