Prelinking

shadowhand · 2005-04-22 04:54:59

Has Arch thought about prelinking ( previous discussion ) at all?

I just installed the prelink program, but there are a LOT of non-PIC libraries. Does Arch enable the -fPIC option by default? Personally, I've been a big fan of prelink for a long time (since I used Yoper for the first time [badly executed distro, but good intentions]) and loved how everything just seemed to snap.

Just my $.02.

phrakture · 2005-04-22 05:04:20

it'd be alot of work, but I'd like to see some recent benchmarks/numbers to go with this.... figure out the point of diminishing returns...

Pajaro · 2005-04-22 07:58:45

I investigated about prelinking and I don't remember what it was (so I can be wrong), but I ended forgeting it because nowadays it is not needed.

shadowhand · 2005-04-22 08:18:18

I still notice a difference when I use prelink, but maybe it's just me... But when you load KDE after it's prelinked, damn does it rip. Try running Yoper for about 5 minutes (and if it doesn't crash), you'll see what I mean.

arooaroo · 2005-04-22 10:44:44

I agree. I prelinked KDE in my Gentoo days and I have to say I did notice a significant improvement. (Especially as my hardware was on the slow side). I seem to recall a lot of discussion though as it seemed to do great things for some, and nothing for others.

I was thinking about prelinking KDE on Arch as it happens. Would just like it to start up a tad quicker.

LB06 · 2005-04-22 11:29:44

I also the startup times of many programs improve significantly. Not exactly what one would call a scientific benchmark, but the differences were sometimes measurable with a stopwatch.

But when I tried to rebuild xorg with -fPIC I got an error when I tried to startx. So that didn't work for me.

darkcoder · 2005-04-22 17:45:18

I'm working right as a hobby on the prelinking stuff. Right now the first candidates that need to be modified are xorg/xfree and libtiff. Xorg is specially important for KDE users since for no apparent reason many KDE binaries are linked to libGL.so even when they do not handle GL stuff at all.

Right now, I have a PKGBUILD for xorg which allow files to prelink to libGL.so. During the weekend I will work on libtiff and maybe other problematic packages.

darkcoder · 2005-04-22 17:47:51

Also KDE needs the enviromental value KDE_IS_PRELINKED=1, but haven't found a place to assign it. Tryied rc.local, and as part of one of the services called from rc.conf without sucess. After booting to my user, check if the value is available and it is not.

darkcoder · 2005-04-22 17:54:24

xorg need 2 patches, that can be found at Linux From Scratch
http://www.linuxfromscratch.org/patches … IC-1.patch
http://www.linuxfromscratch.org/patches … IC-1.patch

Apply the patches in that order just after the vm86-tls.patch currently applied by the xorg PKGBUILD. Also add the following lines before the # Katmai: PIII SSE instructions line

  echo $'#define BuildLibGlxWithoutPIC NOn' >> config/cf/host.def
  echo $'#define StaticNeedsPicForShared YESn' >> config/cf/host.def

And now rebuild Xorg.

i3839 · 2005-04-22 18:16:51

I don't think that prelinking is a good idea, using -W,-O1 and -W,--as-needed as compile options seems much better. At least compare the performance of the two.

That said, to add environment variables globally make an executable file in /etc/profile.d/.

darkcoder · 2005-04-22 19:25:08

what I read is that the process I found for xorg just compile with fPIC the libraries, but not the whole xorg. Right now my xorg is working without problems with the patches included, and when prelinking the libGL.so errors are now gone.

darkcoder · 2005-04-23 04:46:34

Maybe this list is far from complete, but you'll got an idea.

I have a full KDE and GNOME desktops installed, and when prelinking my system got some errors during the process. The errors right now points to the following packages (in no particular order) libtiff (-fPIC gcc flag)
xorg (patch needed)
db (-fPIC gcc flag)
libcap (patch needed)
libpcap **(still unresolved)
gettext (-fPIC gcc flag)
imlib2 (-fPIC gcc flag)
ffmpeg **(still unresolved)
The packages include in parentesis the way I was able to make them prelinkable. those with the -fPIC gcc flag comment means that in order to fix them, a simple CFLAG declaration in the PKGBUILD is needed. The declaration itself is

export CFLAGS="$CFLAGS -fPIC"

, and can be added just before the ./configure of their packages specified above.

Xorg and libcap needs patches to enable the fPIC linking.

I will try to provide during the weekend a new Wiki or tutorial and include the patches, or submit them to Arch developers.

Pajaro · 2005-04-23 11:19:53

if your finish having it working you could post a comparison of running kde and gnome with prelink and without. You could also experiment with gcc 4 new optimitzation that kde 3.4 already implements.

mxcl · 2005-04-23 14:01:45

You can remove the libGL dependence of KDE stuff by compiling Qt with the dlopen GL configure flag. This is what I have done. The GL link requirement is earlier in the chain than KDE.

Personally I think this should definitely be done by default by Arch as there are no disadvantages, but at least two advantages (prelinking/llinking is easier and startup will be minutely faster).

darkcoder · 2005-04-23 16:55:48

The only issue I had with KDE is that prelinking supposed to replace the kdeinit execution, and as found on Gentoo prelink guide and LFS there is a definition that must be done KDE_IS_PRELINKED or something like that. The problem is that I put it on rc.local and on one service I added that also re-enable hdparm optimizations, and is apparently either ignored, or even worst reset. Because when I open KDE, and check the value of the definition is like it has never been set.

Recently found a suggestion to add it at the begining of startkde. I will test it and post results.

If anyone can tell me how bench the startup time of apps, I gladly do the test.

i3839 · 2005-04-23 17:39:11

darkcoder wrote:

If anyone can tell me how bench the startup time of apps, I gladly do the test.

You'd be surprised how accurate a simple stopwatch is.

Main thing is that you keep your file cache consistent. So either all files cached in memory, and measure then, or all not cached and also take in account disk seeking and reading.

mxcl · 2005-04-25 12:56:38

Here's how to gauge the prelink effect:

DISPLAY="" time konqueror

Do it with a prelinked copy and non prelinked copy of the same app and see the difference in time. Basically this test causes konqueror to bail very soon after main() starts execution, so almost all of the measured time is the dynamic linking time. Of course this doesn't tell you the difference in startup time, only the difference in link time. But that's all that prelinking can improve anyway.

I generally found that with GCC 3.4 I could get a 200ms boost to startup time. It used to be a lot more significant than that. With SuSE 9.0 I cut startup times in half. With modern build toolchains and Arch, prelinking seems a lot less worth it.

Apparently the hidden visibility changes to GCC should make prelinking unecessary. I personally think prelinking should still help plenty with large libraries like Qt and KDElibs, but I am no expert, this is just an educated guess.

The biggest improvements I found with a prelinked system was startup time for Qt Designer and Qt Assistant. Both these went from several second startups to less than 0.5s.

I had one major issue with prelinking, namely I couldn't manage to compile a PAM that was safe to prelink against. It needed fPIC but even when I added this flag I couldn't prelink against it. I cannot say if this impacted the maximum performance I got out of prelinking.[/code]

nightfrost · 2005-06-06 16:25:05

darkcoder; I can't seem to find the patch necessary for libcap. Do you think you could post a link here, perhaps? And thank for the info on which packages other then xorg needed recompiling

Dreameen · 2005-06-21 11:36:20

I also can't find the patches for libcap and I have one additional question concerning Kde. Do all the kde packages need to be recompiled with --with-pic flag in order to successfully prelink Kde??

Gullible Jones · 2005-06-21 16:39:10

Tried it once, on my old box... Dunno about the actual decrease in startup times, but it was definitely not anything the user would notice.

Also, I might add that QT apps seem to dislike prelinking...

Dreameen · 2005-06-21 18:16:56

I'm using prelink atm and have no problems with it whatsoever. What qt apps are you reffering to?

My impressions after trying prelink, well it's kind of overhyped. In some kde applications the difference can be noticed, but it's nothing big really. Anyway I'm going to stick with it as long as it doesn't cause any problems to my arch system.

keevn7 · 2005-06-21 19:31:22

I tried prelink about a year ago while I was in Gentoo and did not see enough improvement to consider more than hype. Then again, I did not use KDE nor do I use it now. Cheers.

Arch Linux

#1 2005-04-22 04:54:59

Prelinking

#2 2005-04-22 05:04:20

Re: Prelinking

#3 2005-04-22 07:58:45

Re: Prelinking

#4 2005-04-22 08:18:18

Re: Prelinking

#5 2005-04-22 10:44:44

Re: Prelinking

#6 2005-04-22 11:29:44

Re: Prelinking

#7 2005-04-22 17:45:18

Re: Prelinking

#8 2005-04-22 17:47:51

Re: Prelinking

#9 2005-04-22 17:54:24

Re: Prelinking

#10 2005-04-22 18:16:51

Re: Prelinking

#11 2005-04-22 19:25:08

Re: Prelinking

#12 2005-04-23 04:46:34

Re: Prelinking

#13 2005-04-23 11:19:53

Re: Prelinking

#14 2005-04-23 14:01:45

Re: Prelinking

#15 2005-04-23 16:55:48

Re: Prelinking

#16 2005-04-23 17:39:11

Re: Prelinking

#17 2005-04-25 12:56:38

Re: Prelinking

#18 2005-06-06 16:25:05

Re: Prelinking

#19 2005-06-21 11:36:20

Re: Prelinking

#20 2005-06-21 16:39:10

Re: Prelinking

#21 2005-06-21 18:16:56

Re: Prelinking

#22 2005-06-21 19:31:22

Re: Prelinking

Board footer