You compile, I compile. Whose code is better?

schivmeister · 2008-02-29 18:37:35

The one with better hardware?

Well, it seems to be annoying me for a while. I'm pretty confident of my software setup, but it just so happens that whenever I compile a kernel it "works" worse than Arch's stock (at least in a way or two). Picture this:

cp -r /var/abs/core/base/kernel26 /var/abs/local/
cd /var/abs/local/kernel26
makepkg -ci

And the resultant kernel waits 14 seconds more at loading udev uevents (although the solution may only be as deep as /sbin/udevtrigger and /sbin/udevsettle). Furthermore, JACK implies that the audio subsystem could be better, with xruns shooting beyond 1 just on loading an application. I didn't think much of this before as this is the first time I'd compiled the stock kernel as-is with default compiler flags.

I've tested this on other binary kernels with similar results, so software compiled on other geeks' systems appear to be "better"

Oh, what am I to do dear brethren? It's just a humble dual-coal laptop running 1,600 horses and around 2,000 sheeps. What other magic do I need to master to let the milk and honey flow as they should?

Last edited by schivmeister (2008-02-29 18:42:38)

brebs · 2008-02-29 18:59:52

Set up /etc/makepkg.conf for your processor, e.g. for 64-bit:

CARCH="x86_64"
CHOST="x86_64-unknown-linux-gnu"
CFLAGS="-march=nocona -O2 -pipe"
CXXFLAGS="-march=nocona -O2 -pipe"
MAKEFLAGS="-j6"

blu3ness · 2008-02-29 19:25:57

aye, you have to specify your optimization flags in makepkg.conf

My kernel is at least 10s faster than the stock kernel.

shining · 2008-02-29 19:42:54

schivmeister wrote:

And the resultant kernel waits 14 seconds more at loading udev uevents (although the solution may only be as deep as /sbin/udevtrigger and /sbin/udevsettle).

Are you using testing? There is a regression with last udev package (arch specific issue, not udev one).
You should experience the same with arch kernel (or any kernel using modules actually).
http://bugs.archlinux.org/task/9648

For a workaround, see http://wiki.archlinux.org/index.php/Speedup_udev

skottish · 2008-02-29 19:46:38

Setting MAKEFLAGS to j > 1 can break compilation of many programs. Watch that one.

broch · 2008-02-29 20:07:00

flags, though nice will not help much:
arch default is not optimized for anything except cpu (I assume) still boots faster (as you pointed out) than your not optimized kernel. Moreover some flags are set default already.

My kernel boots faster than default, but this is caused in my opinion by the fact that custom kernel always has less stuff.

If you want to play with flags, try something kernel specific. e.g.
CONFIG_CUSTOM_CFLAGS=""
CONFIG_CUSTOM_LDFLAGS=""
CONFIG_CUSTOM_AFLAGS=""
CONFIG_CUSTOM_MAKEFLAGS=""
these should be listed in your kernel .config file

Setting MAKEFLAGS to j > 1 can break compilation of many programs. Watch that one.

I never had any problems with makeflags = -j4 or -j6
And I am using this when compiling linux or BSD kernels/apps
why this would break anything?

skottish · 2008-02-29 20:12:01

broch wrote:

And I am using this when compiling linux or BSD kernels/apps
why this would break anything?

http://bbs.archlinux.org/viewtopic.php?id=43030

shining · 2008-02-29 20:24:37

blu3ness wrote:

aye, you have to specify your optimization flags in makepkg.conf
My kernel is at least 10s faster than the stock kernel.

Using the same config?

broch · 2008-02-29 21:54:42

skottish wrote:

broch wrote:
And I am using this when compiling linux or BSD kernels/apps
why this would break anything?
http://bbs.archlinux.org/viewtopic.php?id=43030

shrug,
it is more or less like saying:
compiling is dangerous because it may fail.

I understand that some flags break code (e.g. -O3 on 64-bit), but running in parallel is not really a problem. At least never was for me for many years. Maybe I am lucky, if so still will use -j4

not to mention that users are using quite often more than one core cpus

shining · 2008-02-29 22:05:50

broch wrote:

I understand that some flags break code (e.g. -O3 on 64-bit), but running in parallel is not really a problem. At least never was for me for many years. Maybe I am lucky, if so still will use -j4

Actually it is a problem. It can be *very* confusing when it does break.
If you are aware of it, and always remember it in case of a breakage, I guess it might not be a big problem. But only in that case

brebs · 2008-02-29 22:09:41

Compiling in parallel *can* be a problem, depending on how well the Makefile is written. There's a hundred threads in the Gentoo forums about this, if you doubt it. This is why some ebuilds contain the magic switch "-j1", to force parallel builds *off*.

shining · 2008-02-29 22:14:26

brebs wrote:

Compiling in parallel *can* be a problem, depending on how well the Makefile is written. There's a hundred threads in the Gentoo forums about this, if you doubt it. This is why some ebuilds contain the magic switch "-j1", to force parallel builds *off*.

Ah yes, I learned that the hard way. I was using -j2 ages ago without problems on Gentoo, and tried to build lfs with the same setting. It didn't work out so well

skottish · 2008-02-29 22:21:33

It's not a big deal, I simply wanted to point out that if one is to leave the makeflags like that, it may cause some problems. I still use j > 1 in most cases.

dyscoria · 2008-03-01 00:23:25

brebs wrote:

Set up /etc/makepkg.conf for your processor, e.g. for 64-bit:
CARCH="x86_64"
CHOST="x86_64-unknown-linux-gnu"
CFLAGS="-march=nocona -O2 -pipe"
CXXFLAGS="-march=nocona -O2 -pipe"
MAKEFLAGS="-j6"

Can you explain why the CHOST has 'unknown' instead of 'pc'?

Also, wouldn't '-march=native' be more effective?

(just interested, as i'm looking to try and compile some stuff myself)

brebs · 2008-03-01 01:32:32

CHOST appears *nowhere* in the sourcecode for gcc or gcc-libs, after a case-insensitive search.

I just compiled darkplaces with CHOST="absolutely-useless-junk-bleh" and it still works

Most undocumented setting ever, given that it's supposed to be essential and that changing it brings bad voodoo.

As for -march, I'm not terribly bothered. When gcc 4.3 arrives, I'll be using -march=core2

schivmeister · 2008-03-01 11:50:53

I could've made myself clearer =/

That was the first time I'd compiled Arch's stock kernel from core - without modification, without custom compiler flags. I'd suspected something to be wrong when all the kernels I'd compiled since the *.24 upgrade took a significantly longer time at the uevents stage. As for why I didn't have any issue =< *.23, maybe it was made more obvious due to the larger size of newer kernels. It certainly isn't the kernel's problem either, since most of you claim to get a better-performing platform, and I expect that to be the case too. So I tested a few scenarios to try to determine the root of the cause. I normally make with 3 or 5 job servers, which isn't really a matter as we can adjust any kind of variable dynamically when needed.

CARCH="i686"
CHOST="i686-pc-linux-gnu"
CFLAGS="-march=prescott -O2 -pipe"
LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,--enable-new-dtags -Wl,--sort-common -s"
CXXFLAGS="${CFLAGS}"
MAKEFLAGS="-j3"

I presume the default Arch makepkg.conf would closely resemble that of a maintainer's environment, so to reduce possible bottlenecks as far as software is concerned, I went ahead and did everything as "vanilla" as was possible. I only tweaked the local version so that kernel itself would install alongside my primary one, which is the same standard (binary) kernel of course. I really hoped it'd function at least similarly, if not better. The sad fact is, it performs worse. The only clue I can pin-point is hardware, or a cleverly mistaken/overlooked configuration somewhere.

Shining: Say, has rc.sysinit been updated recently (I don't see pacnew/pacsave files)? Currently neither kernel26 nor any binary kernel exhibits such behaviour. I suppose 3-4 seconds isn't much, but 14-16 is. The latter is the case for the compiled ones.

Last edited by schivmeister (2008-03-01 12:19:25)

shining · 2008-03-01 12:01:34

schivmeister wrote:

I could've made myself clearer =/
That was the first time I'd compiled Arch's stock kernel from core - without modification, without custom compiler flags. I'd suspected something to be wrong when all the kernels I'd compiled since the *.24 upgrade took a significantly longer time at the uevents stage.

You could also have answered to what I said about udev events, so that we could at least know if you are concerned by that problem or not.

schivmeister · 2008-03-01 12:27:46

I was late in editing ^^

So this is only the case for compiled kernels on this system, that's all I can tell at this stage.

Arch Linux

#1 2008-02-29 18:37:35

You compile, I compile. Whose code is better?

#2 2008-02-29 18:59:52

Re: You compile, I compile. Whose code is better?

#3 2008-02-29 19:25:57

Re: You compile, I compile. Whose code is better?

#4 2008-02-29 19:42:54

Re: You compile, I compile. Whose code is better?

#5 2008-02-29 19:46:38

Re: You compile, I compile. Whose code is better?

#6 2008-02-29 20:07:00

Re: You compile, I compile. Whose code is better?

#7 2008-02-29 20:12:01

Re: You compile, I compile. Whose code is better?

#8 2008-02-29 20:24:37

Re: You compile, I compile. Whose code is better?

#9 2008-02-29 21:54:42

Re: You compile, I compile. Whose code is better?

#10 2008-02-29 22:05:50

Re: You compile, I compile. Whose code is better?

#11 2008-02-29 22:09:41

Re: You compile, I compile. Whose code is better?

#12 2008-02-29 22:14:26

Re: You compile, I compile. Whose code is better?

#13 2008-02-29 22:21:33

Re: You compile, I compile. Whose code is better?

#14 2008-03-01 00:23:25

Re: You compile, I compile. Whose code is better?

#15 2008-03-01 01:32:32

Re: You compile, I compile. Whose code is better?

#16 2008-03-01 11:50:53

Re: You compile, I compile. Whose code is better?

#17 2008-03-01 12:01:34

Re: You compile, I compile. Whose code is better?

#18 2008-03-01 12:27:46

Re: You compile, I compile. Whose code is better?

Board footer