[SOLVED] All kernel versions are panicing when compiled with gcc 4.5 ?

combuster · 2010-04-30 19:16:17

Ok, I'm not certain if this is 100% sure, so I'm asking u guys for help.

I have compiled 2.6.34-rc5 with gcc 4.4 and I had no problems at all. Since then gcc 4.5 popped up in the repo's, I've decided to play a li'l with localmodconfig. Compiled 2.6.34-rc5 - failed to boot. Compiled 2.6.34-rc5 again with slight modifications to config file - failed too. Ok I've decided to give it a rest, 2.6.34-rc6 showed up, compiled it with the same config as the rc5 wich worked when compiled with gcc 4.4 and no localmodconfig. FAILED !!!

Kernel panic is appearing very early in the bootstraping phase, right after KMS is being activated. I get message like this:

BUG: scheduling while atomic: init/1/0x00000002
1 lock held by init1:
#0 (rcu_read_lock){,+,+,+,+{, at: [<ffffffff8105603b>] __atomic_notifier_call_chain+0x0/0x90
ok not tainted 2.6.34-rc6 and then call trace wich mostly consists of scheduling timeout, i915_crtc, intel_crtc, so it suggests that something is wrong when KMS fires up.

While performing an atomic function - scheduling is supposed to be forbidden so looks like this is some sort of a deadlock. I would be very gratefull if someone report here their experience with compiling a custom kernel with gcc 4.5. Prefferably if you have a Intel IGP with KMS on and driver built into kernel image (not as module).

Test machine is Dell Vostro 1310 with 965GM, 64bit Arch...

Any information will prove valuable before this bug report goes upstream. TIA !

Last edited by combuster (2010-05-01 08:58:30)

tomk · 2010-04-30 19:32:25

I built a custom kernel with gcc 4.5 just two days ago, and it's working fine. However, I have ATI graphics.

wonder · 2010-04-30 19:35:30

also the current kernel from core is built with gcc 4.5

combuster · 2010-04-30 19:39:01

Ok, I'm going to try and bisect the bug, going to start with 2.6.33.3 because this is the working kernel in the repo's...

wonder · 2010-04-30 19:54:50

combuster wrote:

Ok, I'm going to try and bisect the bug, going to start with 2.6.33.3 because this is the working kernel in the repo's...

before doing that try to build the kernel in a clean chroot. maybe you got a broken toolchain on your system.

http://wiki.archlinux.org/index.php/Dev … ean_Chroot

combuster · 2010-04-30 20:21:33

2.6.33.3 also failed. I'm going to try and build the kernel in a clean chroot as you suggested, but if that fails also - first step would be to disable kms in the kernel and see if the problem persists. Thanks for the info...

combuster · 2010-04-30 21:02:30

Same damn thing with kernel built from clean chroot... Next - disabling kms (it will suffice to get even to init3)...

combuster · 2010-04-30 22:07:33

Disabling KMS and building intel driver as module reproduces a different kernel panic now.

init[1] segfault - then registers and physical adressess - error code 15
kernel not syncing atempting to kill init

This happens when Init 2.28 version message appears right before Juds and Aarons royalties... At the same spot as the previous panic. It mounts the root filesystem, starting the init and then dies.

Oh and if someone knows how to capture early printk's before syslog starts that would be really helpfull.

Allan · 2010-05-01 00:37:26

What CFLAGS are you using?

Anaki · 2010-05-01 06:41:36

last week, when the gcc4,5 hit core i recompiled my custom kernel with it, and had the same issue.
I tried everything, until i realised that gcc4,5 has something to do with function inlining, so i disable this in
kernel config:CONFIG_OPTIMIZE_INLINING. (it is in the kernel hacking section,at the bottom)

Now everything is working fine

combuster · 2010-05-01 08:58:06

Anaki, disabling CONFIG_OPTIMIZE_INLINING got the job done, thank you so much.

So gcc 4.5 screwed up it's inlining algorithm... Marking this as solved.

Allan · 2010-05-01 09:12:11

Well... other people can compile the kernel so it would still be good to know what CFLAGS you use. I wonder if this is related: http://bugs.archlinux.org/task/19248

combuster · 2010-05-01 09:35:41

I dont pass any custom CFLAGS during make and I don't use makepkg (i build it with make modules, make bzImage, make modules_install etc) so CFLAGS in makepkg.conf should be irrelevant in my case.

gcc -Q -march=native --help=target
The following options are target specific:
  -m128bit-long-double                [disabled]
  -m32                                [disabled]
  -m3dnow                             [disabled]
  -m3dnowa                            [disabled]
  -m64                                [enabled]
  -m80387                             [enabled]
  -m96bit-long-double                 [enabled]
  -mabi=                              
  -mabm                               [disabled]
  -maccumulate-outgoing-args          [disabled]
  -maes                               [disabled]
  -malign-double                      [disabled]
  -malign-functions=                  
  -malign-jumps=                      
  -malign-loops=                      
  -malign-stringops                   [enabled]
  -march=                             core2
  -masm=                              
  -mavx                               [disabled]
  -mbranch-cost=                      
  -mcld                               [disabled]
  -mcmodel=                           
  -mcrc32                             [disabled]
  -mcx16                              [enabled]
  -mfancy-math-387                    [enabled]
  -mfma                               [disabled]
  -mfma4                              [disabled]
  -mforce-drap                        [disabled]
  -mfp-ret-in-387                     [enabled]
  -mfpmath=                           
  -mfused-madd                        [enabled]
  -mglibc                             [enabled]
  -mhard-float                        [enabled]
  -mieee-fp                           [enabled]
  -mincoming-stack-boundary=          
  -minline-all-stringops              [disabled]
  -minline-stringops-dynamically     [disabled]
  -mintel-syntax                      [disabled]
  -mlarge-data-threshold=             
  -mlwp                               [disabled]
  -mmmx                               [disabled]
  -mmovbe                             [disabled]
  -mms-bitfields                      [disabled]
  -mno-align-stringops                [disabled]
  -mno-fancy-math-387                 [disabled]
  -mno-push-args                      [disabled]
  -mno-red-zone                       [disabled]
  -mno-sse4                           [enabled]
  -momit-leaf-frame-pointer           [disabled]
  -mpc                                
  -mpclmul                            [disabled]
  -mpopcnt                            [disabled]
  -mpreferred-stack-boundary=         
  -mpush-args                         [enabled]
  -mrecip                             [disabled]
  -mred-zone                          [enabled]
  -mregparm=                          
  -mrtd                               [disabled]
  -msahf                              [enabled]
  -msoft-float                        [disabled]
  -msse                               [disabled]
  -msse2                              [disabled]
  -msse2avx                           [disabled]
  -msse3                              [disabled]
  -msse4                              [disabled]
  -msse4.1                            [disabled]
  -msse4.2                            [disabled]
  -msse4a                             [disabled]
  -msseregparm                        [disabled]
  -mssse3                             [disabled]
  -mstack-arg-probe                   [disabled]
  -mstackrealign                      [enabled]
  -mstringop-strategy=                
  -mtls-dialect=                      
  -mtls-direct-seg-refs               [enabled]
  -mtune=                             core2
  -muclibc                            [disabled]
  -mveclibabi=                        
  -mxop                               [disabled]

mtune and march are correct in my case.

uname -a
Linux vostro 2.6.34-rc6-DELL #1 SMP PREEMPT Sat May 1 10:43:45 CEST 2010 x86_64 Intel(R) Core(TM)2 Duo CPU T5670 @ 1.80GHz GenuineIntel GNU/Linux

I saw Thomas commented on discouraging using custom CFLAGS during kernel compilation. I've never use them, sometimes I just pass -j3 if I'm in a rush but that's it.

combuster · 2010-05-01 09:51:06

Oh and just one more question. I always wondered from where gcc reads CFLAGS on arch, on gentoo it was make.conf (as I recall) but on Arch I have to pass CFLAGS as parametars during make or to change Makefile right ? I saw CFLAGS section in makepkg.conf but that's just for building with ABS right?

Allan · 2010-05-01 10:10:15

Yes, makepkg.conf is just for use in makepkg. If you want to use a set of CFLAGS all the time, export them in /etc/profile (of something like that...)

broch · 2010-05-01 15:55:23

flags in makepkg.conf (or make.conf or any profile you want) have nothing to do with kernel. You can put there whatever you want.

kernel flags are located in /usr/src/kernel_name/Makefile and anything outside is ignored.

In your case, this looks like unfortunate kernel configuration, to see if bug is gcc version related, install downgrade gcc, compile kernel, if works, then gcc 4.5 probably exposed some bug in intel driver (or simply your .config is messed up). Which would be easy to configm (and submit bug/ask at lkml)

Last edited by broch (2010-05-01 16:40:22)

combuster · 2010-05-01 16:17:39

In your case, this looks like unfortunate kernel configuration, to see if bug is gcc version related, install downgrade gcc, compile kernel, if works, then gcc 4.5 probably exposed some bug in intel driver (or simply your .config is messed up). Which would be easy to configm (and submit bug/ask at lkml)

Well everything worked with gcc 4.4 so there is no reason to downgrade. This also doesn't have to do anything with intel driver just with an option in the kernel to allow gcc to uninline functions marked as inline. GCC have inlining algorithm wich enables him to uninline these functions resulting in smaller kernel image. It worked great with gcc 4.4 and 4.3 but in 4.5 something got messed up. This is solely gcc bug not a kernel one.

I've enabled this option a long time ago for testing purposes but when gcc triggered this bug I completely forgot about it and at first thought that localmodconfig screwed up my .config, then when I've tried .config from a known working custom kernel - thought that it's an intel driver bug, afterwards I thought that gcc 4.5 was to blame or mkinitcpio-busybox had some sort of a bug - now we know what was causing all this trouble.

So this should be reported as a gcc bug upstream...

broch · 2010-05-01 16:28:28

if official Arch kernel is compiled with gcc 4.5 and it works for you, then I doubt that this is gcc problem.

combuster · 2010-05-01 17:15:10

Well you can check with zcat /proc/config.gz | grep CONFIG_OPTIMIZE_INLINING if this option is enabled in arch stock kernel. I seriously doubt that it is

attila · 2010-05-01 17:27:28

I'm not sure that this is a problem of gcc adn/or "CONFIG_OPTIMIZE_INLINING". I compiled yesterday my own kernel 2.6.33 package (with bfs scheduler and which is based on localmodconfig too) which has CONFIG_OPTIMIZE_INLINING on and it works without a problem as before with gcc 4.4. I could be better to ask on the lkml first (or too) where is the problem because you use a rc version of the next kernel. The big difference to you is that i don't use kms but at the moment i can't say how this could get affected from the CONFIG_OPTIMIZE_INLINING setting.

broch · 2010-05-01 18:04:04

combuster wrote:

Well you can check with zcat /proc/config.gz | grep CONFIG_OPTIMIZE_INLINING if this option is enabled in arch stock kernel. I seriously doubt that it is

thank God this option is disabled because it is designed for gcc 3.x:

is desirable for the gcc 3.x series of compilers. The gcc 4.x series have a rewritten inlining algorithm and enabling this option will generate a smaller kernel there.

the only thing you get is (in theory) smaller kernel. Which is as counterproductive as it is enabling CC_OPTIMIZE_FOR_SIZE from performance stand point.

Last edited by broch (2010-05-01 18:05:27)

combuster · 2010-05-01 18:13:45

thank God this option is disabled because it is designed for gcc 3.x

You got it wrong, forcing gcc to expand inline functions in every line of code where this function is called is in fact an impact on kernel image size. GCC 3.x have to do this because it doesn't have an advanced algorithm like the 4.x series wich may decide whether it will unline an inline function or not (so by any means this option is not designed for gcc 3.x but just the contrary - allowing 4.x gcc to uninline functions marked as inline). I guess that changes in this algorithm in 4.5 version lead to uninlining functions it shouldn't. One of them could be a function called in intel driver, another one could be in any other peace of kernel code - that is why it's not a kernel bug, functions marked inline suppose to stay inlined - but a gcc optimization wich worked fine so far broke.

Last edited by combuster (2010-05-01 18:15:13)

falconindy · 2010-05-01 19:15:15

The way Linus talks about inlining, gcc has never been all that smart about it: http://lwn.net/Articles/314856/

By no means do I consider myself educated on the matter, but it seems that there are far greater implications than just size when it comes to (un)inlining functions.

combuster · 2010-05-01 19:35:35

We get oopses that have a nice symbolic back-trace, and it reports an
error IN TOTALLY THE WRONG FUNCTION, because gcc "helpfully" inlined
things to the point that only an expert can realize "oh, the bug was
actually five hundred lines up, in that other function that was just
called once, so gcc inlined it even though it is huge".

Well I've experienced this pretty badly That is the whole point, when intel driver is compiled in kernel image - kernel panics with scheduling while atomic and then a nice stack trace pointing out that there is something wrong within intel_crt.c but that is not necessarily the case, it could be like Linus said a bug in totally different piece of code that got manifested later because of gcc heuristics regarding uninlining/inlining functions. When I've compiled intel driver as a module and disabled KMS, then kernel paniced at totally different place without even mentioning drm so that's when I've suspected that the call traces are totally bogus...

I just dont realise why they've even included that option as a DEFAULT back in 2.6.28 kernel versions, Bryce Harrington was ticked off because of this:

http://www.mail-archive.com/ubuntu-bugs … 89286.html

Oh and I'm no expert too on the matter, just some basic stuff...

Last edited by combuster (2010-05-01 19:36:17)

broch · 2010-05-01 19:56:41

combuster wrote:

thank God this option is disabled because it is designed for gcc 3.x
You got it wrong, forcing gcc to expand inline functions in every line of code where this function is called is in fact an impact on kernel image size. GCC 3.x have to do this because it doesn't have an advanced algorithm like the 4.x series wich may decide whether it will unline an inline function or not (so by any means this option is not designed for gcc 3.x but just the contrary - allowing 4.x gcc to uninline functions marked as inline). I guess that changes in this algorithm in 4.5 version lead to uninlining functions it shouldn't. One of them could be a function called in intel driver, another one could be in any other peace of kernel code - that is why it's not a kernel bug, functions marked inline suppose to stay inlined - but a gcc optimization wich worked fine so far broke.

nope, this is explained in kernel option function. size ins only impact on gcc 4.x.

in gcc 3.x one had for force inline, in gcc 4.x inline set by dev is honored..
gcc 3.x was "stupid" gcc 4.4 and gcc 4.5 are not

Last edited by broch (2010-05-01 20:05:46)

Arch Linux

#1 2010-04-30 19:16:17

[SOLVED] All kernel versions are panicing when compiled with gcc 4.5 ?

#2 2010-04-30 19:32:25

Re: [SOLVED] All kernel versions are panicing when compiled with gcc 4.5 ?

#3 2010-04-30 19:35:30

Re: [SOLVED] All kernel versions are panicing when compiled with gcc 4.5 ?

#4 2010-04-30 19:39:01

Re: [SOLVED] All kernel versions are panicing when compiled with gcc 4.5 ?

#5 2010-04-30 19:54:50

Re: [SOLVED] All kernel versions are panicing when compiled with gcc 4.5 ?

#6 2010-04-30 20:21:33

Re: [SOLVED] All kernel versions are panicing when compiled with gcc 4.5 ?

#7 2010-04-30 21:02:30

Re: [SOLVED] All kernel versions are panicing when compiled with gcc 4.5 ?

#8 2010-04-30 22:07:33

Re: [SOLVED] All kernel versions are panicing when compiled with gcc 4.5 ?

#9 2010-05-01 00:37:26

Re: [SOLVED] All kernel versions are panicing when compiled with gcc 4.5 ?

#10 2010-05-01 06:41:36

Re: [SOLVED] All kernel versions are panicing when compiled with gcc 4.5 ?

#11 2010-05-01 08:58:06

Re: [SOLVED] All kernel versions are panicing when compiled with gcc 4.5 ?

#12 2010-05-01 09:12:11

Re: [SOLVED] All kernel versions are panicing when compiled with gcc 4.5 ?

#13 2010-05-01 09:35:41

Re: [SOLVED] All kernel versions are panicing when compiled with gcc 4.5 ?

#14 2010-05-01 09:51:06

Re: [SOLVED] All kernel versions are panicing when compiled with gcc 4.5 ?

#15 2010-05-01 10:10:15

Re: [SOLVED] All kernel versions are panicing when compiled with gcc 4.5 ?

#16 2010-05-01 15:55:23

Re: [SOLVED] All kernel versions are panicing when compiled with gcc 4.5 ?

#17 2010-05-01 16:17:39

Re: [SOLVED] All kernel versions are panicing when compiled with gcc 4.5 ?

#18 2010-05-01 16:28:28

Re: [SOLVED] All kernel versions are panicing when compiled with gcc 4.5 ?

#19 2010-05-01 17:15:10

Re: [SOLVED] All kernel versions are panicing when compiled with gcc 4.5 ?

#20 2010-05-01 17:27:28

Re: [SOLVED] All kernel versions are panicing when compiled with gcc 4.5 ?

#21 2010-05-01 18:04:04

Re: [SOLVED] All kernel versions are panicing when compiled with gcc 4.5 ?

#22 2010-05-01 18:13:45

Re: [SOLVED] All kernel versions are panicing when compiled with gcc 4.5 ?

#23 2010-05-01 19:15:15

Re: [SOLVED] All kernel versions are panicing when compiled with gcc 4.5 ?

#24 2010-05-01 19:35:35

Re: [SOLVED] All kernel versions are panicing when compiled with gcc 4.5 ?

#25 2010-05-01 19:56:41

Re: [SOLVED] All kernel versions are panicing when compiled with gcc 4.5 ?

Board footer