GCC vs. LLVM/clang executable performance. [CLOSED]

rossmcd · 2012-06-01 15:17:37

I know these tests may be a bit crude, but they are consistent.

I have written a small C program which loops and prints from 1000000 to 0.

#include<stdio.h>
int main(void)
{
      for ( int x = 1000000; x >= 0; x-- )
     {
         printf("%d", x);
     }
}

I compiled it with GCC:

$ gcc test.c -o gcc

And with LLVM/clang:

$ clang test.c -o clang

RESULTS:

I ran the test (time <./compiler>):

$ time ./gcc
....... A lot of numbers ......
real	0m10.726s
user	0m0.163s
sys	0m0.070s

$ time ./clang
....... A lot of numbers ......
real	0m7.386s
user	0m0.183s
sys	0m0.047s

This seems to suggest that clang produces faster executables than GCC. I repeated the tests and it produced very similar results.

Is this an actual result or just a freak occurance?

Last edited by rossmcd (2012-06-01 17:43:01)

ewaller · 2012-06-01 15:37:59

Not much of a benchmark. It could be real, I've seen other compilers (Intel) produce much faster code. Have you played with the optimization switches for gcc? Are the two compilers both being told to optimize for speed?

Grinch · 2012-06-01 17:11:41

It's absolutely worthless from a compiler benchmark standpoint as you don't even set a -On optimization level of atleast -O2, preferably -O3. When no optimization flag is used GCC defaults to no optimization (-O0). In this extremely simple test I doubt there can be much if any difference between the compilers though.

As for GCC vs Clang/LLVM, I do quite alot of benchmarking and have tried both (and ICC, Ekopath, Open64 etc) quite extensively and GCC typically generates 5-15% faster code on my core i5/i7 machines than clang/llvm at -O3/-Ofast. Note that I primarily benchmark real-world programs, like p7zip, x264, emulators, renderers etc, rather than synthethic benchmarks.

rossmcd · 2012-06-01 17:43:13

Closed

Trilby · 2012-06-01 17:50:10

Er ... not really closed.

Perhaps your reception wasn't warm. It's a good idea to test out such things, and it's great that you want to contribute to that. I applaud you for that. However, there are many much more robust benchmarks out there - no need to reinvent the wheel (and make it square).

On point of note is that you say you repeated the test many times, but this is the wrong sort of replicate. If I find one woman who is 2 meters tall and one man who is 1.8 meters tall I can't really conclude anything about gender and height. But I could say "I measured each of them several times, and their height was consistent", this also doesn't contribute anything more: I'd still be measuring the same two people. My sample size would still be ONE. (In statistics they give this a fancy name of "pseudoreplication"; it happens far more than you'd like to know in "respected" published research).

Your results are undoubtedly real. But since they are a sample of one, it would be premature to infer anything about these compilers based on that one sample.

Edit: typo. I'm sure there are more.

Last edited by Trilby (2012-06-01 17:51:06)

Grinch · 2012-06-01 19:23:02

Yes I would like to add that I in no way wanted to belittle your interest in benchmarking, particularly since I am myself someone who is seriously interested in compiler optimization technology.

This test however (apart from not actually measuring 'optimization' due to omitting an -Ox flag) is a poor candidate for compiler optimization comparison. It's one straight up countdown loop, one variable (which won't stress the register allocator in any way), and the only 'expensive' part of this benchmark is a call to an external function (printf). As such it's woefully inadequate as something to draw conclusions from.

Arch Linux

#1 2012-06-01 15:17:37

GCC vs. LLVM/clang executable performance. [CLOSED]

#2 2012-06-01 15:37:59

Re: GCC vs. LLVM/clang executable performance. [CLOSED]

#3 2012-06-01 17:11:41

Re: GCC vs. LLVM/clang executable performance. [CLOSED]

#4 2012-06-01 17:43:13

Re: GCC vs. LLVM/clang executable performance. [CLOSED]

#5 2012-06-01 17:50:10

Re: GCC vs. LLVM/clang executable performance. [CLOSED]

#6 2012-06-01 19:23:02

Re: GCC vs. LLVM/clang executable performance. [CLOSED]

Board footer