FYI: Phoronix GCC vs. LLVM-GCC benchmarks

http://www.phoronix.com/scan.php?page=article&item=apple_llvm_gcc&num=1

Regards,
Stefano

Unfortunately, they don't specify what flags are used, what architecture is compiled for etc. It's entirely possible that they are accidentally compiling the llvm-gcc binaries for x86-32 and the gcc ones for x86-64 for example.

If anyone has time to try to reproduce this, it would definitely be useful!

-Chris

screw that site, its useless info run by a linux gnu zealot.

I wouldn't be so quick to dismiss the numbers. For example, I know llvm-gcc does significantly worse on LAME. On the other hand, the dhrystone, md5, and blowfish numbers are out of line with my expectation.

It would be nice if someone can perform the tests independently and file bugzilla reports.

Thanks,

Evan

Well, being a Linux GNU zealot doesn't invalidate numbers. What does
invalidate is that he has no numbers!

He just fired a few applications and counted the seconds, this is
utterly pointless. No reference to how many times he ran, standard
deviations, possible effects of OS cache, compilation parameters, etc,
etc. There simply isn't anything in there. It even lack text in
between the charts, a conclusion, and... well... everything.

"Ignore that site", would also be more polite... :wink:

cheers,
--renato

Reclaim your digital rights, eliminate DRM, learn more at
http://www.defectivebydesign.org/what_is_drm

2009/9/14 Edward O’Callaghan <eocallaghan@auroraux.org>:

screw that site, its useless info run by a linux gnu zealot.

Well, being a Linux GNU zealot doesn’t invalidate numbers. What does
invalidate is that he has no numbers!

He just fired a few applications and counted the seconds, this is
utterly pointless. No reference to how many times he ran, standard
deviations, possible effects of OS cache, compilation parameters, etc,
etc. There simply isn’t anything in there. It even lack text in
between the charts, a conclusion, and… well… everything.

“Ignore that site”, would also be more polite… :wink:

Nevertheless, the site has a large audience. From a publicity perspective, it may be worthwhile to work with the author to get more representative numbers.

Keir

Hi everyone,
I have replayed the “unix benchmark v3.6 dhrystone 2” test.

You can find the archive of the test here : http://www.phoronix-test-suite.com/benchmark-files/byte-benchmark-1.tar.gz
To play the test on gcc :
make
./Run dhry2
To play the test on llvm-gcc :
Replace in Makefile : CC=gcc by CC=llvm-gcc
in Run : CC=gcc by CC=llvm-gcc
make
./Run dhry2

Some information on the test platform :

cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=9.04
DISTRIB_CODENAME=jaunty
DISTRIB_DESCRIPTION=“Ubuntu 9.04”

uname -a
Linux zaraki 2.6.28-15-generic #49-Ubuntu SMP Tue Aug 18 18:40:08 UTC 2009 i686 GNU/Linux

/proc/cpuinfo
dual core Intel(R) Core™2 CPU X6800 @ 2.93GHz (cache size 4096)

llvm-gcc -v
Using built-in specs.
Target: i686-pc-linux-gnu
Configured with: …/llvm-gcc4.2-2.5.source/
configure --prefix=/mounts/zion/disks/0/localhome/tbrethou/2.5/prerelease2/llvm-gcc/obj/…/install --program-prefix=llvm- --enable-llvm=/localhome/tbrethou/2.5/prerelease2/llvm-2.5/ --disable-bootstrap --enable-languages=c,c++,fortran
Thread model: posix
gcc version 4.2.1 (Based on Apple Inc. build 5636) (LLVM build 2.5)
(Binary from official x86 2.5 release)

gcc -v
Using built-in specs.
Target: i486-linux-gnu
Configured with: …/src/configure -v --with-pkgversion=‘Ubuntu 4.3.3-5ubuntu4’ --with-bugurl=file:///usr/share/doc/gcc-4.3/README.Bugs --enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr --enable-shared --with-system-zlib --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --enable-nls --with-gxx-include-dir=/usr/include/c++/4.3 --program-suffix=-4.3 --enable-clocale=gnu --enable-libstdcxx-debug --enable-objc-gc --enable-mpfr --enable-targets=all --with-tune=generic --enable-checking=release --build=i486-linux-gnu --host=i486-linux-gnu --target=i486-linux-gnu
Thread model: posix
gcc version 4.3.3 (Ubuntu 4.3.3-5ubuntu4)

The test displays its results in form of a number with unit seems to be “loops per seconds”. So it’s logical to say that the bigger is better.
I run the test 10 times for each compiler (for an approximate duration of 20 minutes each compiler).

Results:
LLVM average : 10696687.14
GCC average : 9563046.99
LLVM is on average 10.58% faster than GCC on this test.

If you need more informations or if you think my test was not correctly made… ping me. :slight_smile:

Here a quick report on the test. (shows individual result of each test) :


LLVM GCC Difference Difference %





Run 1 10720897.7 9875431 845466.7 7.89
Run 2 10411474.6 9666138.5 745336.1 7.16
Run 3 10580705.2 9265904.6 1314800.6 12.43
Run 4 10746408 9539499.9 1206908.1 11.23
Run 5 10818845.5 9766210.7 1052634.8 9.73
Run 6 10904404.3 10034620.8 869783.5 7.98
Run 7 10861482.8 9028718.6 1832764.2 16.87
Run 8 10480568.5 9568849.4 911719.1 8.7
Run 9 10857968.2 9306331.6 1551636.6 14.29
Run 10 10584116.6 9578764.8 1005351.8 9.5





Average 10696687.14 9563046.99 1133640.15 10.58
Standard deviation 172714.92 300445.48 345992.26 3.13

Cheers,
Olivier.

Different targets. It's no surprise 486 code would perform much worse than
i686 code. These compilers need to target the same architecture. Ideally the
Makefile would include appropriate -march/-mtune/whatever the gcc architecture
switch of the day is.

All test reports should also include switches passed to the compiler for each
run.

Right now this isn't a fair test. It's practically meaningless.

Remember, the goal here isn't to show how great LLVM is. It's to get an
honest assessment of where we are at. Phoronix did us a big favor. Getting
more details about his tests would help us even more.

For one thing, we (Cray) have a number of LLVM enhancements geared toward
Opteron that make a big difference in performance. I'm planning to contribute
much of this to our open source community. So there is definitely room for
improvement to LLVM. The biggest mistake we can make is to think we're the
best.

                                 -Dave

Wow, that's beautiful: and thanks

It’s exactly what I’m trying to do. I’m not even a LLVM contributor, so I don’t feel responsible for a good or bad performances of LLVM. :slight_smile:
Anyway, thanks for pointing this target differences. After some hours at recompiling gcc, here we are :

Some information on the test platform :

cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=9.04
DISTRIB_CODENAME=jaunty
DISTRIB_DESCRIPTION=“Ubuntu 9.04”

uname -a
Linux zaraki 2.6.28-15-generic #49-Ubuntu SMP Tue Aug 18 18:40:08 UTC 2009 i686 GNU/Linux

/proc/cpuinfo
dual core Intel(R) Core™2 CPU X6800 @ 2.93GHz (cache size 4096)

llvm-gcc -v
Using built-in specs.
Target: i686-pc-linux-gnu
Configured with: …/llvm-gcc4.2-2.5.source/configure --prefix=/mounts/zion/disks/0/localhome/tbrethou/2.5/prerelease2/llvm-gcc/obj/…/install --program-prefix=llvm- --enable-llvm=/localhome/tbrethou/2.5/prerelease2/llvm-2.5/ --disable-bootstrap --enable-languages=c,c++,fortran
Thread model: posix
gcc version 4.2.1 (Based on Apple Inc. build 5636) (LLVM build 2.5)
(Binary from official x86 2.5 release)

gcc -v
Using built-in specs.
Target: i686-pc-linux-gnu
Configured with: ./configure --prefix=/usr/local
Thread model: posix
gcc version 4.4.1 (GCC)

The compilation line are :
gcc -o bin src.c -O3
llvm-gcc -o bin src.c -O3

Results :
LLVM average : 13553161.95
gcc average : 14624441.45
Average : gcc is better with 8%

The full report :
“LLVM” “GCC” “Difference” “Difference %”

“Run 1” 13842616 14693115.8 -850499.8 -6.14
“Run 2” 13773202.9 14785734.9 -1012532 -7.35
“Run 3” 13671309.3 14985114 -1313804.7 -9.61
“Run 4” 12916939 14640392.9 -1723453.9 -13.34
“Run 5” 13572133.3 14425575 -853441.7 -6.29
“Run 6” 13810096.5 14694240.6 -884144.1 -6.4
“Run 7” 13427435.8 14695094.6 -1267658.8 -9.44
“Run 8” 14056659.9 14468968.4 -412308.5 -2.93
“Run 9” 13699125.3 14373360.9 -674235.6 -4.92
"Run 10 12762101.5 14482817.4 -1720715.9 -13.48

“Average” 13553161.95 14624441.45 -1071279.5 -7.99
“Standard deviation” 412810.77 187898.68 430895.31 3.46

Is there again something wrong on this test ?

Olivier.

Hello, Oliver

The compilation line are :
gcc -o bin src.c -O3
llvm-gcc -o bin src.c -O3

I'm not quite sure which target processor the compiler will optimize
code for. I'd suggest adding explicit -march=core2 here. Also, could
you please do a separate run with O2, not O3.

Thanks!

    Remember, the goal here isn't to show how great LLVM is. It's to
    get an
    honest assessment of where we are at. Phoronix did us a big
    favor. Getting
    more details about his tests would help us even more.

It's exactly what I'm trying to do. I'm not even a LLVM contributor,
so I don't feel responsible for a good or bad performances of LLVM. :slight_smile:
Anyway, thanks for pointing this target differences. After some hours
at recompiling gcc, here we are :

Some information on the test platform :

cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=9.04
DISTRIB_CODENAME=jaunty
DISTRIB_DESCRIPTION="Ubuntu 9.04"

uname -a
Linux zaraki 2.6.28-15-generic #49-Ubuntu SMP Tue Aug 18 18:40:08 UTC
2009 i686 GNU/Linux

/proc/cpuinfo
dual core Intel(R) Core(TM)2 CPU X6800 @ 2.93GHz (cache size 4096)

llvm-gcc -v
Using built-in specs.
Target: i686-pc-linux-gnu
Configured with: ../llvm-gcc4.2-2.5.source/configure
--prefix=/mounts/zion/disks/0/localhome/tbrethou/2.5/prerelease2/llvm-gcc/obj/../install
--program-prefix=llvm-
--enable-llvm=/localhome/tbrethou/2.5/prerelease2/llvm-2.5/
--disable-bootstrap --enable-languages=c,c++,fortran
Thread model: posix
gcc version 4.2.1 (Based on Apple Inc. build 5636) (LLVM build 2.5)
(Binary from official x86 2.5 release)

gcc -v
Using built-in specs.
Target: i686-pc-linux-gnu
Configured with: ./configure --prefix=/usr/local
Thread model: posix
gcc version 4.4.1 (GCC)

Phoronix tested with gcc 4.2.1 (Apple's).

The compilation line are :
gcc -o bin src.c -O3
llvm-gcc -o bin src.c -O3

The Makefile uses -O by default, perhaps that is what Phoronix used too.

Results :
LLVM average : 13553161.95
gcc average : 14624441.45
Average : gcc is better with 8%

The full report :
        "LLVM" "GCC" "Difference" "Difference %"

"Run 1" 13842616 14693115.8 -850499.8 -6.14
"Run 2" 13773202.9 14785734.9 -1012532 -7.35
"Run 3" 13671309.3 14985114 -1313804.7 -9.61
"Run 4" 12916939 14640392.9 -1723453.9 -13.34
"Run 5" 13572133.3 14425575 -853441.7 -6.29
"Run 6" 13810096.5 14694240.6 -884144.1 -6.4
"Run 7" 13427435.8 14695094.6 -1267658.8 -9.44
"Run 8" 14056659.9 14468968.4 -412308.5 -2.93
"Run 9" 13699125.3 14373360.9 -674235.6 -4.92
"Run 10 12762101.5 14482817.4 -1720715.9 -13.48

"Average" 13553161.95 14624441.45 -1071279.5 -7.99
"Standard deviation" 412810.77 187898.68
430895.31 3.46

Is there again something wrong on this test ?

No, it seems fine.

I get different results on x86-64 though (with -O), the difference is
very small (4% with 3% stddev, or 3% with 1.5% stddev).

Maybe you should try to reproduce one of the other benchmarks that show
a higher difference, according to phoronix JOhn the Ripper shown 40%
performance drop, there was also a huge drop for blowfish.

  GCC 4.4.1 GCC 4.2.4 LLVM-GCC 2.5 LLVM-GCC 2.6 GCC-4.4/LLVM2.5
GCC-4.4/LLVM2.6 GCC-4.4/GCC-4.2

  144663277 139072037 154747143 151467440 0.935 0.955 1.040

  144413249 139042455 150949846 150322055 0.957 0.961 1.039

  144915166 138131283 150630261 150650540 0.962 0.962 1.049

  145602711 138587982 154993781 147281539 0.939 0.989 1.051

  145547868 139224734 150650813 147998859 0.966 0.983 1.045

  145719970 139224734 144055081 152474829 1.012 0.956 1.047
average 145143706.83 138880537.5 151004487.5 150032543.67 0.962
0.968 1.045
stdev 551855.54 435155.79 3961926.31 2009642.82 0.027 0.015 0.005

Using built-in specs.
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Debian 4.4.1-4'
--with-bugurl=file:///usr/share/doc/gcc-4.4/README.Bugs
--enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr
--enable-shared --enable-multiarch --enable-linker-build-id
--with-system-zlib --libexecdir=/usr/lib --without-included-gettext
--enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.4
--program-suffix=-4.4 --enable-nls --enable-clocale=gnu
--enable-libstdcxx-debug --enable-mpfr --enable-objc-gc
--with-arch-32=i486 --with-tune=generic --enable-checking=release
--build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 4.4.1 (Debian 4.4.1-4)

llvm2.5:
Using built-in specs.
Target: x86_64-unknown-linux-gnu
Configured with: ../llvm-gcc-4.2/configure --program-prefix=llvm-
--enable-llvm=/home/edwin/llvm2.5/obj --enable-languages=c,c++,fortran
Thread model: posix
gcc version 4.2.1 (Based on Apple Inc. build 5636) (LLVM build)

llvm2.6pre1:
Using built-in specs.
Target: x86_64-unknown-linux-gnu
Configured with: ../llvm-gcc4.2-2.6.source/configure
--program-prefix=llvm- --enable-languages=c,c++,fortran
--enable-llvm=/home/edwin/llvm2.6/obj/
Thread model: posix
gcc version 4.2.1 (Based on Apple Inc. build 5649) (LLVM build)

Using built-in specs.
Target: x86_64-linux-gnu
Configured with: ../src/configure -v
--enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr
--enable-shared --with-system-zlib --libexecdir=/usr/lib
--without-included-gettext --enable-threads=posix --enable-nls
--with-gxx-include-dir=/usr/include/c++/4.2 --program-suffix=-4.2
--enable-clocale=gnu --enable-libstdcxx-debug --enable-objc-gc
--enable-mpfr --with-tune=generic --enable-checking=release
--build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 4.2.4 (Debian 4.2.4-6)

Best regards,
--Edwin

Some additional tests :

With : -O2 -march=core2 for both gcc and llvm-gcc : LLVM is better with 10%.

LLVM GCC Difference Difference %

Run 1 13771597.2 13105010.6 666586.6 4.84
Run 2 13813420.8 12536327.1 1277093.7 9.25
Run 3 13769573.8 12124207.3 1645366.5 11.95
Run 4 13883222.6 12794159.1 1089063.5 7.84
Run 5 13890414.6 12105946 1784468.6 12.85
Run 6 13784627.3 11648245 2136382.3 15.5
Run 7 13920748.9 11781692.4 2139056.5 15.37
Run 8 13816843.9 12460451.3 1356392.6 9.82
Run 9 13864682.1 12959671.4 905010.7 6.53
Run 10 13849863.4 12964021.5 885841.9 6.4

Average 13836499.46 12447973.17 1388526.29 10.03
Standard deviation 53189.13 515638.56 522400.98 3.77

With : -O3 -march=core2 for both compiler : GCC is better with 5.75%

LLVM GCC Difference Difference %

Run 1 13790728.3 14902550.9 -1111822.6 -8.06
Run 2 13840499.3 14759093.2 -918593.9 -6.64
Run 3 13975018.5 14545983.5 -570965 -4.09
Run 4 13853799.5 15126446.4 -1272646.9 -9.19
Run 5 13993633.5 14824100.4 -830466.9 -5.93
Run 6 13903645.3 12768793.8 1134851.5 8.16
Run 7 13816401.7 15003783.8 -1187382.1 -8.59
Run 8 13735232.8 14976854.3 -1241621.5 -9.04
Run 9 13902380.7 14778159.5 -875778.8 -6.3
Run 10 13936761.8 15022553.5 -1085791.7 -7.79

Average 13874810.14 14670831.93 -796021.79 -5.75
Standard deviation 82443.34 688446.69 712123.9 5.14

Olivier.

2009/9/16 Török Edwin <edwintorok@gmail.com>

That was pretty much what I was expecting from the article... numbers.
It doesn't matter who's best, you can't be best in all areas, but
profiling must be done right. The standard deviation is, at least, one
order of magnitude lower than the difference between the two averages,
which means the measurements have a meaning and express some reality.

There is little value in the results on the article. People had to
re-do it properly again, anyway...

Shouldn't it be part of the standard release? I mean, add profiling as
an automated task before every big release, comparing to previous
versions of LLVM and other important compilers. Not to be waving the
results about, but to know the weakness and work on them on the next
release. It might take a while to build such infrastructure, but it's
a good thing to do, I guess.

cheers,
--renato

Reclaim your digital rights, eliminate DRM, learn more at
http://www.defectivebydesign.org/what_is_drm

I have run the john the ripper test.
I have used the official archive (same version as phoronix) from http://www.openwall.com/john/g/john-1.7.3.1.tar.bz2

To build with llvm-gcc, replace the line CC = gcc with CC = llvm-gcc.
I have used the following command to build : make clean linux-x86-sse2 (seems to be the best on x86-32)
The makefile invocation is “gcc -c -Wall -O2 -fomit-frame-pointer -funroll-loops src.c” and “llvm-gcc -c -Wall -O2 -fomit-frame-pointer -funroll-loops src.c”.

I have runned 10 tests : “./john --test” in the run directory. It does 6 benchmarking on various algorithms :

Benchmarking: Traditional DES [128/128 BS SSE2]… DONE

LLVM GCC Difference Difference %

Run 1 2371 2358 13 0.55
Run 2 2499 2497 2 0.08
Run 3 2489 2487 2 0.08
Run 4 2305 2504 -199 -8.63
Run 5 2499 2445 54 2.16
Run 6 2404 2503 -99 -4.12
Run 7 2482 2502 -20 -0.81
Run 8 2479 2475 4 0.16
Run 9 2463 2489 -26 -1.06
Run 10 2484 2483 1 0.04

Average 2447.5 2474.3 -26.8 -1.15
Std dev 65.69 44.5 71.81 3.07

==> Similar results

Benchmarking: BSDI DES (x725) [128/128 BS SSE2]… DONE

LLVM GCC Difference Difference %

Run 1 72584 81280 -8696 -11.98
Run 2 76620 79795 -3175 -4.14
Run 3 79820 75264 4556 5.71
Run 4 76339 81484 -5145 -6.74
Run 5 81484 76441 5043 6.19
Run 6 80742 81433 -691 -0.86
Run 7 81510 79104 2406 2.95
Run 8 81049 79872 1177 1.45
Run 9 80409 81100 -691 -0.86
Run 10 80204 80921 -717 -0.89

Average 79076.1 79669.4 -593.3 -0.92
Std dev 2937.56 2181.15 4262.03 5.59

==> Similar results

Benchmarking: FreeBSD MD5 [32/32]… DONE

LLVM GCC Difference Difference %

Run 1 7552 8009 -457 -6.05
Run 2 7739 7724 15 0.19
Run 3 7997 7696 301 3.76
Run 4 8038 8041 -3 -0.04
Run 5 7474 7938 -464 -6.21
Run 6 7871 8078 -207 -2.63
Run 7 7884 7980 -96 -1.22
Run 8 7870 8025 -155 -1.97
Run 9 7989 8046 -57 -0.71
Run 10 7986 7989 -3 -0.04

Average 7840 7952.6 -112.6 -1.49
Std dev 193.87 133.78 227.92 2.98

==> Similar results

Benchmarking: OpenBSD Blowfish (x32) [32/32]… DONE

LLVM GCC Difference Difference %

Run 1 494 495 -1 -0.2
Run 2 457 485 -28 -6.13
Run 3 492 474 18 3.66
Run 4 494 492 2 0.4
Run 5 486 469 17 3.5
Run 6 491 495 -4 -0.81
Run 7 495 493 2 0.4
Run 8 490 490 0 0
Run 9 493 494 -1 -0.2
Run 10 493 492 1 0.2

Average 488.5 487.9 0.6 0.08
Std dev 11.37 9.19 12.56 2.67

==> Similar results

Benchmarking: Kerberos AFS DES [48/64 4K MMX]… DONE

LLVM GCC Difference Difference %

Run 1 399001 403712 -4711 -1.18
Run 2 396697 377292 19405 4.89
Run 3 395520 401971 -6451 -1.63
Run 4 392396 404172 -11776 -3
Run 5 392294 376217 16077 4.1
Run 6 395571 404172 -8601 -2.17
Run 7 400128 402995 -2867 -0.72
Run 8 397516 395110 2406 0.61
Run 9 396748 403507 -6759 -1.7
Run 10 396263 403712 -7449 -1.88

Average 396213.4 397286 -1072.6 -0.27
Std dev 2497.59 11150.79 10620.52 2.69

==> Similar results

Benchmarking: LM DES [128/128 BS SSE2]… DONE

LLVM GCC Difference Difference %

Run 1 6879 11433 -4554 -66.2
Run 2 8984 12252 -3268 -36.38
Run 3 9142 12182 -3040 -33.25
Run 4 8802 12205 -3403 -38.66
Run 5 8756 11971 -3215 -36.72
Run 6 9227 12224 -2997 -32.48
Run 7 8667 12191 -3524 -40.66
Run 8 9163 11942 -2779 -30.33
Run 9 9117 12254 -3137 -34.41
Run 10 9076 12166 -3090 -34.05

Average 8781.3 12082 -3300.7 -38.31
Std dev 695.06 252.95 487.97 10.26

==> This one is interesting as gcc is better with near 40%
I have no idea why but anyone interested could take a look at LM_fmt.c which seems to define test and source for this algorithm.

Olivier.

Olivier Meurant:

I have run the john the ripper test.
I have used the official archive (same version as phoronix) from http://www.openwall.com/john/g/john-1.7.3.1.tar.bz2

To build with llvm-gcc, replace the line CC = gcc with CC = llvm-gcc.
I have used the following command to build : make clean linux-x86-sse2 (seems to be the best on x86-32)
The makefile invocation is "gcc -c -Wall -O2 -fomit-frame-pointer -funroll-loops src.c" and "llvm-gcc -c -Wall -O2 -fomit-frame-pointer -funroll-loops src.c".

I don't know what you think about this, but shouldn't it be more meaningful to make these tests with -O3? I mean, we ought to make the comparisons with the highest level of optimization available for both of the compilers. It is difficult to compare an intermediate level.

Stefano

A quick test confirms that the 40% difference stays even with an -O3 switch on both compilers.
I just “choose” -O2 as it’s the default of the original makefile.

Olivier.

Comparing -O3 (and even -O4) is interesting, but we want all optimization levels to perform better than GCC :). Lots of people use -O2 and -Os, so comparing against other compiler's -O2 and -Os levels is just as interesting as comparing -O3 vs -O3.

-Chris

Chris Lattner:

Comparing -O3 (and even -O4) is interesting, but we want all optimization levels to perform better than GCC :). Lots of people use -O2 and -Os, so comparing against other compiler's -O2 and -Os levels is just as interesting as comparing -O3 vs -O3.

My thinking was that, for instance, -02 for GCC and -02 for LLVM(-GCC) do not necessarily mean the same thing, they may be not /commensurable/.
But perhaps, my ignorance, you are saying that they _are_, that LLVM assigns the same types of optimizations as GCC to the different levels.

Regards,
Stefano

Right, we want them to be roughly comparable.

O0 -> best debug experience, fastest compile times.
O2 -> optimize without bloating the code too much and without burning *too* many cycles.
O3 -> take more time and produce fatter code to get faster code.

O1 is something of a wasteland with no clear purpose :slight_smile:

-Chris