Clang versus GCC speed

Hi,

Has anyone done any tests comparing gcc compilation speed to clang recently? I just did some tests with Objective-C programs and found clang much slower than GCC 4.2, with an optimized build of LLVM and Clang and a stock build of GCC.

David

-- Sent from my Apple II

Has anyone done any tests comparing gcc compilation speed to clang
recently?

Yes.

I just did some tests with Objective-C programs and found
clang much slower than GCC 4.2, with an optimized build of LLVM and
Clang and a stock build of GCC.

This isn't what we're seeing. We're seeing Clang being consistently faster than GCC 4.2.

Are you certain that you built LLVM and Clang in Release-Asserts? Debug builds are horribly slow, and Release builds are still a bit slower than Release-Asserts builds.

  - Doug

Just a quick test gave me this on an Objective-C project:

Clang (r89342) (Note that I replaced Xcode's clang binaries by a svn version so it uses it when I pass com.apple.compilers.llvm.clang.1_0)
time xcodebuild -configuration Release GCC_VERSION=com.apple.compilers.llvm.clang.1_0 ARCHS=i386
  68.314u 16.219s 1:07.79 124.6% 0+0k 17+476io 0pf+0w

GCC-4.2:
time xcodebuild -configuration Release ARCHS=i386
  127.909u 45.820s 2:02.94 141.3% 0+0k 0+639io 0pf+0w

-- Jean-Daniel

I'm guessing that you're using precompiled headers. Clang usually tops out at about 30% faster than GCC without precompiled headers, with much better performance when we're using precompiled headers.

  - Doug

You're right. I'm using a precompile header that imports <Cocoa/Cocoa.h>.

-- Jean-Daniel

I was using the wrong binary. Running the tests again, there's about a 10% variation in speeds each time I run clang and gcc, but they both end up being approximately the same speed. Using -fsyntax-only on GNUstep's Cocoa.h takes slightly longer with clang than GCC on average, although the fastest result for clang is slightly better than the slowest result for gcc.

Are your tests relative to Apple's GCC? It's a lot slower than FSF GCC for Objective-C, at least - not sure why, but FSF GCC can usually compile a couple of files before Apple GCC reports the first warning in one.

With a Release-Asserts build, clang is very slightly slower than the GCC 4.2.1 build included with the FreeBSD 8 base system for Objective-C (although it does produce warnings that GCC doesn't produce and supports a few more language features).

I took the preprocessed Cocoa.h from GNUstep to a Mac and ran clang and gcc -fsyntax-only and, indeed Apple's GCC takes a bit more than twice as long to run. The same file on FreeBSD takes about the same time between FSF GCC and Clang (Release-Asserts build). I'm not sure if this is just an Objective-C thing, or if Apple's GCC is much slower for C too.

David

-- Sent from my Difference Engine

I just did some tests with Objective-C programs and found
clang much slower than GCC 4.2, with an optimized build of LLVM and
Clang and a stock build of GCC.

This isn't what we're seeing. We're seeing Clang being consistently
faster than GCC 4.2.

Are you certain that you built LLVM and Clang in Release-Asserts?
Debug builds are horribly slow, and Release builds are still a bit
slower than Release-Asserts builds.

I was using the wrong binary. Running the tests again, there's about
a 10% variation in speeds each time I run clang and gcc, but they both
end up being approximately the same speed. Using -fsyntax-only on
GNUstep's Cocoa.h takes slightly longer with clang than GCC on
average, although the fastest result for clang is slightly better than
the slowest result for gcc.

Are your tests relative to Apple's GCC? It's a lot slower than FSF
GCC for Objective-C, at least - not sure why, but FSF GCC can usually
compile a couple of files before Apple GCC reports the first warning
in one.

Apple's default for objc is the non-fragile abi, while FSF's is the fragile-abi.
Make sure to use the same abi for both.

- Fariborz

Hi David,

This is interesting stuff, can you provide more concrete tests cases &
numbers? For example, it would be interesting to break down the
numbers just to verify that the extra time is in Sema (compare -MM
numbers (which is ~= -Eonly) to -fsyntax-only to -S).

I just did some tests with Objective-C programs and found
clang much slower than GCC 4.2, with an optimized build of LLVM and
Clang and a stock build of GCC.

This isn't what we're seeing. We're seeing Clang being consistently
faster than GCC 4.2.

Are you certain that you built LLVM and Clang in Release-Asserts?
Debug builds are horribly slow, and Release builds are still a bit
slower than Release-Asserts builds.

I was using the wrong binary. Running the tests again, there's about
a 10% variation in speeds each time I run clang and gcc, but they both
end up being approximately the same speed. Using -fsyntax-only on
GNUstep's Cocoa.h takes slightly longer with clang than GCC on
average, although the fastest result for clang is slightly better than
the slowest result for gcc.

Are your tests relative to Apple's GCC? It's a lot slower than FSF
GCC for Objective-C, at least - not sure why, but FSF GCC can usually
compile a couple of files before Apple GCC reports the first warning
in one.

With a Release-Asserts build, clang is very slightly slower than the
GCC 4.2.1 build included with the FreeBSD 8 base system for Objective-
C (although it does produce warnings that GCC doesn't produce and
supports a few more language features).

I took the preprocessed Cocoa.h from GNUstep to a Mac and ran clang
and gcc -fsyntax-only and, indeed Apple's GCC takes a bit more than
twice as long to run. The same file on FreeBSD takes about the same
time between FSF GCC and Clang (Release-Asserts build). I'm not sure
if this is just an Objective-C thing, or if Apple's GCC is much slower
for C too.

It's hard to evaluate this -- Apple GCC has many other changes so this
could be almost anything (e.g., better optimization by GCC of GCC
itself, or alternately FreeBSD GCC compiling Clang worse). I'm
curious, but I'm also only really interested in making Clang faster,
so if you wanted to investigate more it would probably make sense to
see if Clang is doing something "bad" on FreeBSD, or if Clang is
somehow slower on FreeBSD than it should be (compare on equivalent
hardware?). It would also make sense to evaluate C compile time first
before looking at Obj-C; do a benchmark compile of gcc, say.

In my tests on Linux, Clang was substantially faster at building stuff
(I sent the Blender compile time numbers to the list, and that was
something like 50% faster IIRC) than GCC, so I don't have any reason
to believe the performance difference is Apple specific.

- Daniel