Success compiling Adium

I just successfully compiled and ran Adium for the first time using ccc. Thanks to everyone involved for the huge progress that's been made :slight_smile: At the moment compile times are around 50% slower than GCC 4.2 for the whole project, but I haven't yet investigated why that is. The next thing I plan to try is compiling with precompiled headers disabled to see how much that changes the ratio.

              David Smith

  I just successfully compiled and ran Adium for the first time using
ccc. Thanks to everyone involved for the huge progress that's been
made :slight_smile: At the moment compile times are around 50% slower than GCC 4.2
for the whole project, but I haven't yet investigated why that is. The
next thing I plan to try is compiling with precompiled headers
disabled to see how much that changes the ratio.

Thanks for the update/info!

Two things to verify...

#1: Make sure you are running "Release" build of clang.
#2: You might try using clang's recently added "token cache" feature.

snaroff

I did make ENABLE_OPTIMIZED=1 when building llvm, then did:
export PATH=/Volumes/Other/llvm/Release/bin/:$PATH
export CCC=/Volumes/Other/llvm/tools/clang/tools/ccc/ccc
time xcodebuild -parallelizeTargets CC=$CCC LD=$CCC
So I believe that should be with a release build of clang. However, it didn't feel as much faster than the debug build I had as I would have expected, so I will double check that.

How would I go about using the token cache? Looking at Tools.py, it seems to attempt to do some things with it automatically, but it's not completely clear to me that what I'm doing would be sufficient to trigger that.

      David

  I just successfully compiled and ran Adium for the first time using
ccc. Thanks to everyone involved for the huge progress that's been
made :slight_smile: At the moment compile times are around 50% slower than GCC 4.2
for the whole project, but I haven't yet investigated why that is. The
next thing I plan to try is compiling with precompiled headers
disabled to see how much that changes the ratio.

Thanks for the update/info!

Two things to verify...

#1: Make sure you are running "Release" build of clang.
#2: You might try using clang's recently added "token cache" feature.

snaroff

I did make ENABLE_OPTIMIZED=1 when building llvm, then did:
export PATH=/Volumes/Other/llvm/Release/bin/:$PATH
export CCC=/Volumes/Other/llvm/tools/clang/tools/ccc/ccc
time xcodebuild -parallelizeTargets CC=$CCC LD=$CCC
So I believe that should be with a release build of clang. However, it didn't feel as much faster than the debug build I had as I would have expected, so I will double check that.

It should feel much faster. If I recall, the release build is 5-10x faster than the debug build.

How would I go about using the token cache? Looking at Tools.py, it seems to attempt to do some things with it automatically, but it's not completely clear to me that what I'm doing would be sufficient to trigger that.

If your project is set up to use a pre-compiled prefix header (with GCC), then it should be straightforward.

Here is an example using clang directly (I don't have any experience with using Tools.py):

clang -x c-header PrefixHeader.h -o /tmp/PrefixHeader.pth
clang -token-cache /tmp/PrefixHeader.pth SomeModule.m

snaroff

There are actually three kinds of builds: Debug, Release, Release-Asserts (which should be read as Release build without asserts). CCC probably grabs the clang binary from your path, so its worth checking that its grabbing the right binary. Debug builds go in llvm/Debug, Release builds in llvm/Release, etc.

BTW, to perform a Release-Asserts build (which is the fastest build you'll get, and would be considered a production build), do:

make ENABLE_OPTIMIZED=1 DISABLE_ASSERTIONS=1

Hi David,

That's great news! Just to confirm, but you did set CCC_CLANG right?
As currently set up, the new ccc driver doesn't default to calling
clang, and for now even with it it only calls clang on x86/32/Darwin!
:slight_smile:

Otherwise, ccc should do that magic to invoke clang using PTH,
assuming the Xcode project is already set up to use precompiled
headers. This is a bit of a hack currently and is done in a way so
that Xcode will always rebuild the full project, but assuming you are
timing full builds this shouldn't impact that build time.

ccc will search in a limited set of paths for the clang binary (most
notably the directly that ccc itself is in). This does the right thing
generally if you have done a make install and are using the installed
ccc, but otherwise as Ted noted you may be running a clang from your
path. If you set CCC_ECHO=1 during the build ccc will echo the command
it is running and you can tell which clang it is using (if it doesn't
have an absolute path, it is the one from the path).

Note that ccc being written in Python *does* effect the build time
quite a bit; if Adium consists of a large number of relatively small
files then this could have a significant impact (although not 50%).

Also, you can easily test compiling w/o PCH from the command line by
passing GCC_PRECOMPILE_PREFIX_HEADER=NO to Xcode.

- Daniel

Heh, actually I wasn't aware of that flag. Sounds like perhaps my rejoicing was premature. I'll retry later and let you know how it goes.

        David

I retried with that, and it definitely wasn't using clang previously. Sorry about that; still a ways to go after all. I'll see if I can pin down some of the specific issues I'm seeing and file bugs tomorrow for ones that don't already have them.

        David

After andersca's exception handling changes recently, AIUtilities (a framework of various GUI widgets, categories, and utility classes that we use in Adium) compiles successfully with clang. This time I doublechecked that it was actually running clang before getting excited :wink:

  Timing results indicate that I don't have PTH working yet, but ignoring that it's looking good:
    GCC with PCH: real 0m19.376s
    GCC without PCH: real 0m50.649s
    CCC without PTH: real 0m33.290s

  The main Adium application isn't compiling yet, but I'm not entirely certain why. It seems to be running into assertion failures in xcodebuild among other things, which makes it hard to determine what's actually going on.

                  David

Great results! If you run into problems, please do file bugzillas. A goal of mine is to be able to claim that ccc is 2x faster than GCC without PCH. CCC is still implemented in python, hopefully the python -> C++ rewrite and other improvements will close the gap.

-Chris

Nice!

Any idea what is wrong with PTH? The driver transparently tries to
translate the gcc PCH options into PTH compatible options, so this
should "just work". Feel free to file a bug if you think it is a
driver issue; if you attach a log file of the xcodebuild (preferably
with run with
CCC_ADD_ARGS=-ccc-echo
also set in the environment) I can investigate.

- Daniel

Hmmm... it's possible it's actually working, but not helping as much as it helps GCC. I reran a number of times just now with:

time xcodebuild -parallelizeTargets CC=$CCC LD=$CCC > /dev/null
vs
time xcodebuild -parallelizeTargets CC=$CCC LD=$CCC GCC_PRECOMPILE_PREFIX_HEADER=NO > /dev/null

The typical results were around 28s for the first and 33s for the latter. GCC was about 15s (I didn't have the redirect to /dev/null before, so I suspect the 19s time earlier was largely due to xcodebuild spamming my terminal).

  David

You'd have to cut and paste the assertion for me to guess. Ted just fixed one issue that might have caused a problem with pch, you might update and try again. If the problem is in llvm (reproduces with ccc from the command line), be sure to file a bug report with llvm.

Thanks, I'll retry the pch test. Here's the assertion. Additionally, ReportCrash is crashing earlier in the process, which indicates that something else is crashing first. A few days ago that was ld, so I'm betting it still is.

2009-02-10 18:07:12.277 xcodebuild[55271:6303] [BT] ASSERTION FAILURE in /SourceCache/DevToolsBase/DevToolsBase-1148/pbxcore/Infrastructure/XCWorkQueue.m:352
Details: Assertion failed: candidateCommandPhase == _currentPhaseNumber
Object: <XCWorkQueue:0x063c9470>
Method: -activateNextProcessableCommand
Thread: <NSThread: 0x34909d0>{name = (null), num = 4}
Backtrace:
   0 0x00784597 -[XCAssertionHandler handleFailureInMethod:object:fileName:lineNumber:messageFormat:arguments:] (in DevToolsCore)
   1 0x0078430c _XCAssertionFailureHandler (in DevToolsCore)
   2 0x0064b9fa -[XCWorkQueue activateNextProcessableCommand] (in DevToolsCore)
   3 0x0064872b -[XCWorkQueueOperation runOperation] (in DevToolsCore)
   4 0x00630d48 -[XCOperation run] (in DevToolsCore)
   5 0x00633b29 -[XCBuildOperation runOperationInBackground] (in DevToolsCore)
   6 0x006333d9 -[XCThreadedOperation _runOperationInBackground] (in DevToolsCore)
   7 0x943d17ed -[NSThread main] (in Foundation)
   8 0x943d1394 __NSThread__main__ (in Foundation)
   9 0x917d7095 _pthread_start (in libSystem.B.dylib)
  10 0x917d6f52 thread_start (in libSystem.B.dylib)
2009-02-10 18:07:12.279 xcodebuild[55271:6303] -runOperationInBackground raised an exception: ASSERTION FAILURE in /SourceCache/DevToolsBase/DevToolsBase-1148/pbxcore/Infrastructure/XCWorkQueue.m:352
Details: Assertion failed: candidateCommandPhase == _currentPhaseNumber
Object: <XCWorkQueue:0x063c9470>
Method: -activateNextProcessableCommand
Thread: <NSThread: 0x34909d0>{name = (null), num = 4}
Backtrace:
   0 0x00784597 -[XCAssertionHandler handleFailureInMethod:object:fileName:lineNumber:messageFormat:arguments:] (in DevToolsCore)
   1 0x0078430c _XCAssertionFailureHandler (in DevToolsCore)
   2 0x0064b9fa -[XCWorkQueue activateNextProcessableCommand] (in DevToolsCore)
   3 0x0064872b -[XCWorkQueueOperation runOperation] (in DevToolsCore)
   4 0x00630d48 -[XCOperation run] (in DevToolsCore)
   5 0x00633b29 -[XCBuildOperation runOperationInBackground] (in DevToolsCore)
   6 0x006333d9 -[XCThreadedOperation _runOperationInBackground] (in DevToolsCore)
   7 0x943d17ed -[NSThread main] (in Foundation)
   8 0x943d1394 __NSThread__main__ (in Foundation)
   9 0x917d7095 _pthread_start (in libSystem.B.dylib)
  10 0x917d6f52 thread_start (in libSystem.B.dylib)