Green Dragon LLDB Xcode build update: TSAN support

Hi all,

I’ve made a minor change to the Green Dragon LLDB OS X Xcode build located here:
http://lab.llvm.org:8080/green/job/LLDB/

  1. Previously, the python test run used the default C/C++ compiler to build test inferiors. Now it uses the just-built clang/clang++ to build test inferiors. At some point in the future, we will change this to a matrix of important clang/clang++ versions (e.g. some number of official Xcode-released clangs). For now, however, we’ll continue to build with just one, and that one will be the one in the clang build tree.

  2. The Xcode llvm/clang build step now includes compiler-rt and libcxx. This, together with the change above, will allow the newer LLDB TSAN tests to run.

If you’re ever curious how the Xcode build is run, it uses the build.py script in the zorg repo (http://llvm.org/svn/llvm-project/zorg/trunk) under zorg/jenkins/build.py. The build constructs the build tree with a “derive-lldb” command, and does the Xcode build with the “lldb” command.

Please let me know if you have any questions.

I’ll address any hiccups that may show up ASAP.

Thanks!

One more update:

The Green Dragon OS X LLDB builder now actually runs the gtests instead of just building them.

The gtests run as a phase right before the Python test suite. A non-zero value returning from the gtests will cause the OS X LLDB build to fail. Right now, tracking down the cause of the failure will require looking at the console log for the build and test job. I’m excited to see our gtest test count has gone from roughly 17 to over 100 now!

Pavel or Tamas, are we running the gtests on the Linux buildbots?

-Todd

I think we don’t. If we consider them stable enough for enabling them on a buildbot AND we agree to revert changes breaking the unittests then I am happy with enabling them (doing it should take very little effort from our side). Otherwise I would prefer to wait until we can get them to a stable state.

We're not running them yet. I'd like to add that at some point, but I
haven't gotten around to that yet...

Okay, thanks all.

FWIW I am running them on the OS X side. I haven’t seen any stability problems yet. I’d also expect them to be very stable, much more like a compiler test, since there are far fewer parts in a small-scoped C++ unit test (vs., say, our Python tests which are end-to-end and have many moving parts).

We’ve talked about it a bit over here, which I think you’re alluding to, which is how do we handle them assuming they were treated like any other test (i.e. revert or fix fast if they break). That implies we need to run them, which means we should make that easy. (Particularly easy to not forget). On the OS X side we currently have a target and scheme that will build and run them, but it’s a separate target. I haven’t done anything on the Linux side to make that run, although plugging it into the test targets shouldn’t be hard.

I don’t think the gtests replace the purpose of our Python tests, but I think there’s a wide-open place for them, particularly when initially testing new classes or hard-to-expose places that would be overly cumbersome to test (and therefore don’t get tested).

For now we can see how it goes on the OS X side and see if there are any gotchas we need to watch out for. I’ve had the OS X builder build them for quite a while, just not run them until the other day.

Okay, thanks all.

FWIW I am running them on the OS X side. I haven't seen any stability
problems yet. I'd also expect them to be very stable, much more like a
compiler test, since there are far fewer parts in a small-scoped C++ unit
test (vs., say, our Python tests which are end-to-end and have many moving
parts).

I've been running them locally, and I haven't noticed any flakyness.
It should be also be much easier to remove it, if we find any, because
of the smaller scope and number of tests.

We've talked about it a bit over here, which I think you're alluding to,
which is how do we handle them assuming they were treated like any other
test (i.e. revert or fix fast if they break). That implies we need to run
them, which means we should make that easy. (Particularly easy to not
forget). On the OS X side we currently have a target and scheme that will
build and run them, but it's a separate target. I haven't done anything on
the Linux side to make that run, although plugging it into the test targets
shouldn't be hard.

In cmake, you can run these tests with the check-lldb-unit target. I
like the idea of being able to run both, but I think we should also
keep the option of running them separately. How about:
check-lldb: runs unit and python tests
check-lldb-unit: runs unit tests only
check-lldb-XXX: runs only python tests (for some value of XXX)

I don't think the gtests replace the purpose of our Python tests, but I
think there's a wide-open place for them, particularly when initially
testing new classes or hard-to-expose places that would be overly cumbersome
to test (and therefore don't get tested).

+1