Run tests after building with CMake and Xcode?

Hello all,

I'm just starting out with Clang, and have successfully
compiled & run LLVM and Clang from SVN on
Mac OS X 10.6.4 with CMake and Xcode. However, it appears that
the tests in the Index folder are broken when running them from
Xcode (I'm trying to invoke the clang-test target from Xcode).
All the tests work if I build with configure & make, and run the
tests with 'make test' from the clang directory.

It looks like the test scripts are looking for the clang
executable in the wrong location:
<llvm-obj-dir>/lib/Debug/../bin/clang
instad of where clang actually resides, which is
<llvm-obj-dir>/bin/Debug

I've looked around, but after some searching I can't find where
the scripts are getting this path. Can anyone point me in the
right direction?

Thank you,
Ryan

Ryan Gerleve <aikavanak@gmail.com> writes:

[snip]

It looks like the test scripts are looking for the clang
executable in the wrong location:
<llvm-obj-dir>/lib/Debug/../bin/clang
instad of where clang actually resides, which is
<llvm-obj-dir>/bin/Debug

I've looked around, but after some searching I can't find where
the scripts are getting this path. Can anyone point me in the
right direction?

tools/clang/tests/CMakeLists.txt configures the paths for lit. The key
variable is LLVM_TOOLS_DIR.

tools/clang/tests/lit.cfg, at line 33, sets the PATH environment
variable so it includes the place where clang supposedly is, based on
LLVM_TOOLS_DIR. That is later used at line 104.

If you can't figure out the problem please show the piece of output that
makes you think that the test scripts are looking at the wrong directory
for executing clang, plus version numbers, etc.

Thank you for the quick & informative reply, Óscar.

tools/clang/tests/CMakeLists.txt configures the paths for lit. The key
variable is LLVM_TOOLS_DIR.

This variable is defined in terms of LLVM_TOOLS_BINARY_DIR,
and I haven't been able to figure out where that one gets set, either.
Even so, I suspect LLVM_TOOLS_DIR is correct, based on the
output I'm getting (see below).

[snip]

If you can't figure out the problem please show the piece of output that
makes you think that the test scripts are looking at the wrong directory
for executing clang, plus version numbers, etc.

Here's the output from running the tests, through the first failure:
(Note: /Users/Ryan/llvm-xcode is where CMake put its output,
and /Users/Ryan/llvm is the unmodified SVN checkout folder.)

Running Clang regression tests
/usr/bin/python2.6 /Users/Ryan/llvm/utils/lit/lit.py --param
clang_site_config=/Users/Ryan/llvm-xcode/tools/clang/test/lit.site.cfg
--param build_config=Debug -sv --no-progress-bar
/Users/Ryan/llvm-xcode/tools/clang/test
lit.py: lit.cfg:129: note: using clang: '/Users/Ryan/llvm-xcode/bin/Debug/clang'
-- Testing: 2552 tests, 2 threads --
FAIL: Clang :: Index/code-complete-errors.c (1155 of 2552)
******************** TEST 'Clang :: Index/code-complete-errors.c'
FAILED ********************
Script:

Ryan Gerleve <aikavanak@gmail.com> writes:

[snip]

It's the "error invoking /Users/Ryan/llvm-xcode/lib/Debug/../bin/clang"
part that led me to believe the path was set up incorrectly. Notice that
the "note: using clang ..." in the beginning of the output has the
correct path.

There are 34 more failures like this, most with the same error. A few fail
with just "expected string not found in input," and no "error invoking ..."

[snip]

So it is needed to locate the point where the path to clang breaks. I
don't have an OS/X machine, so can't help, sorry. My guess is that the
problem happens in the `c-index-test' tool.

Please file a bug report with the information you provided on your last
message. Hopefully it will reach someone with the means to debug it.

You are correct. It's not a problem with CMake or lit; c-index-test uses the libclang shared library, which expects to find "clang" in ../bin/clang relatively to itself. It's actually a holdover from an old, to-be-deprecated API that implements some of the Clang C API via invocations of the clang binary.

  - Doug

[snip]

So it is needed to locate the point where the path to clang breaks. I
don't have an OS/X machine, so can't help, sorry. My guess is that the
problem happens in the `c-index-test' tool.

You are correct. It's not a problem with CMake or lit; c-index-test uses the libclang shared library, which expects to find "clang" in ../bin/clang relatively to itself. It's actually a holdover from an old, to-be-deprecated API that implements some of the Clang C API via invocations of the clang binary.

I am willing to help out with updating libclang to use
an updated API (what API would that be? Linking with
clang libraries directly instead of calling the binary?).
But, I am new to clang development - would this be a
good project with which to get my feet wet?

Also, I'm curious as to the need for maintaining two
different build procedures, especially since CMake
can generate makefiles as well as project files. Are
there plans to migrate to a single system exclusively
in the future? If not, what's the advantage to maintaing
both of them?

Thank you,
Ryan

The autoconf'ed one is the one that was developed first. Currently, CMake is used to cover systems where autoconf's shell scripts don't work (in particular, MSVC).

I can't say anything regarding official plans, and do not care to speculate about advantages as seen by the main project developers. While I will make a point of exercising both autoconf and CMake build systems if/when I go active again (which may be never), I will note that CMake currently ships broken for non-MSYS MingW32 systems that use the MingW32-provided bash, and locally built make. This happens to include my preferred development environment, so I can't use stock CMake to build anything in my preferred development environment.

I did develop a workaround for my case, but I'd have to intentionally map how the current CMake projects work before proposing how to upstream it into CMake. [Basically, my system autodetects as MSYS due to bash/sh being in my path, but since it's really MingW32 it chokes on the MSYS paths. Forcing generation of MingW32 fails because of an explicit rejection of MingW32-provided sh in the path. So I need yet another project type, as removing the sh-rejector from the stock MingW32 project generator is not a real option.]

Kenneth

Sorry for the delayed response, but the short answer is "no", it's probably not a good first project. There's a bit of refactoring that needs to happen across a few different modules, which I don't think you want to tackle for your first project.

My usual recommendation for a first project is to take on an easy-looking bug in Bugzilla, usually something that improves diagnostics or fixes a crasher of some sort. Those changes are usually localized and easy to verify.

  - Doug

My usual recommendation for a first project is to take on an easy-looking bug in Bugzilla, usually something that improves diagnostics or fixes a crasher of some sort. Those changes are usually localized and easy to verify.

Doug,
Thanks for the reply, and the advice! I will find & fix an appropriate
bug when I have the chance.

- Ryan