[libcxx] build system questions

Hi

I downloaded libcxx and built it. I noticed that the buildit script doesn't do
dependency checking. It seems pretty non-standard. Most projects' build system
do dependancy checking to avoid rebuilding stuff that doesn't need to be built.
What's the reason for this? Is it just a case of not enough time?

Also the testit script didn't work and failed with these kind of errors:

  ailed 1 tests in /home/sashan/code/repos/git/llvm/src/libcxx/test/algorithms/alg.sorting/alg.heap.operations
  g++: /usr/lib/libSystem.B.dylib: No such file or directory
  g++: x86_64: No such file or directory
  g++: ../lib/libc++.so.1.0: No such file or directory

If I spent more time looking into it I'd figure out what to do to work around
it, but ideally someone downloading this library with intent to use it shouldn't
need to.

While I agree with you, we should keep in mind that libcxx is mainly
written by a single person, and he has presumably not had the need for
any cross-platform build system yet. But I am sure that he would welcome
contributions to improve this situation.

I simply made my own wrapper scripts to circumvent to problem on Linux.
Instead of calling buildit directly I used the script below:

export CC=g++
export RC_FLAGS="-Wall -std=c++0x"
export RC_BUILDIT=1
${SHELL} buildit

Of course, that does not solve the dependency problem, but at least I
did not experience any problems with .dylib afterwards.

I downloaded libcxx and built it. I noticed that the buildit script doesn't do
dependency checking. It seems pretty non-standard. Most projects' build system

While I agree with you, we should keep in mind that libcxx is mainly
written by a single person, and he has presumably not had the need for
any cross-platform build system yet. But I am sure that he would welcome
contributions to improve this situation.

I simply made my own wrapper scripts to circumvent to problem on Linux.
Instead of calling buildit directly I used the script below:

export CC=g++
export RC_FLAGS="-Wall -std=c++0x"
export RC_BUILDIT=1
${SHELL} buildit

Thanks Bjorn.

Of course, that does not solve the dependency problem, but at least I
did not experience any problems with .dylib afterwards.

On the dependency-checking issue: The build time is currently about a minute. Imho this does not rise to the level of needing dependency checking to speed things up. That being said, if someone wants to add dependency checking and it /does/ result in a speedup, I'd welcome it. One requirement: The list of headers and sources must be implicit. I dislike explicit lists because they seem to me a continual source of errors.

-Howard

>
> Of course, that does not solve the dependency problem, but at least I
> did not experience any problems with .dylib afterwards.

On the dependency-checking issue: The build time is currently about a minute.
Imho this does not rise to the level of needing dependency checking to speed
things up. That being said, if someone wants to add dependency checking and
it /does/ result in a speedup, I'd welcome it. One requirement: The list of
headers and sources must be implicit. I dislike explicit lists because they
seem to me a continual source of errors.

Yeah I can look into this. What are your thoughts on using autoconf/automake for this?
The 'parent' projects clang and llvm use them.

For reference, they don’t use automake, and autoconf is one of the most painful constructs ever devised in the realm of build systems. I would want evidence of a very large problem indeed before solving it with autoconf, and there doesn’t seem to be one currently.

If you must bring an advanced build system in here (and I’m not necessarily advocating it as there doesn’t seem to be a need yet) I’d spend my time looking at CMake which LLVM and Clang also use, works better for cross-platform development, and is generally saner than autotools. Sadly, it also works best with explicit lists of files. =/

I don't really care for the autotools stuff either. I suggested it because it
was in use by the parent projects and it's widely used in other projects.

As soon as things get either expensive to build, or complicated to configure, I'm going to rebel. But if things stay fast and simple, I'm ok. I'd much rather have fast, simple and primitive, than slow, complicated and complex.

I think we should start out with a simple question: What problem are we attempting to solve?

-Howard

>>> On the dependency-checking issue: A The build time is currently about
>> a minute.
>>> Imho this does not rise to the level of needing dependency checking to
>> speed
>>> things up. A That being said, if someone wants to add dependency
>> checking and
>>> it /does/ result in a speedup, I'd welcome it. A One requirement:
>> A The list of
>>> headers and sources must be implicit. A I dislike explicit lists
>> because they
>>> seem to me a continual source of errors.
>>
>> Yeah I can look into this. What are your thoughts on using
>> autoconf/automake for this?
>> The 'parent' projects clang and llvm use them.
>>
>> For reference, they don't use automake, and autoconf is one of the most
>> painful constructs ever devised in the realm of build systems. I would
>> want evidence of a very large problem indeed before solving it with
>> autoconf, and there doesn't seem to be one currently.
>
> I don't really care for the autotools stuff either. I suggested it because it
> was in use by the parent projects and it's widely used in other projects.

As soon as things get either expensive to build, or complicated to configure,
I'm going to rebel. But if things stay fast and simple, I'm ok. I'd much
rather have fast, simple and primitive, than slow, complicated and complex.

I think we should start out with a simple question: What problem are we
attempting to solve?

-Howard

I pretty much agree with those sentiments regarding the build system. The
problem, and it's a minor one in this case since the build time is small, for me
was the lack of dependency checking that resulted in files being compiled
unnecessarily. I guess I found it surprising that it was using a shell script to
build. However, on the grand scale of problems where a score of 10 is a zombie
apocalypse and 1 is not worth waking up for, I'd give this issue a 3.

I could look into sorting something out using make or cmake. Those are lighter
than autotools and I do think having dependency checking under the conditions
you stipulated previously would be an overall win, albeit small.

If we want Windows support (which I do :P), CMake is the way to go.
I'm willing to do it, targeting POSIX-like platforms for now, as doing
a Windows port is a much larger project. It's also possible to do it
without requiring explicit source lists, even though there aren't that
many to begin with.

- Michael Spencer

Windows support seems motivating to me. The build system will probably be the easy part. locale support will be more challenging. Another challenging part may be the low-level stuff such as type_info and exception_ptr.

-Howard

locale will be a lot of work, but not too hard. type_info should be
easy for clang, but hard for MSVC (if we even plan to support MSVC).
I'm afraid to look at exception_ptr :P. LLVM/Clang can't even do
exceptions on Windows currently.

I've attached a very rough patch for Linux CMake support. It has a few
hard coded paths for my system which will need to be configured by
CMake instead. It compiles with both g++ and clang++, however, the lit
tests only work with g++ because clang++ doesn't understand
-nodefaultlibs.

I just finished running all the tests. 2506 out of 4217 passed (~60%).
Most failures are due to undefined references, and the rest are due to
-fno_exceptions or -fno-rtti which I had to set because
__gxx_personality_v0 is part of libstdc++ (I also set
_LIBCXX_NO_{EXCEPTIONS,RTTI}, but it doesn't always seem to be
honored).

TODO:
* Fix RPATH for shared libraries. I gave up and made it static.
* Add support for host and target triples, and auto-select host.
* Support Darwin
* Add CMake options for all the features.
* Configure the lit test config for the build or install dir
* Make clang honor -nodefaultlibs
* Figure out what libs are needed on Linux.
* Add "make check"

- Michael Spencer

really-hacky-cmake.patch (5.4 KB)

I've reviewed this patch. Thanks for working on this. I think it is premature to check it in at this point (with bigcheese and all :-)). However I can see something like this coexisting with the current build system. And I think getting libc++ to work with clang/Windows is a very good goal.

I'm also going to be working on modifying libc++ to run on top of gcc's libsupc++. Perhaps this will help the clang/Windows port?

-Howard

I agree that it's not ready. I'm currently working on getting it to a
usable state. One thing that will be an issue with keeping the current
system is __config. I'm currently rewriting it using the CMake config
checks system, although it would be simple to just have the build dir
have higher include precedence in the CMake build so it doesn't select
the hard coded one.

Adding libsupc++ support will help with Windows via MinGW.

Thanks for the support. I'll send a real patch for CMake support once
it's sane :P.

- Michael Spencer

Attached is a patch that adds a sane CMake build and changes to
compile the code on Linux. The buildit script still works, but I don't
have access to a Darwin box to make sure testit and non-CMake-lit
still work. The only thing that's missing for Darwin CMake support is
the linker flags.

It also has a check target that runs the tests. 85% currently pass on
Linux with gcc 4.4.5. Most of the failures are due to locale not being
implemented, however, some seem unrelated.

- Michael Spencer

add-cmake.patch (35.5 KB)

I've reviewed this patch. Thanks for making it. Please go ahead and commit it. I've tested everything on Mac OS X and all looks good.

-Howard

The attached gnumake file uses $(wildcard...) to avoid explicit
lists. It also has target.o and target.dM. Both require TARGET=xxx
where xxx is some base name of the .cpp file in the src directory.
The 1st just compiles $(TARGET).cpp, the second creates $(TARGET).dM
which contains all the #define's. The second was useful to me to find
if there was some macro #defined for the unbuntu OS.

The equivalent to Howard's buildit can be achieved with:

  make -f libcxx.gmake --always-make

as described here:

  GNU make

The dependencies are handled by:

  ifneq ($(INCDEPS),no)
    include $(OBJ.dir)/*.d
  endif

near the bottom of the file. The $(OBJ.dir)/*.d files are produced
with the -MMD compiler option. If you wish to ignore the
dependencies, pass INCDEPS=no to the makefile, like:

  make -f libcxx.gmake INCDEPS=no

HTH.

-Larry

libcxx.gmake (1.39 KB)

Actually, with this:

  make -f libcxx.gmake --always-make mk_lib

and with the revised(several typos) make file, which is attached.

libcxx.gmake (1.34 KB)