Mark Kromis wrote:
Just a curiosity question, why push for gtest vs Boost Test or a different test suite?
I normally use Boost, and their test suite, so I'm more familiar with that. So I was wondering is one better then the other, or is it just that someone makes a patch for it?
I did a very minimal amount of research into unit testing frameworks before I started using gtest for my own work. Since then I've become quite familiar with gtest's features, which made it a natural choice for me. I presume that other testing frameworks will have their own defenders on this list, and I am certainly willing to listen to what they have to say.
The unit test comparison article mentioned earlier (http://gamesfromwithin.com/?p=29) is a good starting point for evaluating different unit test frameworks. Although gtest is too new to have been mentioned in the article directly, if you actually compare gtest's features against the various criteria used by the author of the article, it comes off quite well.
Note that the author of the article later went and created his own unit test framework (UnitTest++), based on his experience with all of the other frameworks he tested. This was the only framework that I seriously looked at other than gtest, and I found it to be a bit too minimal for what I needed.
I looked more into Boost.Test to see what's in it. Boost.Test doesn't seem to be stand-alone -- I don't see a way to use Boost.Test without importing some other chunks of Boost that the testing library depends on. While Boost is a fine set of libraries, I don't think we want to increase the LLVM distribution by sizeof(Boost) just to enable unittesting, nor do we want to spend the time on maintaining a subset of Boost that's "just enough" to build and use the unittest library, along a modified configure/build process that Boost wants to use (Boost.Build? Boost.Jam?).
Although I haven't actually tried Boost.Test, I kind of figured that this would be the case - that you pretty much have to drink the "Boost Kool-Aid" in order to use it.
So are you planning on maintaining whatever test system, or just have them as a pre-requisite. For example are you going to have the gtest incorporated, or have them install it separately first? I was under the impression that the user would have to install gtest first.
So the plan is to take a snapshot of gtest and check it in to the LLVM tree, rather than have it installed separately. I was able to integrate gtest into LLVM's build system fairly easily, as gtest is designed to be integrated into a foreign build system - basically I just ignored the makefile that comes with gtest, and wrote an LLVM-style makefile rule for it. There's a special source file in gtest which includes all other sources that is intended for just such a purpose. I did not need to modify the gtest sources in any way.
This means that keeping the gtest snapshot up to date will be trivial, since it will only require copying in the latest gtest snapshot and checking it in to LLVM - presuming that gtest remains backwards compatible, which I assume it will.
Licensing-wise, both LLVM and gtest are distributed under a fairly permissive BSD-style license. I don't know who would make the judgement call as to whether or not the licenses can co-exist. However, since neither license is "viral" in the sense of wanting to apply any sort of restriction on derived works or the "work as a whole", I see no barrier to shipping a combined product with different portions falling under different licenses. Thus, the unit tests themselves would still fall under the LLVM license, and linking the unit tests with gtest would not violate either license. Of course, IANAL.
From a maintenance standpoint, we have already heard from several enthusiastic volunteers who are involved in both the development of LLVM and gtest. So I doubt there will be much problems on that score.
My personal goal is that one should be able to check out the head of LLVM on a generic Linux/OS-X system, with only the standard development environment (i.e. make/gcc/etc) and type "make unittest" and have the tests run.
In the longer term, I'd like to see LLVM have an automated build that runs the unit tests as part of the build. I noticed that gtest has an option to output test results in XML form, although I have not played with this personally, it might be useful in this regard.