Testing downstream targets in Clang

Hi all,

I’m currently playing with a target of my own in a downstream clang branch. Specifically, this prototype has no C++ library support, nor does it have support for thread local storage.

To that end, I’ve made “AddCXXStdlibLibArgs” a stub, in addition to setting Targetnfo::TLSSupported to false in lib/Basic/Targets/.cpp

However, I’m finding multiple issues in lit and unittests during a check-all which flag not supporting TLS as an error, in addition to a test which throws a C++ library-specific option and expects the target to do something. How are these artificial issues supposed to be resolved for a new target? I can’t find a way to avoid these errors outside of completely disabling the test via UNSUPPORTED/XFAIL or just deleting them in the unit test.

  • Is it expected that the default target is the host, and therefore likely has support for all the things being tested?
  • Is it expected that TLSSupported is never false?

I’m curious how those which develop other targets with TLSSupported = false test their clang.

Specific tests:

  • clang/test/CXX/dcl.dcl/dcl.spec/dcl.constexpr/p3-2b.cpp
  • clang/test/CXX/dcl.dcl/dcl.spec/dcl.constexpr/p3.cpp
  • unittests/AST/ASTImporterTest.cpp (ImportedVarDeclPreservesThreadLocalStorage)

I’ve also wondered about that, testing a cross-compile target as default, so would be grateful for any comments. (Sorry I can’t help with TLS.)

For the lit tests, nothing is actually run on the target, except I think for a few components such as lldb and possibly the runtimes (libcxx etc). A cross-compile target ought to work in general. For PS4/PS5 (which is cross-OS) we have sometimes fiddled with tests, on occasion just marking them unsupported or xfail. But it’s common to build LLVM with multiple targets enabled, and the cross-target-specific tests ought to Just Work.

Lacking a general feature such as TLS might be more problematic. For unittests, what you can do is add code at the top of the problematic TEST function that does something like check for your target triple and call GTEST_SKIP(), which will essentially disable that one TEST function for you. That’s done in a number of cases where the OS is relevant (because unittests always run on the host); if you grep for GTEST_SKIP you should find some useful examples.

For the clang tests, my inclination would be to xfail them for your target. This loses some coverage, but what we do at Sony is have our CI build and run clang for both the host and for our target. That way, the compiler with our changes runs in both environments and so we don’t really lose coverage.

XFAIL and such work, but I’m mildly worried about coverage.

In the dcl.constexpr* tests, the test is not TLS-focused, but XFAIL would disable the entire test, including the 90% of cases that don’t use TLS. I might be too much of a stickler for precision, though.

Thank you for the suggestion of using GTEST_SKIP. I’m much more well versed in lit/LLVM regressions than GoogleTest, so that pointer is helpful. I’ll check it out.

I do think our team should institute a host build/test to ensure our generic changes don’t break other upstream targets, so I’ll bring that up next time we talk about it.