How to run test on Windows?

Clang Frontend
1

I’m having some trouble with this. The documentation says: The cmake build tool is set up to create Visual Studio project files for running the tests, “clang-test” being the root. Therefore, to run the test from Visual Studio, right-click the clang-test project and select “Build”.

I don’t see this project anywhere? There is a filter called Clang tests with 4 projects (ASTTests, BasicTests, ClangUnitTests, FrontendTests). ClangUnitTests can’t be run, and other 3 projects run 9 tests in total.

Trying to run from command line:

python D:\Projects\llvm\utils\lit\lit.py -sv --no-progress-bar --param=build_config=Debug D:\Projects\llvm\tools\clang\test\Sema\warn-freestanding-complex.c

gives:

lit.py: LitConfig.py:99: note: Unable to find ‘bash.exe’.
lit.py: lit.cfg:87: fatal: No site specific configuration available!

What am I doing wrong?

2

Getting started and Hacking on clang don’t mention MSYS, is it needed?

3

Getting started and Hacking on clang don't mention MSYS, is it needed?

It claims it's not, and specifically mentions that the grep from MSYS
is not the same as the grep from GnuWin32. So you have to do some
PATH manipulation to make things work. However, I ultimately was
unable to get it to work.

~Aaron

4

I suppose {llvm|clang} tests should work with both MSYS toolchain and gnuwin32.
If unable, please file a bug.

If you use cmake, you may override with LLVM_LIT_TOOLS_DIR to take
gnuwin32 for tests.
Tests can run faster with gnuwin32 than msys (and bash).

...Takumi

5

I'm having some trouble with this. The documentation says: The cmake build
tool is set up to create Visual Studio project files for running the tests,
"clang-test" being the root. Therefore, to run the test from Visual Studio,
right-click the clang-test project and select "Build".
I don't see this project anywhere? There is a filter called Clang tests with
4 projects (ASTTests, BasicTests, ClangUnitTests, FrontendTests).
ClangUnitTests can't be run, and other 3 projects run 9 tests in total.

I guess your cmake does not find python. Try to set PYTHON_EXECUTABLE
explicitly on cmake.
(Do you use cmake-gui.exe?)

Trying to run from command line:
python D:\Projects\llvm\utils\lit\lit.py -sv --no-progress-bar
--param=build_config=Debug
D:\Projects\llvm\tools\clang\test\Sema\warn-freestanding-complex.c
gives:
lit.py: LitConfig.py:99: note: Unable to find 'bash.exe'.
lit.py: lit.cfg:87: fatal: No site specific configuration available!
What am I doing wrong?

You would also need --param clang_site_config=\path\to\lit.site.cfg

...Takumi

6

Oh, I missed the topic.

7

I still get no luck when I try it.

C:\Users\Aaron Ballman\Desktop\llvm\build>bash
bash-3.1$ exit
exit

C:\Users\Aaron Ballman\Desktop\llvm\build>python ..\llvm\utils\lit\lit.py -sv --
no-progress-bar --param=build_config=Debug --param clang_site=tools\clang\test\l
it.site.cfg ..\llvm\tools\clang\test\Sema\warn-freestanding-complex.c
lit.py: LitConfig.py:99: note: Unable to find 'bash.exe'.
lit.py: lit.cfg:87: fatal: No site specific configuration available!

So I can run bash (it's definitely on my PATH and working), but
LitConfig.py still cannot locate it for some reason.

~Aaron

8

not "clang_site" but "clang_site_config", let's retry!

...Takumi

9

I guess your cmake does not find python. Try to set PYTHON_EXECUTABLE
explicitly on cmake.
(Do you use cmake-gui.exe?)

You’re right, after I added python to my path cmake created clang-test project. Building the project gives:

lit.py: LitConfig.py:99: note: Unable to find ‘bash.exe’.
lit.py: lit.cfg:143: note: using clang: ‘D:/Projects/llvm-msvc/bin/Debug/clang.EXE’
Traceback (most recent call last):
File “D:/Projects/llvm/utils/lit/lit.py”, line 5, in lit.main()
File “D:\Projects\llvm\utils\lit\lit\main.py”, line 515, in main testSuiteCache, localConfigCache)[1])
File “D:\Projects\llvm\utils\lit\lit\main.py”, line 230, in getTests ts,path_in_suite = getTestSuite(path, litConfig, testSuiteCache)
File “D:\Projects\llvm\utils\lit\lit\main.py”, line 199, in getTestSuite ts, relative = search(item)
File “D:\Projects\llvm\utils\lit\lit\main.py”, line 183, in search cache[path] = res = search1(path)
File “D:\Projects\llvm\utils\lit\lit\main.py”, line 174, in search1 cfg = TestingConfig.frompath(cfgpath, None, litConfig, mustExist = True)
File “D:\Projects\llvm\utils\lit\lit\TestingConfig.py”, line 45, in frompath exec f in cfg_globals
File “D:\Projects\llvm-msvc\tools\clang\test\lit.site.cfg”, line 21, in lit.load_config(config, “D:/Projects/llvm/tools/clang/test/…/test/lit.cfg”)
File “D:\Projects\llvm\utils\lit\lit\LitConfig.py”, line 65, in load_config config = config)
File “D:\Projects\llvm\utils\lit\lit\TestingConfig.py”, line 45, in frompath exec f in cfg_globals
File “D:/Projects/llvm/tools/clang/test/…/test/lit.cfg”, line 199, in registered_targets = getRegisteredTargets(os.path.join(llvm_tools_dir, ‘llc’))
File “D:/Projects/llvm/tools/clang/test/…/test/lit.cfg”, line 183, in getRegisteredTargets cmd = subprocess.Popen([tool, ‘-version’], stdout=subprocess.PIPE)
File “C:\Python\lib\subprocess.py”, line 679, in init errread, errwrite)
File “C:\Python\lib\subprocess.py”, line 893, in _execute_child startupinfo)
WindowsError: [Error 2] The system cannot find the file specified

I have only GnuWin32 installed at C:\GnuWin32 and LLVM_LIT_TOOLS_DIR set to C:\GnuWin32\bin.

You would also need --param clang_site_config=\path\to\lit.site.cfg

…Takumi

Which one, tried both lit.site.cfg.in and lit.cfg from llvm\tools\clang\test without success.

10

Made some progress. lit.site.cfg is generated by cmake and is located inside build\test.

Python in my path
LLVM_LIT_TOOLS_DIR set to C:\GnuWin32\bin
LIT_TOOLS_ARGS set to -sv --no-progress-bar --param=build_config=Debug --param=clang_site_config=D:\Projects\build\test\lit.site.cfg

Visual Studio project clang-test is present and runs the tests.

Testing on the command line is still giving me some trouble. It seems to be working except that it can’t find any tests (also tried format-strings-fixit.c and warn-unused-parameters.c)?
python llvm\utils\lit\lit.py -sv --no-progress-bar --param=build_config=Debug --param=clang_site_config=build\test\lit.site.cfg llvm\tools\clang\test\Sema\wchar.c

gives:

lit.py: LitConfig.py:99: note: Unable to find ‘bash.exe’.
lit.py: LitConfig.py:99: note: Unable to find ‘bash.exe’.
lit.py: main.py:517: warning: input ‘llvm\tools\clang\test\Sema\wchar.c’ contained no tests
– Testing: 0 tests, 2 threads –

Testing Time: 0.00s

1 warning(s) in tests.

11

Nikola,

Made some progress. lit.site.cfg is generated by cmake and is located inside
build\test.

Ooh, it is regression :smiley:
$(BUILD)/test/lit.site.cfg is for LLVM tests.

Lemme know;

Q0) Show me your CMake configuration. (it's ok with gzip-ed CMakeCache.txt)

Q1) Show me your $(BUILD)/tools/clang/test/lit.site.cfg

Q2) Are you able to run "check" and "check-all"?

Q3) Make sure "llc.exe" is generated in your $(BUILD)/bin/Debug.

Q4) verson of CMake, Python and Visual studio?
  (eg. cmake-2.8.4, python-2.7 distributed by python.org, Visual
Studio 2010 SP1)

...Takumi

12

Aha, everything is working as expected after I started using $(BUILD)/tools/clang/test/lit.site.cfg. I think Hacking on clang should be updated to explain this a bit better and make it easier for everyone else?