XFAIL and XTARGET

How are XFAIL and XTARGET supposed to work in a test?

It would be nice if these were documented on the Testing Infrustructure page.

In looking at the TestRunner.py file, it seems that you should be able to put something this this in a test to tell it that the test is expected to fail on Windows:

// XFAIL: i686-pc-win32,x86_64-pc-win32

But it seems that regardless of what I put after XFAIL, the test comes out as an expected failure if the test fails, or an unexpected pass if the test passes. I’ve been bouncing between Linux and Windows trying different things. It seems that only “// XFAIL” is currently used.

I don’t know Python enough to really understand what should be happening.

Basically, the Driver/hello.c test doesn’t pass on Windows. I though if I put MinGW gcc in the PATH it might pass, but I get the following errors from just running the compile command line:

C:\Tools\llvm\tools\clang\test>clang -ccc-echo -v -o tmp.o Driver/hello.c
clang version 1.1 (trunk 84674)
Target: i686-pc-win32
Thread model: posix
“C:/Tools/llvm/bin/Debug/clang-cc.exe” -triple i686-pc-win32 -S -disable-free -main-file-name hello.c --relocation-model static --disable-f
p-elim --unwind-tables=0 --fmath-errno=1 -v -fexceptions=0 -fdiagnostics-show-option -o C:/DOCUME~1/fcadmin/LOCALS~1/Temp/cc-000000.s -x c D
river/hello.c
clang-cc version 1.1 based upon llvm 2.7svn hosted on i686-pc-win32
ignoring nonexistent directory “c:\Program Files\Microsoft Visual Studio 9.0\VC\PlatformSDK\Include”
ignoring nonexistent directory “/usr/local/include”
ignoring nonexistent directory “/System/Library/Frameworks”
ignoring nonexistent directory “/Library/Frameworks”
#include “…” search starts here:
#include <…> search starts here:
C:/Tools/llvm/bin/lib/clang/1.1/include
c:\Program Files\Microsoft Visual Studio 9.0\VC\include
/usr/include
End of search list.
“c:/mingw/bin/gcc.exe” -v -c -o C:/DOCUME~1/fcadmin/LOCALS~1/Temp/cc-000001.o -x assembler C:/DOCUME~1/fcadmin/LOCALS~1/Temp/cc-000000.s
Using built-in specs.
Target: mingw32
Configured with: …/gcc-4.4.0/configure --enable-languages=c,ada,c++,fortran,java,objc,obj-c++ --disable-sjlj-exceptions --enable-shared --e
nable-libgcj --enable-libgomp --with-dwarf2 --disable-win32-registry --enable-libstdcxx-debug --enable-version-specific-runtime-libs --prefi
x=/mingw --with-gmp=/mingw/src/gmp/root --with-mpfr=/mingw/src/mpfr/root --build=mingw32
Thread model: win32
gcc version 4.4.0 (GCC)
COLLECT_GCC_OPTIONS=’-v’ ‘-c’ ‘-o’ ‘C:/DOCUME~1/fcadmin/LOCALS~1/Temp/cc-000001.o’ ‘-mtune=i386’
c:/mingw/bin/…/lib/gcc/mingw32/4.4.0/…/…/…/…/mingw32/bin/as.exe -o C:/DOCUME~1/fcadmin/LOCALS~1/Temp/cc-000001.o C:/DOCUME~1/fcadmin/L
OCALS~1/Temp/cc-000000.s
C:/DOCUME~1/fcadmin/LOCALS~1/Temp/cc-000000.s: Assembler messages:
C:/DOCUME~1/fcadmin/LOCALS~1/Temp/cc-000000.s:4: Error: no such instruction: align 16' C:/DOCUME~1/fcadmin/LOCALS~1/Temp/cc-000000.s:20: Error: no such instruction: align 16’
C:/DOCUME~1/fcadmin/LOCALS~1/Temp/cc-000000.s:22: Error: no such instruction: `db “I’m a little driver, short and stout.\000”’
clang: error: assembler command failed with exit code 1 (use -v to see invocation)

How are XFAIL and XTARGET supposed to work in a test?

For Clang they behave differently than for LLVM, this is a historical
wart. It's on my list to move Clang to the same model that LLVM uses.

It would be nice if these were documented on the Testing Infrustructure
page.

Yes, yes it would.

In looking at the TestRunner.py file, it seems that you should be able to
put something this this in a test to tell it that the test is expected to
fail on Windows:

// XFAIL: i686-pc-win32,x86_64-pc-win32

But it seems that regardless of what I put after XFAIL, the test comes out
as an expected failure if the test fails, or an unexpected pass if the test
passes. I've been bouncing between Linux and Windows trying different
things. It seems that only "// XFAIL" is currently used.

Right, Clang just looks for XFAIL and doesn't have any support for
failing on a particular target, or failing on all except a particular
target.

I don't know Python enough to really understand what should be happening.

Basically, the Driver/hello.c test doesn't pass on Windows. I though if I
put MinGW gcc in the PATH it might pass, but I get the following errors from
just running the compile command line:

C:\Tools\llvm\tools\clang\test>clang -ccc-echo -v -o tmp.o Driver/hello.c
clang version 1.1 (trunk 84674)
Target: i686-pc-win32
Thread model: posix
"C:/Tools/llvm/bin/Debug/clang-cc.exe" -triple i686-pc-win32 -S
-disable-free -main-file-name hello.c --relocation-model static --disable-f
p-elim --unwind-tables=0 --fmath-errno=1 -v -fexceptions=0
-fdiagnostics-show-option -o C:/DOCUME~1/fcadmin/LOCALS~1/Temp/cc-000000.s
-x c D
river/hello.c
clang-cc version 1.1 based upon llvm 2.7svn hosted on i686-pc-win32
ignoring nonexistent directory "c:\Program Files\Microsoft Visual Studio
9.0\VC\PlatformSDK\Include"
ignoring nonexistent directory "/usr/local/include"
ignoring nonexistent directory "/System/Library/Frameworks"
ignoring nonexistent directory "/Library/Frameworks"
#include "..." search starts here:
#include <...> search starts here:
C:/Tools/llvm/bin/lib/clang/1.1/include
c:\Program Files\Microsoft Visual Studio 9.0\VC\include
/usr/include
End of search list.
"c:/mingw/bin/gcc.exe" -v -c -o
C:/DOCUME~1/fcadmin/LOCALS~1/Temp/cc-000001.o -x assembler
C:/DOCUME~1/fcadmin/LOCALS~1/Temp/cc-000000.s
Using built-in specs.
Target: mingw32
Configured with: ../gcc-4.4.0/configure
--enable-languages=c,ada,c++,fortran,java,objc,obj-c++
--disable-sjlj-exceptions --enable-shared --e
nable-libgcj --enable-libgomp --with-dwarf2 --disable-win32-registry
--enable-libstdcxx-debug --enable-version-specific-runtime-libs --prefi
x=/mingw --with-gmp=/mingw/src/gmp/root --with-mpfr=/mingw/src/mpfr/root
--build=mingw32
Thread model: win32
gcc version 4.4.0 (GCC)
COLLECT_GCC_OPTIONS='-v' '-c' '-o'
'C:/DOCUME~1/fcadmin/LOCALS~1/Temp/cc-000001.o' '-mtune=i386'
c:/mingw/bin/../lib/gcc/mingw32/4.4.0/../../../../mingw32/bin/as.exe -o
C:/DOCUME~1/fcadmin/LOCALS~1/Temp/cc-000001.o C:/DOCUME~1/fcadmin/L
OCALS~1/Temp/cc-000000.s
C:/DOCUME~1/fcadmin/LOCALS~1/Temp/cc-000000.s: Assembler messages:
C:/DOCUME~1/fcadmin/LOCALS~1/Temp/cc-000000.s:4: Error: no such instruction:
`align 16'
C:/DOCUME~1/fcadmin/LOCALS~1/Temp/cc-000000.s:20: Error: no such
instruction: `align 16'
C:/DOCUME~1/fcadmin/LOCALS~1/Temp/cc-000000.s:22: Error: no such
instruction: `db "I'm a little driver, short and stout.\000"'
clang: error: assembler command failed with exit code 1 (use -v to see
invocation)

Dunno, what's going on here.

By the way, I would recommend filing some these things as bugzillas
instead of just emailing them to cfe-dev, its makes them easier to
track over time and makes sure they don't get lost.

- Daniel