question regarding the C++ test case under LLVM test

Clang Frontend
#1

Hi,

I would like to have the following test case only running under Linux instead of Windows. Can someone shed me the light on how to make it work?

The second question is on how to modify the RUN command to run it under windows. If clangxx is used, the dirver-mode is set to g++. I have to specify the driver mode to be msvc. Please note that the test caseI has header file so that I cannot simply use clang_cc1.

Thanks,

Jin

FILE

// RUN: %clangxx -target x86_64-unknown-linux-gnu -Xclang -fintel-compatibility -O3 %s -Xclang -disable-llvm-optzns -emit-llvm -S -o - | FileCheck %s

// Check that llvm.intel.std.container.ptr intrinsic is generated.

//

#include

#define CONST_VECSIZE 2050

float TIME,RESULT;

std::vector< std::vector > a(CONST_VECSIZE);

std::vector< std::vector > b(CONST_VECSIZE),c(CONST_VECSIZE);

extern void foo(std::vector< std::vector > *);

void test_vector()

{

int i, j, k;

for(i=0;i<CONST_VECSIZE;i++)

for(k=0;k<CONST_VECSIZE;k++)

for(j=0;j<CONST_VECSIZE;j++)

c[i][j] = c[i][j] + a[i][k]* b[k][j];

}

// CHECK: %{{.}} = call %“class.std::vector.0”** @“llvm.std.container.ptr.p0p0class.std::vector.0”(%“class.std::vector.0”** %{{.}})

#2

Hi,

I would like to have the following test case only running under Linux
instead of Windows. Can someone shed me the light on how to make it work?

The second question is on how to modify the RUN command to run it under
windows. If clangxx is used, the dirver-mode is set to g++. I have to
specify the driver mode to be msvc. Please note that the test caseI has
header file so that I cannot simply use clang_cc1.

The RUN command doesn't do what you think it does. The RUN line you've got just runs the compiler, and checks the IR that comes out of it for particular patterns (using FileCheck). However, it seems like you're asking about how to *execute* a testcase... In general, clang tests should not be execution tests. Rather, they should only test codegen, and codegen should be the same no matter what the host is. There are a few exceptions here and there where host-specific functionality needs to be tested, but codegen is not one of them.

Jon

Thanks,

Jin

FILE

// RUN: %clangxx -target x86_64-unknown-linux-gnu -Xclang
-fintel-compatibility -O3 %s -Xclang -disable-llvm-optzns -emit-llvm
-S -o - | FileCheck %s

// Check that llvm.intel.std.container.ptr intrinsic is generated.

//

#include <vector>

p.s. you probably want to expand this ^ out so that your testcase doesn't depend on the host environment. Rhetorical question: what happens to your testcase if the host's <vector> comes from libstdc++ vs if it comes from libc++?

#3

The RUN command doesn't do what you think it does. The RUN line you've got just runs the compiler, and checks the IR that comes out of it for particular patterns (using FileCheck). However, it seems like you're asking about how to *execute* a testcase... In general, clang tests should not be execution tests. Rather, they should only test codegen, and codegen should be the same no matter what the host is. There are a few exceptions here and there where host-specific functionality needs to be tested, but codegen is not one of them.

#4

[mailto:jonathan@codesourcery.com] Sent: Tuesday, September 20, 2016
1:18 PM To: Lin, Jin <jin.lin@intel.com>; cfe-dev@lists.llvm.org
Subject: Re: [cfe-dev] question regarding the C++ test case under
LLVM test

Hi,

I would like to have the following test case only running under
Linux instead of Windows. Can someone shed me the light on how to
make it work?

The second question is on how to modify the RUN command to run it
under windows. If clangxx is used, the dirver-mode is set to g++.
I have to specify the driver mode to be msvc. Please note that the
test caseI has header file so that I cannot simply use clang_cc1.

The RUN command doesn't do what you think it does. The RUN line
you've got just runs the compiler, and checks the IR that comes out
of it for particular patterns (using FileCheck). However, it seems
like you're asking about how to *execute* a testcase... In general,
clang tests should not be execution tests. Rather, they should only
test codegen, and codegen should be the same no matter what the host
is. There are a few exceptions here and there where host-specific
functionality needs to be tested, but codegen is not one of them.

************** I used "llvm-lit -v" to test my test case. So I know
exactly what the script is. I am not asking to execute the test case.
In fact, what I ask for is to compile the test case and emit the LLVM
IR under Linux and Windows.

Sorry for the misunderstanding.

The difference of this test file from
other files in the directory clang/test is that the test file
includes the header file. My point is, with different OS, the header
file might be different.

Even with the same OS, the header might be different (see my "p.s." from below).

Your best bet is to write the minimal version of std::vector that you need to test the functionality you're trying to test, and have it inline in your testcase.... depending on the host environment via #include's is an anti-pattern in clang's testsuite.

Jon

Script: -- clang --driver-mode=g++ -target
x86_64-unknown-linux-gnu -O3
./llvm/tools/clang/test/CodeGen/std-container-ptr.cpp -Xclang
-disable-llvm-optzns -emit-llvm -S -o - | FileCheck
./llvm/tools/clang/test/CodeGen/intel/std-container-ptr.cpp

Jon

Thanks,

Jin

FILE

// RUN: %clangxx -target x86_64-unknown-linux-gnu -Xclang
-fintel-compatibility -O3 %s -Xclang -disable-llvm-optzns
-emit-llvm -S -o - | FileCheck %s

// Check that llvm.intel.std.container.ptr intrinsic is generated.

//

#include <vector>

p.s. you probably want to expand this ^ out so that your testcase
doesn't depend on the host environment. Rhetorical question: what
happens to your testcase if the host's <vector> comes from libstdc++
vs if it comes from libc++?

#define CONST_VECSIZE 2050

float TIME,RESULT;

std::vector< std::vector<float> > a(CONST_VECSIZE);

std::vector< std::vector<float> >
b(CONST_VECSIZE),c(CONST_VECSIZE);

extern void foo(std::vector< std::vector<float> > *);

void test_vector()

{

int i, j, k;

for(i=0;i<CONST_VECSIZE;i++)

for(k=0;k<CONST_VECSIZE;k++)

for(j=0;j<CONST_VECSIZE;j++)

c[i][j] = c[i][j] + a[i][k]* b[k][j];

}

// CHECK: %{{.*}} = call %"class.std::vector.0"**
@"llvm.std.container.ptr.p0p0class.std::vector.0"(%"class.std::vector.

0"**

#5

Hi Jon,

Many thanks for your suggestions.

Jin

#6

[mailto:jonathan@codesourcery.com] Sent: Tuesday, September 20, 2016
1:18 PM To: Lin, Jin <jin.lin@intel.com>; cfe-dev@lists.llvm.org
Subject: Re: [cfe-dev] question regarding the C++ test case under LLVM
test

Hi,

I would like to have the following test case only running under Linux
instead of Windows. Can someone shed me the light on how to make it
work?

The second question is on how to modify the RUN command to run it
under windows. If clangxx is used, the dirver-mode is set to g++.
I have to specify the driver mode to be msvc. Please note that the
test caseI has header file so that I cannot simply use clang_cc1.

The RUN command doesn't do what you think it does. The RUN line you've
got just runs the compiler, and checks the IR that comes out of it for
particular patterns (using FileCheck). However, it seems like you're
asking about how to *execute* a testcase... In general, clang tests
should not be execution tests. Rather, they should only test codegen,
and codegen should be the same no matter what the host is. There are a
few exceptions here and there where host-specific functionality needs
to be tested, but codegen is not one of them.

************** I used "llvm-lit -v" to test my test case. So I know
exactly what the script is. I am not asking to execute the test case.
In fact, what I ask for is to compile the test case and emit the LLVM
IR under Linux and Windows.

Sorry for the misunderstanding.

The difference of this test file from
other files in the directory clang/test is that the test file
includes the header file. My point is, with different OS, the header
file might be different.

Even with the same OS, the header might be different (see my "p.s." from below).

Your best bet is to write the minimal version of std::vector that you need to test the functionality you're trying to test, and have it inline in your testcase.... depending on the host environment via #include's is an anti-pattern in clang's testsuite.

I think a better solution is to extend the utility llvm-lit to process the -target option. If the target option and the host match, the script will be executed. Otherwise just ignore the test case. What do you think?

Thanks,

Jin

Jon

Script: -- clang --driver-mode=g++ -target
x86_64-unknown-linux-gnu -O3
./llvm/tools/clang/test/CodeGen/std-container-ptr.cpp -Xclang
-disable-llvm-optzns -emit-llvm -S -o - | FileCheck
./llvm/tools/clang/test/CodeGen/intel/std-container-ptr.cpp

Jon

Thanks,

Jin

FILE

// RUN: %clangxx -target x86_64-unknown-linux-gnu -Xclang
-fintel-compatibility -O3 %s -Xclang -disable-llvm-optzns -emit-llvm
-S -o - | FileCheck %s

// Check that llvm.intel.std.container.ptr intrinsic is generated.

//

#include <vector>

p.s. you probably want to expand this ^ out so that your testcase
doesn't depend on the host environment. Rhetorical question: what
happens to your testcase if the host's <vector> comes from libstdc++
vs if it comes from libc++?

#define CONST_VECSIZE 2050

float TIME,RESULT;

std::vector< std::vector<float> > a(CONST_VECSIZE);

std::vector< std::vector<float> >
b(CONST_VECSIZE),c(CONST_VECSIZE);

extern void foo(std::vector< std::vector<float> > *);

void test_vector()

{

int i, j, k;

for(i=0;i<CONST_VECSIZE;i++)

for(k=0;k<CONST_VECSIZE;k++)

for(j=0;j<CONST_VECSIZE;j++)

c[i][j] = c[i][j] + a[i][k]* b[k][j];

}

// CHECK: %{{.*}} = call %"class.std::vector.0"**
@"llvm.std.container.ptr.p0p0class.std::vector.0"(%"class.std::vector.

0"**

#7

I think a better solution is to extend the utility llvm-lit to
process the -target option. If the target option and the host match,
the script will be executed. Otherwise just ignore the test case.
What do you think?

That's actually not better.

The problem you're going to run into is that your host system's headers are leaking into a target test. Now you have a testcase that could pass on one machine, but fail on another (even if they're both linux machines). The buildbots are very good at noticing this kind of thing because they all run on different host platforms.

If you look through Clang's codegen tests, you'll find that none of them include anything from the host system... that is not an accident. It was done very purposefully: all of the codegen tests should behave precisely the same, no matter what the host platform is, and no matter what versions of libc/libm/c++ standard library/etc are installed.

There are solutions to the specific questions you're asking (about how
to prevent a testcase from running on a specific platform), but I'm
avoiding telling you them because I think they're solving the wrong
problem (specifically, this is a case of an XY problem:
http://http://xyproblem.info/).

Jon

#8

That's actually not better.

The problem you're going to run into is that your host system's headers are leaking into a target test. Now you have a testcase that could pass on one machine, but fail on another (even if they're both linux machines). The buildbots are very good at noticing this kind of thing because they all run on different host platforms.

If you look through Clang's codegen tests, you'll find that none of them include anything from the host system... that is not an accident. It was done very purposefully: all of the codegen tests should behave precisely the same, no matter what the host platform is, and no matter what versions of libc/libm/c++ standard library/etc are installed.

There are solutions to the specific questions you're asking (about how to prevent a testcase from running on a specific platform), but I'm avoiding telling you them because I think they're solving the wrong problem (specifically, this is a case of an XY problem:
http://http://xyproblem.info/).

#9

[mailto:jonathan@codesourcery.com] Sent: Tuesday, September 20, 2016
3:33 PM To: Lin, Jin <jin.lin@intel.com>; cfe-dev@lists.llvm.org
Subject: Re: [cfe-dev] question regarding the C++ test case under
LLVM test

I think a better solution is to extend the utility llvm-lit to
process the -target option. If the target option and the host
match, the script will be executed. Otherwise just ignore the test
case. What do you think?

That's actually not better.

The problem you're going to run into is that your host system's
headers are leaking into a target test. Now you have a testcase that
could pass on one machine, but fail on another (even if they're both
linux machines). The buildbots are very good at noticing this kind of
thing because they all run on different host platforms.

If you look through Clang's codegen tests, you'll find that none of
them include anything from the host system... that is not an
accident. It was done very purposefully: all of the codegen tests
should behave precisely the same, no matter what the host platform
is, and no matter what versions of libc/libm/c++ standard library/etc
are installed.

There are solutions to the specific questions you're asking (about
how to prevent a testcase from running on a specific platform), but
I'm avoiding telling you them because I think they're solving the
wrong problem (specifically, this is a case of an XY problem:
http://http://xyproblem.info/).

************* The purpose of my test is to check whether the clang
generates the new intrinsic for STL containers such as vector.

Here is the LLVM IR of the vector constructor under Linux and
windows. We can see that the LLVM IR are different. In order to
instrument the new Intrinsic correctly, the clang has to call the
utility getTarget().getTriple().isKnownWindowsMSVCEnvironment() to
understand the OS and perform different action.

This ^ depends entirely on the '-target' triple, and not at all on the host.

For the STL iterator, it is more complicated. Under Linux it is in
gnucxx namespace and under Windows it is in std namespace.

Let's assume I create a file with minimal version of std::vector. Now
the question which version (windows/linux) should I include. One of
them will definitely fail if the script is blindly executed under
Linux and Windows.

Which one you use should depend only on the _target_ triple, and not at all on the _host_.

If you want to stick them in the same file, the preprocessor can be a good way to do that:

// RUN: %clangxx -target i686-pc-linux-gnu ... -DLINUX_32 | FileCheck %s
// RUN: %clangxx -target x86_64-pc-mingw32 ... -DWIN_32 | FileCheck %s

#if defined(LINUX_32)
   // linux-specific version of the stuff you want to test
#elif defined(WIN_32)
   // windows-specific version of the stuff you want to test
#endif

#10

[mailto:jonathan@codesourcery.com] Sent: Tuesday, September 20, 2016
3:33 PM To: Lin, Jin <jin.lin@intel.com>; cfe-dev@lists.llvm.org
Subject: Re: [cfe-dev] question regarding the C++ test case under LLVM
test

I think a better solution is to extend the utility llvm-lit to
process the -target option. If the target option and the host match,
the script will be executed. Otherwise just ignore the test case.
What do you think?

That's actually not better.

The problem you're going to run into is that your host system's
headers are leaking into a target test. Now you have a testcase that
could pass on one machine, but fail on another (even if they're both
linux machines). The buildbots are very good at noticing this kind of
thing because they all run on different host platforms.

If you look through Clang's codegen tests, you'll find that none of
them include anything from the host system... that is not an accident.
It was done very purposefully: all of the codegen tests should behave
precisely the same, no matter what the host platform is, and no matter
what versions of libc/libm/c++ standard library/etc are installed.

There are solutions to the specific questions you're asking (about how
to prevent a testcase from running on a specific platform), but I'm
avoiding telling you them because I think they're solving the wrong
problem (specifically, this is a case of an XY problem:
http://http://xyproblem.info/).

************* The purpose of my test is to check whether the clang
generates the new intrinsic for STL containers such as vector.

Here is the LLVM IR of the vector constructor under Linux and windows.
We can see that the LLVM IR are different. In order to instrument the
new Intrinsic correctly, the clang has to call the utility
getTarget().getTriple().isKnownWindowsMSVCEnvironment() to understand
the OS and perform different action.

This ^ depends entirely on the '-target' triple, and not at all on the host.

For the STL iterator, it is more complicated. Under Linux it is in
gnucxx namespace and under Windows it is in std namespace.

Let's assume I create a file with minimal version of std::vector. Now
the question which version (windows/linux) should I include. One of
them will definitely fail if the script is blindly executed under
Linux and Windows.

Which one you use should depend only on the _target_ triple, and not at all on the _host_.

If you want to stick them in the same file, the preprocessor can be a good way to do that:

// RUN: %clangxx -target i686-pc-linux-gnu ... -DLINUX_32 | FileCheck %s // RUN: %clangxx -target x86_64-pc-mingw32 ... -DWIN_32 | FileCheck %s

#if defined(LINUX_32)
   // linux-specific version of the stuff you want to test #elif defined(WIN_32)
   // windows-specific version of the stuff you want to test #endif

That is very good suggestions. At least it works for my case.

My concern here is that the code size of test case may increase a lot if we keep adding new OS or platform support by following this approach.

Thanks,

Jin

#11

That is very good suggestions. At least it works for my case.

My concern here is that the code size of test case may increase a lot
if we keep adding new OS or platform support by following this
approach.

Your testcase is already *huge* because it #include's all of <vector>.

It'd be better to just write the prototypes for the methods & classes you actually need from <vector>, and keep the testcase as minimal as possible. There's an example of how to do that here:

clang/test/CodeGenCXX/lpad-linetable.cpp

Jon