Initial Intel Cmake Build System Patch

[jpeyton52]

Hello All,

Since everyone wants to see it, this patch is the initial cmake build system we have been developing. Some quick notes:

1. The top-level CMakeLists.txt you all have been working on is moved to CMakeLists.txt.old (the src/CMakeLists.txt file is untouched)

2. There is a Build_With_CMake.txt tutorial that shows the most important features and how to build libiomp5

3. As a special note, the current build.pl system does five “micro-tests” which test certain features of the newly built libiomp5.so library (basic linking/compilation, library dependencies, etc.) If you want these to be off in the new CMake system just specify it when calling cmake with the flag –Dtests=off

Please keep in mind this is the initial system and that it can be trimmed back incrementally to what you need. Right now, it builds nearly identical libraries as build.pl. On a related note, I attempted ( as suggested by David ) to port libiomp5 to NetBSD to see what walls I would hit. I hit some J , but eventually got it working (not included in this initial patch). So I am going to follow up this initial patch with further improvements (to the build system) to make re-targeting in general even easier. Some of these improvements will also help the build.pl system as well.

Just try it. See what you hate and what you like. Leave feedback. I am here on weekdays.

Thanks,

Johnny

initial_intel_cmake.patch (111 KB)

[Jack_Howarth]

Jonathon,
Thanks. Confirmed to work properly on x86_64-apple-darwin12 with…

% cd runtime
% mkdir build
% cd build
% cmake -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -Dos=mac -Darch=32 …
% make all common
% file libiomp5.dylib
libiomp5.dylib: Mach-O dynamically linked shared library i386

and

% cd runtime
% mkdir build
% cd build
% cmake -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -Dos=mac -Darch=32e …
% make all common
% file libiomp5.dylib

libiomp5.dylib: Mach-O 64-bit dynamically linked shared library x86_64

Are there any plans to tweak this to automate the build of a fat libiomp5.dylib on darwin?

[_C_Bergstrom]

Hello All,

Since everyone wants to see it, this patch is the initial cmake build system we have been developing. Some quick notes:

1) The top-level CMakeLists.txt you all have been working on is moved to CMakeLists.txt.old (the src/CMakeLists.txt file is untouched)

2) There is a Build_With_CMake.txt tutorial that shows the most important features and how to build libiomp5

3) As a special note, the current build.pl system does five “micro-tests” which test certain features of the newly built libiomp5.so library (basic linking/compilation, library dependencies, etc.) If you want these to be off in the new CMake system just specify it when calling cmake with the flag –Dtests=off

Please keep in mind this is the initial system and that it can be trimmed back incrementally to what you need. Right now, it builds nearly identical libraries as build.pl. On a related note, I attempted ( as suggested by David ) to port libiomp5 to NetBSD to see what walls I would hit. I hit some J , but eventually got it working (not included in this initial patch). So I am going to follow up this initial patch with further improvements (to the build system) to make re-targeting in general even easier. Some of these improvements will also help the build.pl system as well.

Just try it. See what you hate and what you like. Leave feedback. I am here on weekdays.

After the NetBSD stuff - I may contribute a Solaris patch for fun...

I realize I'm a corner case, but some things I'll hit

The Windows part doesn't seem to give any consideration for other compiler (Fortran/ASM/C/C++)?
if(${WINDOWS})
# I guess it's ok for me to do a follow-up patch for other compiler support? I could see the patch being intrusive or making the code ugly though. :-/
# My first thought is remove the if/elseif logic entirely and move all these variable to an external toolchain file
runtime/cmake/icc.cmake
It would have a lot of variables like
WIN_FC_FLAGS_64 = "-nologo -Qdiag-disable:177,5082 -GS -DynamicBase"

if(${DYNAMIC_LIBRARY})|

list(APPEND ||WIN_FC_FLAGS ||-Zi)|

endif()
if(${RELEASE})|
...

I can probably do this, but your feedback/help is appreciated.

[jpeyton52]

Jack,

Right now, after you build the both the 32 and 32e libraries, you should be able to say “make fat” and it will call the lipo command for you.

Keep in mind, the system is setup so that cmake will only include the “fat” target when the configuration is for 32e, and it will not automatically build the libraries for you if you specify the “fat” target but both 32 and 32e libraries haven’t been built yet, only an error is spit out. There are no plans to automate it further.

There is some instruction on it in Build_With_CMake.txt.

+Mac Fat Libraries

[jpeyton52]

After the NetBSD stuff - I may contribute a Solaris patch for fun...

It is always fun to port things!

The Windows part doesn't seem to give any consideration for other compiler (Fortran/ASM/C/C++)?
if(${WINDOWS})
# I guess it's ok for me to do a follow-up patch for other compiler support? I could see the patch being intrusive or making the code ugly though. :-/ # My first thought is remove the if/elseif logic entirely and move all these > variable to an external toolchain file runtime/cmake/icc.cmake It would have a lot of variables like
WIN_FC_FLAGS_64 = "-nologo -Qdiag-disable:177,5082 -GS -DynamicBase"

if(${DYNAMIC_LIBRARY})|
>>list(APPEND ||WIN_FC_FLAGS ||-Zi)|
endif()
if(${RELEASE})|
...
>

A long time ago, we used to support Visual Studio on Windows. Currently, only icl for the C/C++ compiler and ml/ml64 for the assembler on Windows.
Go ahead and add other compiler support for Windows. That would be great. I don't know about moving icc specific stuff to a different file. I was real relaxed about wrapping the Windows specific flags under
If(${ICC}). It might be strange to have LinkerFlags.cmake, CFlags.cmake, etc. and also an icc.cmake (which would include Windows only icc?).

###
Nits

I think there's a "cmake" way to get the OS version?
+function(set_mac_os_new return_mac_os_new)

Would this do what you want?
http://www.cmake.org/Wiki/CMake_Useful_Variables
CMAKE_SYSTEM

I am not sure there is a useful way to get the OSX version in a purely cmake way. The CMAKE_SYSTEM variable doesn't correspond to the OS X version, only the Darwin Kernel version.
I would love to get rid of this though.

Not everyone (very few) systems will have cmake this new. cmake is typically very easy to bootstrap, but may be more friendly if backed off a bit. (Doesn't impact me, but esp for TIMESTAMP. Custom command or some >default could be provided..)
+# - this function alone causes the need for CMake v2.8.11 (TIMESTAMP)

I was thinking hard about this before and I think I will add some code which will call the date command on unix systems. The problem was Windows does not have a clean way to get the TIMESTAMP. The current way Windows does is using some Perl, but I refuse to add any more Perl. I think you are right to back off a bit.

Why not set SSE2? (Anyone who cries fowl.. are you really going to be running OMP on a processor THAT old?)

I think before changing this, it needs to be shown that real performance improvements could be gained.

[Alp_Toker]

After the NetBSD stuff - I may contribute a Solaris patch for fun...

It is always fun to port things!

The Windows part doesn't seem to give any consideration for other compiler (Fortran/ASM/C/C++)?
if(${WINDOWS})
# I guess it's ok for me to do a follow-up patch for other compiler support? I could see the patch being intrusive or making the code ugly though. :-/ # My first thought is remove the if/elseif logic entirely and move all these > variable to an external toolchain file runtime/cmake/icc.cmake It would have a lot of variables like
WIN_FC_FLAGS_64 = "-nologo -Qdiag-disable:177,5082 -GS -DynamicBase"

if(${DYNAMIC_LIBRARY})|
>>list(APPEND ||WIN_FC_FLAGS ||-Zi)|
endif()
if(${RELEASE})|
...
>

A long time ago, we used to support Visual Studio on Windows. Currently, only icl for the C/C++ compiler and ml/ml64 for the assembler on Windows.
Go ahead and add other compiler support for Windows. That would be great. I don't know about moving icc specific stuff to a different file. I was real relaxed about wrapping the Windows specific flags under
If(${ICC}). It might be strange to have LinkerFlags.cmake, CFlags.cmake, etc. and also an icc.cmake (which would include Windows only icc?).

###
Nits

I think there's a "cmake" way to get the OS version?
+function(set_mac_os_new return_mac_os_new)

Would this do what you want?
http://www.cmake.org/Wiki/CMake_Useful_Variables
CMAKE_SYSTEM

I am not sure there is a useful way to get the OSX version in a purely cmake way. The CMAKE_SYSTEM variable doesn't correspond to the OS X version, only the Darwin Kernel version.
I would love to get rid of this though.

Not everyone (very few) systems will have cmake this new. cmake is typically very easy to bootstrap, but may be more friendly if backed off a bit. (Doesn't impact me, but esp for TIMESTAMP. Custom command or some >default could be provided..)
+# - this function alone causes the need for CMake v2.8.11 (TIMESTAMP)

I was thinking hard about this before and I think I will add some code which will call the date command on unix systems. The problem was Windows does not have a clean way to get the TIMESTAMP. The current way Windows does is using some Perl, but I refuse to add any more Perl. I think you are right to back off a bit.

What's the idea behind TIMESTAMP? All things being equal, the produced binary is meant to be identically reproducible given the same build environment and inputs.

There might be a case for including the SVN revision but a configure TIMESTAMP seems counter-productive -- can we drop this?

Alp.

[_C_Bergstrom]

After the NetBSD stuff - I may contribute a Solaris patch for fun...

It is always fun to port things!

The Windows part doesn't seem to give any consideration for other compiler (Fortran/ASM/C/C++)?
if(${WINDOWS})
# I guess it's ok for me to do a follow-up patch for other compiler support? I could see the patch being intrusive or making the code ugly though. :-/ # My first thought is remove the if/elseif logic entirely and move all these > variable to an external toolchain file runtime/cmake/icc.cmake It would have a lot of variables like
WIN_FC_FLAGS_64 = "-nologo -Qdiag-disable:177,5082 -GS -DynamicBase"

if(${DYNAMIC_LIBRARY})|
>>list(APPEND ||WIN_FC_FLAGS ||-Zi)|
endif()
if(${RELEASE})|
...
>

A long time ago, we used to support Visual Studio on Windows. Currently, only icl for the C/C++ compiler and ml/ml64 for the assembler on Windows.
Go ahead and add other compiler support for Windows. That would be great. I don't know about moving icc specific stuff to a different file. I was real relaxed about wrapping the Windows specific flags under
If(${ICC}). It might be strange to have LinkerFlags.cmake, CFlags.cmake, etc. and also an icc.cmake (which would include Windows only icc?).

You misunderstand - I mean 1 configure file per toolchain. No LinkerFlags.cmake.. etc.. For ICC you'd create 1 file with all the things you'd need. icc.cmake

Sorry, the more I look at this the more I'm apposed to your original design. It will lead to spaghetti mess as time goes on an increasingly become harder to maintain. /* Sure for only ICC or another compiler it's fine, but would fly with multiple gcc versions, clang, icc, MSVC, pathscale, UH compiler.. etc */

###
Nits

I think there's a "cmake" way to get the OS version?
+function(set_mac_os_new return_mac_os_new)

Would this do what you want?
http://www.cmake.org/Wiki/CMake_Useful_Variables
CMAKE_SYSTEM

I am not sure there is a useful way to get the OSX version in a purely cmake way. The CMAKE_SYSTEM variable doesn't correspond to the OS X version, only the Darwin Kernel version.
I would love to get rid of this though.

Custom command is easy enough to do this. There may even be .cmake in the wild which helps. Maybe poking the cmake dev list is a good idea here. Alternatively a feature test..

Not everyone (very few) systems will have cmake this new. cmake is typically very easy to bootstrap, but may be more friendly if backed off a bit. (Doesn't impact me, but esp for TIMESTAMP. Custom command or some >default could be provided..)
+# - this function alone causes the need for CMake v2.8.11 (TIMESTAMP)

I was thinking hard about this before and I think I will add some code which will call the date command on unix systems. The problem was Windows does not have a clean way to get the TIMESTAMP. The current way Windows does is using some Perl, but I refuse to add any more Perl. I think you are right to back off a bit.

Why not set SSE2? (Anyone who cries fowl.. are you really going to be running OMP on a processor THAT old?)

I think before changing this, it needs to be shown that real performance improvements could be gained.

Anecdotal evidence, but SSE2 is probably one of the most helpful x86 extensions from my experience. Why not enable it?

[Jack_Howarth]

Jonathan,
This doesn’t seem to work for me on darwin12…

% cd openmp/runtime
% mkdir build
% cd build
% cmake -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -Darch=32 …
…
% make
…
% cmake -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -Darch=32e …
…
% make

Scanning dependencies of target inc
[ 0%] Generating …/exports/mac_32e/include_compat/iomp_lib.h
[ 5%] Built target inc
Scanning dependencies of target needed-headers
[ 15%] Built target needed-headers
Scanning dependencies of target lib
[ 15%] Generating kmp_dummy.o
[ 21%] Generating external-objects.lst
warning: nm: no name list
[ 21%] Generating external-symbols.lst
[ 21%] Generating iomp.o
ld: warning: ld: warning: ld: warning: ignoring file kmp_version.o, file was built for i386 which is not the architecture being linked (x86_64): kmp_version.oignoring file kmp_ftn_extra.o, file was built for i386 which is not the architecture being linked (x86_64): kmp_ftn_extra.oignoring file kmp_ftn_cdecl.o, file was built for i386 which is not the architecture being linked (x86_64): kmp_ftn_cdecl.o

ld: warning: ignoring file kmp_alloc.o, file was built for i386 which is not the architecture being linked (x86_64): kmp_alloc.o
ld: warning: ld: warning: ignoring file kmp_atomic.o, file was built for i386 which is not the architecture being linked (x86_64): kmp_atomic.oignoring file kmp_csupport.o, file was built for i386 which is not the architecture being linked (x86_64): kmp_csupport.o
ld: warning:
ignoring file kmp_debug.o, file was built for i386 which is not the architecture being linked (x86_64): kmp_debug.o
ld: warning: ld: warning: ld: warning: ignoring file kmp_itt.o, file was built for i386 which is not the architecture being linked (x86_64): kmp_itt.oignoring file kmp_error.o, file was built for i386 which is not the architecture being linked (x86_64): kmp_error.old: warning: ignoring file kmp_environment.o, file was built for i386 which is not the architecture being linked (x86_64): kmp_environment.o

ignoring file kmp_global.o, file was built for i386 which is not the architecture being linked (x86_64): kmp_global.o
ld: warning:
ignoring file kmp_i18n.o, file was built for i386 which is not the architecture being linked (x86_64): kmp_i18n.old: warning:
ignoring file kmp_io.o, file was built for i386 which is not the architecture being linked (x86_64): kmp_io.old: warning: ld: warning:
ignoring file kmp_settings.o, file was built for i386 which is not the architecture being linked (x86_64): kmp_settings.oignoring file kmp_runtime.o, file was built for i386 which is not the architecture being linked (x86_64): kmp_runtime.o
ld: warning:
ignoring file kmp_str.o, file was built for i386 which is not the architecture being linked (x86_64): kmp_str.o
ld: warning: ignoring file kmp_tasking.o, file was built for i386 which is not the architecture being linked (x86_64): kmp_tasking.o
ld: warning: ignoring file kmp_taskq.o, file was built for i386 which is not the architecture being linked (x86_64): kmp_taskq.o
ld: warning: ld: warning: ignoring file kmp_threadprivate.o, file was built for i386 which is not the architecture being linked (x86_64): kmp_threadprivate.oignoring file kmp_utility.o, file was built for i386 which is not the architecture being linked (x86_64): kmp_utility.old: warning:

ignoring file ittnotify_static.o, file was built for i386 which is not the architecture being linked (x86_64): ittnotify_static.old: warning: ld: warning: ld: warning: ld: warning:
ld: warning: ignoring file z_Linux_util.o, file was built for i386 which is not the architecture being linked (x86_64): z_Linux_util.old: warning: ignoring file kmp_gsupport.o, file was built for i386 which is not the architecture being linked (x86_64): kmp_gsupport.old: warning: ignoring file kmp_affinity.o, file was built for i386 which is not the architecture being linked (x86_64): kmp_affinity.oignoring file kmp_dispatch.o, file was built for i386 which is not the architecture being linked (x86_64): kmp_dispatch.oignoring file kmp_sched.o, file was built for i386 which is not the architecture being linked (x86_64): kmp_sched.o
ignoring file kmp_lock.o, file was built for i386 which is not the architecture being linked (x86_64): kmp_lock.old: warning:
ignoring file kmp_taskdeps.o, file was built for i386 which is not the architecture being linked (x86_64): kmp_taskdeps.o

ignoring file kmp_cancel.o, file was built for i386 which is not the architecture being linked (x86_64): kmp_cancel.old: warning:

ignoring file z_Linux_asm.o, file was built for i386 which is not the architecture being linked (x86_64): z_Linux_asm.o
[ 26%] Generating unstripped/libiomp5.dylib
[ 26%] Generating libiomp5.dylib
[ 31%] Generating test-touch-rt/.success
Undefined symbols for architecture x86_64:
“_omp_get_max_threads”, referenced from:
_main in test-touch-a2b314.o
“_omp_get_num_threads”, referenced from:
_main in test-touch-a2b314.o
“_omp_get_wtime”, referenced from:
_main in test-touch-a2b314.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [test-touch-rt/.success] Error 1
make[1]: *** [CMakeFiles/lib.dir/all] Error 2

You need to execute a limited form of make clean in between the two build the would delete the previous object files but not the previously built shared libraries. Does this already exist in the cmake support but is undocumented?
Jack

[jpeyton52]

Alp,

I know we use it when we run an OpenMP program and want to know which libiomp5 was used by setting env variable KMP_VERSION=1. It's the only place it used in the codebase. It would be easy to just put in a dummy time for now if we want to get rid of it. The build date (or configure date in cmake's case) is put into the headers (omp.h) via expand-vars.pl as a macro KMP_BUILD_DATE.

Johnny

[jpeyton52]

C. Bergström,

It's fine if you want to change the flag-grabbing layout. Although, I feel having icc.cmake, gcc.cmake, clang.cmake, common.cmake, etc. would be just as hard to maintain. i.e., If some flag is only valid for a subset of compilers then all those *.cmake files will have to be modified. Each file will also have its own if(OS_TYPE) sections. If you really want to support every possible compiler, you are just going to have to strip away as many of the compiler-specific flags as you can.

Why not set SSE2? (Anyone who cries fowl.. are you really going to be
running OMP on a processor THAT old?)

I think before changing this, it needs to be shown that real performance improvements could be gained.

Anecdotal evidence, but SSE2 is probably one of the most helpful x86 extensions from my experience. Why not enable it?

I'm not opposed. I don't have much to hang my hat on concerning this issue. Maybe someone else can chime in on this? It would be very easy to change!

Johnny

[jpeyton52]

Jack,

I have had this problem before. For some reason, specifying the compiler(s) for consecutive runs of cmake is no good.

From http://www.cmake.org/Wiki/CMake_Useful_Variables

CMAKE_C_COMPILER

the compiler used for C files. Normally it is detected and set during the CMake run, but you can override it at configuration time. Note! It cannot be changed after the first cmake or ccmake run. Although the gui allows to enter an alternative, it will be ignored in the next ‘configure’ run. Use for

It has to go like this:

% cd openmp/runtime

% mkdir build

% cd build

% cmake -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -Darch=32 …

…

% make

…

% cmake -Darch=32e …

…

% make

Or this (adding rm –rf command):

% cd openmp/runtime

% mkdir build

% cd build

% cmake -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -Darch=32 …

…

% make

…

% rm –rf *

% cmake -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -Darch=32e …

…

% make

I’m not sure why it messes up so badly when specifying the compiler again. It appears from the messages below that it thinks the objects from the previous IA-32 build are up to date.

Something else you can try is this (32e build first, 32 build second):

% cd openmp/runtime

% mkdir build

% cd build

% cmake -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -Darch=32e …

…

% make

…

% cmake -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -Darch=32 …

…

Your –Darch=32 will be ignored! And the final configuration will show 32e.

The moral of the story: Only specify the compiler for the first run of cmake in a given build directory, or clean out the build directory for each cmake run.

Johnny

[_C_Bergstrom]

Why is this being set at all? This is really all stuff the user can set and possibly should be moved to documentation if you don't like the idea of icc.cmake. I don't see how removing all the if () mess isn't a good idea.

[Cownie_James_H]

C. Bergström,

It's fine if you want to change the flag-grabbing layout. Although, I feel having icc.cmake, gcc.cmake, clang.cmake, common.cmake, etc.
would be just as hard to maintain. i.e., If some flag is only valid for a subset of compilers then all those *.cmake files will have to
be modified. Each file will also have its own if(OS_TYPE) sections.
If you really want to support every possible compiler, you are just going to have to strip away as many of the compiler-specific
flags as you can.

Why is this being set at all? This is really all stuff the user can set and possibly should be moved to documentation if you don't like the idea of icc.cmake. I don't see how removing all the if () mess isn't a good idea.

[_C_Bergstrom]

Considering this is a pretty small community already I'll try not to limit who should or will be using it

[Finkel_Hal_J]

Jonathan,

Just to be clear, is the plan that we'll have both this and the makefile-based build system for some time to come (similar to how LLVM has both a cmake and makefile-based build system)?

Thanks again,
Hal

[jpeyton52]

Hal,

Yes that is correct.

Johnny

[jpeyton52]

C. Bergström,

I am working on this now. I will create:
common_flags.cmake , icc.cmake , gcc.cmake , clang.cmake

I hope to have this done in the next few days.

Johnny

[_C_Bergstrom]

Hi Johnny,

It's been a couple weeks and I just wanted to ping this thread. Was Intel planning some work on cmake or should (PathScale) or others feel safe to invest time in it?

Thanks

[jpeyton52]

Hello C. Bergström,

Here is an initial look at the revised cmake system. I'm still trying to get some Windows parts settled down and more "CMakey".
The differences from the last one:
1) Does not record a configure timestamp so only cmake v2.8.0 and up is required
2) Uses add_library() command for creating library
3) All compiler specific flags are in cmake/${CMAKE_C_COMPILER_ID}Flags.cmake
     If a particular compiler does not have a corresponding compile flags file, then a warning is issued during configuration that only common flags are going to be used (cmake/CommonFlags.cmake)
4) I moved the micro-tests to a separate file (cmake/MicroTests.cmake) remember these can be turned off via -Dtests=off when calling cmake.
5) Windows builds require MASM compiler so specify -DCMAKE_ASM_MASM_COMPILER=... for this assembler rather than -DCMAKE_ASM_COMPILER=...

[ within the top level directory that contains: runtime/ , offload/ , testsuite/ , www/ ]
To remove the old patch, type:
patch -p0 -RE < initial_intel_cmake.patch

[ within the top level directory that contains: runtime/ , offload/ , testsuite/ , www/ ]
To add new patch, type:
patch -p0 < second_intel_cmake.patch

Johnny

second_intel_cmake.patch (101 KB)

[Jack_Howarth]

Jonathan,
The second_intel-cmake.patch seems to have introduced a race condition when using parallel make on darwin. I now find that…

% cd openmp-3.5.0/runtime
% mkdir build
% cd build
% cmake -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -Dos=mac -Darch=32 …
% make -j 2

– The C compiler identification is Clang 4.2.0
– The CXX compiler identification is Clang 4.2.0
– Check for working C compiler: /usr/bin/clang
– Check for working C compiler: /usr/bin/clang – works
– Detecting C compiler ABI info
– Detecting C compiler ABI info - done
– Check for working CXX compiler: /usr/bin/clang++
– Check for working CXX compiler: /usr/bin/clang++ – works
– Detecting CXX compiler ABI info
– Detecting CXX compiler ABI info - done
– Found Perl: /usr/bin/perl (found version “5.12.3”)

------------------- CONFIGURATION --------------------------
Operating System : mac
Architecture : 32
Build Type : release
OpenMP Version : 40
Lib Type : normal
Stats-Gathering : OFF
Fortran Modules : OFF
Build : development