OpenMP problems with clang 3.7.0

Hi everybody,

I just downloaded the new 3.7 release and also the OpenMP runtime for Darwin (compiled versions) and tried to compile our software package "Normaliz" with it.
Without -fopenmp=libomp it compiles and runs fine.
With the openmp flag it compiles but I ran in problems (even if I set the number of threads to 1).

I debugged it a bit and the problem is that at the entering of a critical section one std::vector, which is firstprivate in the outer parallel loop gets, invalidated.
As a workaround it works when I copy the vector right before the critical, and use that copy inside the critical section.

I don't know under which exact circumstances this happens and was not able to create a small example in which it happens. If you are interested to check it, you can find the source code on github GitHub - Normaliz/Normaliz: Normaliz is an open source tool for computations in affine monoids, vector configurations, lattice polytopes, and rational cones.. To reproduce the problem compile normaliz via cmake and run it with "normaliz -cq -x=1 example/medium". The critical critical is the critical(TRIANG) on line 1113 in process_pyramid and the effected vector is Pyramid_key. Right before the critical section an
assert(Pyramid_key.size() == dim);
succeeds, but in the critical section it fails.

I hope this helps to figure out what happens. If I can any other information please let me know.

Best regards,
Christof

Hi,

I was able to reproduce the error with v2.12.2 but not with the current master.
I think this problem got fixed with https://github.com/Normaliz/Normaliz/commit/62c3237d7d312555dad7026f494c6caa6e438181

The OpenMP standard mentions under restrictions of the critical construct (ironically in section 2.12.2):

A throw executed inside a critical region must cause execution to resume within
the same critical region, and the same thread that threw the exception must catch
it.

Cheers,
Jonas

Hi,

I was able to reproduce the error with v2.12.2 but not with the current master.

Thanks for looking into it! Did you use llvm 3.7.0 or the svn version?

I think this problem got fixed with https://github.com/Normaliz/Normaliz/commit/62c3237d7d312555dad7026f494c6caa6e438181

The OpenMP standard mentions under restrictions of the critical construct (ironically in section 2.12.2):

A throw executed inside a critical region must cause execution to resume within
the same critical region, and the same thread that threw the exception must catch
it.

This is exactly the point that was fixed in the commit. But it is not the problem I'm having. There should be no exception in this place.

I now also tested it on ubuntu 14.04 (with the current master of normaliz), and I get the same problem. Just to clearify it a bit more. If I put asserts right before entering the critical section and inside the critical section like

                 assert (Pyramid_key.size()==dim); // works fine
                 #pragma omp critical(TRIANG)
                 {
                 assert (Pyramid_key.size()==dim); // fails

then the asserts behave as the comments say.

I used the compiled versions of 3.7.0. I will get the svn version and try it with it.

Best,
Christof

I was now able to compile clang and the omp runtime lib from git/svn on Linux and the error is gone. For me it seems to be a bug in 3.7.0. (@Jonas which llvm version did you use?)

I have two more questions concerning the compilation of clang /openmp.
1) Is it possible to build a static version of libomp?
2) I have problems compiling llvm/clang on MacOs X. I tried it with gcc 4.8.2 and the standart compiler supplied by apple (based on llvm 3.3svn), but with both the compilation fails, do you have a suggestion for a compiler to use? Or should I ask this in another place?

Best,
Christof

1) Is it possible to build a static version of libomp?

Compiling a static OpenMP library is, in general, a very bad idea.
It can easily lead to both awful performance (due to over-subscription) and incorrect program execution when you use any libraries which also use OpenMP.

Imagine that your static instance of the OpenMP runtime starts one thread/core. Now you call into some other library which has its own statically linked OpenMP runtime. That runtime knows nothing about the threads you have already created, so it creates a whole new set. Now you have double over-subscription. Ouch.

Or, suppose you execute an omp atomic statement in a case where it is implemented using a static lock inside the runtime. If you have two runtimes, you have two different locks, and its possible for atomicity not to be enforced between code that uses the two different OpenMP runtimes.

For correctness and performance you need to ensure that there's only one OpenMP runtime in a process. As soon as you create a static OpenMP runtime you make it very easy for that not to be the case.

So, I'm sure it's possible (heck, "It's only software"), but it's not made easy because doing it is a really bad idea.

-- Jim

James Cownie <james.h.cownie@intel.com>
SSG/DPD/TCAR (Technical Computing, Analyzers and Runtimes)
Tel: +44 117 9071438

My usecase would be to compile a static executable for easy distribution. But if it is not supported directly I will not try to do it.

Christof

Hi,

from what my .bash_history says, I then used clang 3.7.0. It is compiled from the tar-balls at llvm.org/releases (including the OpenMP runtime...)

For compilation issues related to clang you should ask on llvm-dev and / or cfe-dev.

Cheers,
Jonas

My usecase would be to compile a static executable for easy
distribution.

And that is, indeed, very seductive, and, in your case may work.

The problem comes when, sometime in the future, you decide that you also need to link in some other library that happens to use OpenMP internally (such as Intel's Math Kernel Library), and then you have a horrible (and potentially hard to diagnose) problem.
Or, your program grows and becomes a framework that dlopens user written "plug-ins".

-- Jim

James Cownie <james.h.cownie@intel.com>
SSG/DPD/TCAR (Technical Computing, Analyzers and Runtimes)
Tel: +44 117 9071438