[RFC]Requiring C++14 for the OpenMP subproject


> > ><i>     Hmm, so you propose creating a dependency from the OpenMP runtime to
> </i>><i>     the LLVM libraries? Which classes would you want to use, probably some
> </i>><i>     of the ADTs? Does any of the other runtime libraries (libc++,
> </i>><i>     libc++abi, compiler-rt, sanitizers) already do this? IIRC
> </i>><i>     libc++abi and
> </i>><i>     LLVMSupport share some demangling code, but that is manually copied.
> </i>> > ><i> I am unaware that each subproject reimplements code to avoid a 
> </i>><i> dependency on llvm. My first reaction is "surely not", but it's not 
> </i>><i> totally implausible. I will look into this.
> </i>> ><i> ADT is the obvious contender for reuse. There will also be a lot of OS 
> </i>><i> wrappers somewhere. File IO, dlopen, probably memory allocators. 
> </i>><i> Essentially all the stuff a big cross platform C++ project grows over 
> </i>><i> time. In the case of anything hitting the filesystem, also 
> </i>><i> treacherously difficult to get right across platforms.
> </i>> ><i> There's an open patch against libomptarget that uses dlopen with a 
> </i>><i> hardcoded path length. Opt loads shared libraries on windows afaik so 
> </i>><i> probably has a robust wrapper around dlopen, so I'd start there.
> </i>
> 
> I would separate these two discussions. One motivation for the 
> relicensing effort is to allow sharing code between runtime libraries 
> and the core infrastructure. I expect that, as we move to full coverage 
> on various parts of the relicensing, we'll have a large-scale discussion 
> about how to implement this resuse across the project. However, we can 
> certainly use C++14 language features before we worry about using LLVM's 
> support library.
> 
>   -Hal

I agree wth Hal; there are two different issues here which have potentailly different implications :-

  1. Choice of base language level (C++11 or C++14)
  2. Moving code backwards and forwards between the runtime and the compiler, or adding additional runtime library requirements on users of the OpenMP runtime.

Changing the language dialect seems a small an uncontentious thing to do.
However adding additional dependencies seems more problematic. It is important that the runtime should remain usable with code compiled by GCC, for instance, even in the absence of other libraries. So not just that we support GCC’s OpenMP interfaces when we’re dealing with, say, an OpenMP program compiled by LLVM which links to a GCC compiled library that uses OpenMP, but also an all GCC case, where nothing but libomp has been compiled by LLVM.

This matters because libomp can provide useful performance improvements over libgomp at high core counts, and that is useful to some people.
It’s also a nice sales-pitch for LLVM in general :- “Look, you were able to use some of LLVM without even recompiling your GCC compiled code, and it improved the performance. Just think how good it could be if you moved to LLVM completely…” :slight_smile:

– Jim
James Cownie <jcownie@gmail.com>
Mob: +44 780 637 7146

On 2/11/20 4:56 AM, Jon Chesterfield via Openmp-dev wrote: >//>/Hmm, so you propose creating a dependency from the OpenMP runtime to />/the LLVM libraries? Which classes would you want to use, probably some />/of the ADTs? Does any of the other runtime libraries (libc++, />/libc++abi, compiler-rt, sanitizers) already do this? IIRC />/libc++abi and />/LLVMSupport share some demangling code, but that is manually copied. />//>//>/I am unaware that each subproject reimplements code to avoid a />/dependency on llvm. My first reaction is "surely not", but it's not />/totally implausible. I will look into this. />//>/ADT is the obvious contender for reuse. There will also be a lot of OS />/wrappers somewhere. File IO, dlopen, probably memory allocators. />/Essentially all the stuff a big cross platform C++ project grows over />/time. In the case of anything hitting the filesystem, also />/treacherously difficult to get right across platforms. />//>/There's an open patch against libomptarget that uses dlopen with a />/hardcoded path length. Opt loads shared libraries on windows afaik so />/probably has a robust wrapper around dlopen, so I'd start there. / I would separate these two discussions. One motivation for the relicensing effort is to allow sharing code between runtime libraries and the core infrastructure. I expect that, as we move to full coverage on various parts of the relicensing, we'll have a large-scale discussion about how to implement this resuse across the project. However, we can certainly use C++14 language features before we worry about using LLVM's support library. -Hal

I agree wth Hal; there are two different issues here which have potentailly different implications :-

1. Choice of base language level (C++11 or C++14)
2. Moving code backwards and forwards between the runtime and the
    compiler, or adding additional runtime library requirements on
    users of the OpenMP runtime.

Changing the language dialect seems a small an uncontentious thing to do.
However adding additional dependencies seems more problematic. It is important that the runtime should remain usable with code compiled by GCC, for instance, even in the absence of other libraries. So not just that we support GCC’s OpenMP interfaces when we’re dealing with, say, an OpenMP program compiled by LLVM which links to a GCC compiled library that uses OpenMP, but also an all GCC case, where nothing but libomp has been compiled by LLVM.

This matters because libomp can provide useful performance improvements over libgomp at high core counts, and that is useful to some people.
It’s also a nice sales-pitch for LLVM in general :- “Look, you were able to use some of LLVM without even recompiling your GCC compiled code, and it improved the performance. Just think how good it could be if you moved to LLVM completely…” :slight_smile:

Indeed :slight_smile:

In practice, when we discuss using the LLVM support library in the runtime libraries, I expect that we'll also have some discussion on how it should be split, or not, how it should be linked (statically, dynamically, whatever), what dependencies it may have. I expect that we would want this to be pretty transparent when we do it.

-Hal

> >
> > Hmm, so you propose creating a dependency from the OpenMP runtime to
> > the LLVM libraries? Which classes would you want to use, probably some
> > of the ADTs? Does any of the other runtime libraries (libc++,
> > libc++abi, compiler-rt, sanitizers) already do this? IIRC
> > libc++abi and
> > LLVMSupport share some demangling code, but that is manually copied.
> >
> >
> > I am unaware that each subproject reimplements code to avoid a
> > dependency on llvm. My first reaction is "surely not", but it's not
> > totally implausible. I will look into this.
> >
> > ADT is the obvious contender for reuse. There will also be a lot of OS
> > wrappers somewhere. File IO, dlopen, probably memory allocators.
> > Essentially all the stuff a big cross platform C++ project grows over
> > time. In the case of anything hitting the filesystem, also
> > treacherously difficult to get right across platforms.
> >
> > There's an open patch against libomptarget that uses dlopen with a
> > hardcoded path length. Opt loads shared libraries on windows afaik so
> > probably has a robust wrapper around dlopen, so I'd start there.
>
>
> I would separate these two discussions. One motivation for the
> relicensing effort is to allow sharing code between runtime libraries
> and the core infrastructure. I expect that, as we move to full coverage
> on various parts of the relicensing, we'll have a large-scale discussion
> about how to implement this resuse across the project. However, we can
> certainly use C++14 language features before we worry about using LLVM's
> support library.
>
> -Hal

I agree wth Hal; there are two different issues here which have potentailly different implications :-
Choice of base language level (C++11 or C++14)
Moving code backwards and forwards between the runtime and the compiler, or adding additional runtime library requirements on users of the OpenMP runtime.

Changing the language dialect seems a small an uncontentious thing to
do. However adding additional dependencies seems more problematic. It
is important that the runtime should remain usable with code compiled
by GCC, for instance, even in the absence of other libraries. So not
just that we support GCC’s OpenMP interfaces when we’re dealing with,
say, an OpenMP program compiled by LLVM which links to a GCC compiled
library that uses OpenMP, but also an all GCC case, where nothing but
libomp has been compiled by LLVM.

For the record, that will not work for the device runtime. It has to be
compiled by Clang in the near future (maybe already now). This is not
only for the performance benefits we get when we link it into the
application and remove all the clutter we do actually not need but also
because we are working towards a native OpenMP device runtime. That is,
soon the OpenMP device runtime will contain as little foreign code
(CUDA/HIP/...) as possible in favor of native OpenMP code.

This matters because libomp can provide useful performance
improvements over libgomp at high core counts, and that is useful to
some people. It’s also a nice sales-pitch for LLVM in general :-
“Look, you were able to use some of LLVM without even recompiling your
GCC compiled code, and it improved the performance. Just think how
good it could be if you moved to LLVM completely…” :slight_smile:

I agree and I do not want to change this feature but I believe for the
device runtime it is justified.

Cheers,
  Johannes

For the record, that will not work for the device runtime. It has to be
compiled by Clang in the near future (maybe already now). This is not
only for the performance benefits we get when we link it into the
application and remove all the clutter we do actually not need but also
because we are working towards a native OpenMP device runtime. That is,
soon the OpenMP device runtime will contain as little foreign code
(CUDA/HIP/…) as possible in favor of native OpenMP code.

I am not sure what you are saying “will not work”.I am not really concerned with which compiler is used to build libomp.
(At one level it is clearly preferable to have code which is not compiler dependent, so can be compiled by any compiler,
however if there are significant advantages in forcing the use of CLANG to compile the runtime that’s probably OK.)

What I don’t want is that libomp ends up with dependencies which make it hard to use with code compiled by other
compilers, at least when we’re only dealing with CPUs.
As before, there are two cases there

  1. Executable image compiled by GCC or LLVM, (shared) library compiled by the other compiler where both the executable and library use OpenMP.
  2. All code compiled by GCC.

What I’d like to remain easy is the use of libomp in that second case.

Once we bring in offloading, then everything probably breaks anyway, since I expect that we do things differently from GCC there and haven’t tried to create any shims.

> For the record, that will not work for the device runtime. It has to be
> compiled by Clang in the near future (maybe already now). This is not
> only for the performance benefits we get when we link it into the
> application and remove all the clutter we do actually not need but also
> because we are working towards a native OpenMP device runtime. That is,
> soon the OpenMP device runtime will contain as little foreign code
> (CUDA/HIP/...) as possible in favor of native OpenMP code.

I am not sure what you are saying “will not work”.
I am not really concerned with which compiler is used to build libomp.
(At one level it is clearly preferable to have code which is not compiler dependent, so can be compiled by any compiler,
however if there are significant advantages in forcing the use of CLANG to compile the runtime that’s probably OK.)

What I don’t want is that libomp ends up with dependencies which make it hard to use with code compiled by other
compilers, at least when we’re only dealing with CPUs.
As before, there are two cases there
1) Executable image compiled by GCC or LLVM, (shared) library compiled by the other compiler where both the executable and library use OpenMP.
2) All code compiled by GCC.

What I’d like to remain easy is the use of libomp in that second case.

Once we bring in offloading, then everything probably breaks anyway, since I expect that we do things differently from GCC there and haven’t tried to create any shims.

I agree. My comment was *only* about the offloading part, libomptarget.
The shim layer is less useful here for performance reasons and due to
the fact that we should not commit to an interface. This is legitimate
given that we (should) expect libomptarget to be linked into the
application (as LLVM-IR).

I actually want us to link a single instance of the runtime per "type of
target region" soon. I mean you can then end up with two copies of
libomptarget.ll linked into the application one for SPMD and one for
non-SPMD. This is only necessary as long as we keep global state that
distinguishes the two modes, so getting rid of that would also a
possibility.

We can discuss this further at some point :wink: