[RFC][OpenMP][CUDA] Generic Offload File Bundler Tool

Clang Frontend
1

Hi all,

Programming models that support offloading require different toolchains involved in the compilation process. Usually, one toolchain for host and as many toolchains as different devices one wants to offload to is required. If separate compilation is used this means that for a single source file one may get multiple output files.

Also, one of the goals of programming models that support offloading (e.g. OpenMP, CUDA) is to enable users to offload with little effort, by annotating the code with a few pragmas or attributes. It is also desirable that we can save users the trouble of changing their existent applications’ build system to deal all the different files for the different toolchains. So having the compiler always return a single file instead of one for the host and each target even if the user is doing separate compilation would be in my view a very welcomed feature.

I am proposing a tool that I am naming clang-offload-bundler (happy to change the name if required) that is used to bundle files associated with the same user source file but different targets, or to unbundle a file into separate files associated with different targets. A tentative implementation of what is proposed in this RFC is posted in http://reviews.llvm.org/D13909.

This RFC complements the RFC related with offloading driver changes in [RFC][OpenMP][CUDA] Unified Offloading Support in Clang Driver.

Let me know your thoughts. Any feedback is very welcomed!

Thanks!

Samuel

2

Hi Samuel,

Thanks for this detailed explanation on how you plan to do this.

I really much like the idea of bundling and only giving back one single file to the user. This should make it easier to use offloading techniques.

I have one question but don’t know all formats in depth:

Given that the separation is done by respective comments in that format, a tool would concatenate all bundles if it doesn’t know about the bundling, right?

(Imagine a scenario where one object file is compiled with offloading enabled and is afterwards linked into a “plain” executable…)

Are there any “tricks” that could be employed to make the “old” / “normal” tools fall back to the host bundle or a “default” triple?

For example starting a comment section after the first bundle which the clang-offoad-bundler knows about, ignores and correctly extracts the other bundles, but that hides them for other tools?

Thanks,

Jonas

3

Hi Jonas,

Thanks for the feedback.

4

Hi Samuel,

I suppose a) would then fail at runtime and fallback to the host version? That would be fine.

IMO the main usage of b) would be with object files which would work.

Maybe it’s enough to notify the user by producing textual files that fail the “normal” tools?

Greetings

Jonas

5

Hi Jonas