Compile .ll file with coroutines intrinsics

Hi all,

I’m trying to figure out how to use clang to compile some piece of IR in a .ll generated with coroutines.

I tried to use the -fcoroutines-ts, but it isn’t even forwarded by the driver to CC1. And even if I use -Xclang to forward it manually it is then ignored because LLVM IR input is shortcut by CompilerInvocation::ParseLangArgs.

What’s the best way here? I could add support in CompilerInvocation::ParseLangArgs:

@@ -3547,6 +3546,9 @@ bool CompilerInvocation::ParseLangArgs(L
parseSanitizerKinds("-fsanitize=", Args.getAllArgValues(OPT_fsanitize_EQ),
Diags, Opts.Sanitize);

  • // Allows to use -fcoroutine-ts with IR input.
  • Opts.Coroutines = Args.hasArg(OPT_fcoroutines_ts);

Ping on these questions? Anyone want to chime in?

Cheers,

Guessing you might need to cc/to this to some folks who worked on/implemented the features - lots of folks don’t read the -dev lists terribly much these days, unfortunately.

My guess would be: It’s difficult because we’ve essentially fragmented the IR and now we have before-coroutines-are-lowered IR and after-coroutines-are-lowered IR and no way to tell the difference & I guess the support for IR inputs in clang is currently (probably by accident) only supporting after-coroutines-are-lowered.

Guessing you might need to cc/to this to some folks who worked on/implemented the features - lots of folks don’t read the -dev lists terribly much these days, unfortunately.

Good suggestion, adding a few folks here.

My guess would be: It’s difficult because we’ve essentially fragmented the IR and now we have before-coroutines-are-lowered IR and after-coroutines-are-lowered IR and no way to tell the difference & I guess the support for IR inputs in clang is currently (probably by accident) only supporting after-coroutines-are-lowered.

Right: but can you feed the IR post-lowering and reprocess it through the coroutine lowering passes? I would expect this to be idempotent somehow.

Even if it isn’t the case, I assume that adding support for the -fcoroutines-ts option when processing IR files should be fine: the IR post-lowering can just not use the flag.

Thanks,

Also CCing Xun, who’s done a bunch of coroutines work.

Not sure if I fully understand the question. Do you have a full
example of what you expect but doesn't work?
Are you trying to compile C++ code that uses coroutines, or are you
trying to use opt to compile an .ll file?
There are some coroutine tests in Clang that you could also take a
look for reference:
https://github.com/llvm/llvm-project/blob/main/clang/test/CodeGenCoroutines/coro-symmetric-transfer-01.cpp

Not sure if I fully understand the question. Do you have a full
example of what you expect but doesn’t work?
Are you trying to compile C++ code that uses coroutines, or are you
trying to use opt to compile an .ll file?

I’m trying to compile IR (.ll or .bc) file: clang support IR as input as well.

So for example with the case you mention:

clang -emit-llvm -c -O3 -Xclang -disable-llvm-passes clang/test/CodeGenCoroutines/coro-symmetric-transfer-01.cpp -std=c++20 -o coro.bc

This gets me a nice coro.bc file, but I have no way to compile this with clang right now, it will always crash:

$ clang -c coro.bc
PLEASE submit a bug report to https://bugs.llvm.org/ and include the crash backtrace, preprocessed source, and associated run script.
Stack dump:
0. Program arguments: clang -c coro.bc

  1. Code generation
  2. Running pass ‘Function Pass Manager’ on module ‘coro.bc’.
  3. Running pass ‘X86 DAG->DAG Instruction Selection’ on function ‘@_Z3foov
    Segmentation fault

Coroutines do push the boundaries of LLVM IR in the sense that
(1) there’s a mandatory lowering sequence and (2) because of the
complexity of that lowering, the IR prior to that sequence is
much more of an internal representation than a stable format.
With that said, I don’t know of any inherent reason why running
the coroutine passes multiple times would be a problem. It’s
probably just some bit of coroutine bookkeeping (the attribute?)
that we fail to remove after lowering.

Also, the exact form of the crash is surprising; I don’t know
why running lowering multiple times would add new things that
ISel wouldn’t recognize.

John.

Right now this isn’t running it multiple times, it is instead never running it, the backend crashes because ISel does not know about these intrinsics (I think we should be friendlier in the failure mode, but that’s another story).

I use clang to emit IR without running any pass (this is the -Xclang -disable-llvm-passes part of the invocation).

Then I’d like to use clang to “resume” compilation of this file, but clang does not allow me to have an IR input file and run these passes: this is what I’d like to fix. One option I had in the original email in this thread was to change clang to honor -fcoroutines-ts when the input is an IR file.

Hi Medhi,

It looks like a flaw to me. The .bc generated file should be compilable by invoking the clang.
I think here are two options:

  • Add an option in clang like the option -enable-coroutines in opt.
  • Make coroutine passes run by default in LLVM pipeline.

It depends on how stable we think Coroutine passes are now.
It shouldn’t matter to run coroutine passes for general IR. Coroutine Passes would and should do nothing when they can’t find coroutine intrinsics.
In my mind, the reason why the coroutine passes got control by option is that coroutine is an experimental feature.
So we don’t want to they to break our workflow. But if we think they are stable, I think we could turn it on by default.
This may need more discussion.

Thanks,
Chuanqi

I agree we should run coro passes by default in LLVM. It's no longer
experimental and running it on non-coroutines won't hurt.

Not sure if I fully understand the question. Do you have a full
example of what you expect but doesn’t work?
Are you trying to compile C++ code that uses coroutines, or are you
trying to use opt to compile an .ll file?

I’m trying to compile IR (.ll or .bc) file: clang support IR as input

as

well.

So for example with the case you mention:

clang -emit-llvm -c -O3 -Xclang -disable-llvm-passes
clang/test/CodeGenCoroutines/coro-symmetric-transfer-01.cpp -std=c++20 -o
coro.bc

This gets me a nice coro.bc file, but I have no way to compile this with
clang right now, it will always crash:

$ clang -c coro.bc
PLEASE submit a bug report to https://bugs.llvm.org/ and include the

crash

backtrace, preprocessed source, and associated run script.
Stack dump:
0. Program arguments: clang -c coro.bc

  1. Code generation
  2. Running pass ‘Function Pass Manager’ on module ‘coro.bc’.
  3. Running pass ‘X86 DAG->DAG Instruction Selection’ on function

@_Z3foov

Segmentation fault

Coroutines do push the boundaries of LLVM IR in the sense that
(1) there’s a mandatory lowering sequence and (2) because of the
complexity of that lowering, the IR prior to that sequence is
much more of an internal representation than a stable format.
With that said, I don’t know of any inherent reason why running
the coroutine passes multiple times would be a problem. It’s
probably just some bit of coroutine bookkeeping (the attribute?)
that we fail to remove after lowering.

Also, the exact form of the crash is surprising; I don’t know
why running lowering multiple times would add new things that
ISel wouldn’t recognize.

Right now this isn’t running it multiple times, it is instead never running
it, the backend crashes because ISel does not know about these intrinsics

Ah, I had the opposite problem on my mind, sorry.

(I think we should be friendlier in the failure mode, but that’s another
story).

Yes, that’s tricky because, again, LLVM doesn’t like to admit that
mandatory lowering is a thing outside of the codegen prepare passes.
Ideally we’d be able to say that IR in a certain stage never has
certain properties, but we lack the concept of stages.

I use clang to emit IR without running any pass (this is the -Xclang -disable-llvm-passes part of the invocation).

Then I’d like to use clang to “resume” compilation of this file, but clang
does not allow me to have an IR input file and run these passes: this is
what I’d like to fix. One option I had in the original email in this thread
was to change clang to honor -fcoroutines-ts when the input is an IR file.

Given what I said above about stages, I think clang’s current behavior
of producing fully-lowered IR from -emit-llvm and then expecting
.ll/.bc inputs to be fully-lowered IR is the right default.
Probably what we need is a way to override this, which build systems
would expect to be properly paired with -disable-llvm-passes.
So with that option we’d build exactly the same pipeline as we
would for a source invocation.

Alternatively, we could write something about the expected pipeline
into the IR file. I don’t know if we build the pipeline before
the initial load of the module.

John.

H John,

Probably what we need is a way to override this, which build systems
would expect to be properly paired with -disable-llvm-passes.
So with that option we’d build exactly the same pipeline as we
would for a source invocation

I guess turn coro passes by default could one option. What’s your opinion?

Thanks,
Chuanqi