I am working on a downstream project that would benefit from the ability to attach standard IR metadata on llvm.coro.suspend instructions and have the metadata nodes survive the coroutine lowering process.
The minimal implementation of this that’s viable is using a dedicated MD kind (used nowhere else in LLVM), and then during CoroSplit, preserves all such MD entries by binding them to a “container” MD (of another dedicated MD kind), on the coroutine function itself. The container is simply an array mapping each SuspendPointIdx value, which uniquely identifies an original suspend instruction, to the MD node that was bound to it - if such exists.
Draft implementation can be seen here:
I don’t seem to have alternatives to this, since currently, any suspend-point-specific info I have can’t be made to survive the lowering process:
suspend point indices are not known beforehand, and there are no other preset identifiers for the suspend instructions
the suspend instructions themselves are eliminated by the lowerer, so if I simply attach metadata on them, it will disappear.
The functionality can be gated by a hidden flag, and has relatively low footprint, and only inside the CoroSplit pass. It is expected to have no influence on compiler output, even if the flag is set, since no upstream software uses this kind of MD.
What is your underlying motivation? Debuggability? Better optimizations? Something else?
I wonder, if the feature you are looking for could be turned into a first-class feature, without the need for custom metadata nodes.
E.g., recently I changed the CoroSplit pass to preserve the line numbers associated with a suspend instruction and turn that line number into an artificial DWARF label (see [RFC] Debug info for coroutine suspension locations - take 2). I wonder if your use case could also be turned into a first-class, out-of-the-box use case in a similar way
My motivation in my own project happens to be scheduling control, somewhat similar to folly’s Future<T>::via(...), in a system where there are multiple executors.
But, conceivably, it could be used for other things. For example, generating code that shows you where a live coroutine frame has suspended, just by inspecting its suspend point idx field.
EDIT: Only just saw the debugging use case is exactly what you mentioned here.
Anyway, my identification scheme for executors is known at compile time, but I don’t think it’s right to codify it into LLVM itself, so I like that it’s just arbitrary MD.
I still don’t understand your use case. Open questions include: Is your input language C++? Would this metadata be explicitly added by the frontend (e.g. via C++ attributes) or inferred by some previous compiler pass? What would follow-up passes do with that metadata?
I am wondering those questions primarily because:
I am not sure if your proposal is actually sufficient. E.g., I am not sure what happens to the function annotation if the resume function gets inlined into an outer coroutine function. Afaict, the metadata would get lost? Is that acceptable for your use case?
I am wondering if there is a better solution for your use case
personal curiosity
But if you are unable or unwilling to share additional info (e.g. due to intellectual property concerns), that’s also fine by me.
Based on the information you shared so far, I would neither be a proponent nor an opponent to your original suggestion.
Would still be interested to hear what others think, though…
Unfortunately, there is indeed an issue of intellectual property limiting my ability to disclose precisely how we are intending to use this.
I can say that it is a compiler pass that introduces these scheduling directives - in a manner that is either automatic, or based on hints/pragmas/intrinsics in the source program. After the coroutine lowering, there are other proprietary passes that would use this per-suspend-point scheduling info to generate some code that ensures the resumption will take place at the intended executor.
I can run the pass that consumes this annotation immediately after the coroutine lowering is done, so I don’t need to worry about a later pass deciding to inline a resume function
pseudocode (there might be multiple languages at the front end, let’s not focus on that)
// runs on executor A.
def foo(args) {
for (...) {
if (some_condition) {
// suspend execution on A, resume on B
reschedule_to(B);
...
// any code here will run in executor B.
...
// suspoend execution on B, resume on A
reschedule_to(A);
}
}
}
This foo compiles to a coroutine, reschedule_to compiles to a suspend intrinsic.
I need the metadata to convey the information, whether resumption needs to occur at executor A or B. My model is assuming A and B are known compile time entities, not runtime objects.
I can’t rely on the source language being C++, there can be multiple front ends, some are DSLs that compile through MLIR, and don’t have their own full-fledged notion of a coroutine necessarily.
Would those intellectual property concerns be resolved at some later point? (E.g., if this is patent-related, you could probably share more details after the patent got filed)
Or would the IP concerns be a “forever-problem” since?
In case those IP concerns would resolve themselves over time, we might just want to revisit this discussion in a couple of months when you are able to share more information - and keeping this in downstream for just a few months hopefully wouldn’t increase your maintenance effort, either
Yeah, we can’t accept things we can’t understand. Given your patch is simple, maybe it is a better idea to keep it in downstream for a moment and come back when it is time to make it public.
