Thank you for the reply. I enabled recursion inlining in the current inliner and I already have some promising results. However, I found some trouble extending the current heuristics. I hope you can all give me some feedback.
First, context: the inliner is a CallGraphSCC pass. I detect recursion here by checking that, for each CallSite, the caller is the same as the callee (direct recursion) or at least both belong to the SCC currently being processed (indirect recursion). The decision whether to inline is taken in the InlineCost class, with the help of instruction visitor CallAnalyzer and some constants defined in namespace InlineConstants.
Now the problem. I want to use a different inline cost analysis for recursive functions. This might include a different/extended list of parameters (another InlineConstants namespace?) as well as a different cost function (CallAnalyzer), keeping the same InlineCost class. However, it is not possible to check if a function is recursive inside InlineCost, because it does not know about the SCC.
So my questions are:
Shouldn’t InlineConstants be part of the InlineCost class? It looks like they are intended to be used in all InlineCost analyses, but that might not be what we need.
What would be the best way of making InlineCost aware that it is dealing with a “recursive callsite”, this is, a callsite that closes a loop in the call graph? For now I have it hacked as a boolean passed from the Inliner, but it hurts my eyes every time I see it. Maybe let InlineCost know about about the current SCC? It would be easy if I could get the SCC that a function belongs to.
Another option would be to handle recursive function inlining in a separate Inliner pass, but the callgraph would not be traversed from the leaves to the root in a single pass.
Any thoughts, ideas, recommendations… will help.