Hi,
David:
Good point, it will be interesting to see speculative compilation in this context applying on devirtualization, with high level (type) information if applicable.
Hi David & Bekket,
Thanks your replies 
David, Indent here is: ORC v2 JIT APIs has initial(basic) support for concurrent compilation in background compile threads. This means we can speculatively compile functions before they are even referenced by the clients. In future, if the client refer some function and if that function is already compiled speculatively we can just return the address of function, if this is correctly done we can reduce the JIT compilation latencies. IMO, the possible candidates to speculate(functions) are those which will be executed next. We can use program analysis and dynamic profiling to find such functions.
Three possible approaches are possible:
- Based, on analysis from call graphs select functions to compile speculatively with less aggressive optimization flags in background dedicated compiler threads.
- Since it is a JIT, with the help of dynamic profiling we can find frequently executed functions and recompile them with more aggressive optimization in compile thread.
- With Profile-guide optimization results of previous executions, we can find function that are likely to compile next then compile it with aggressive optimization. [PGOs are app dependent]
for cases 1,2: profile guide optimization results are not used. I hope these techniques collectively improve program execution time in long-time. Of course, program-based prediction is not equal to the accuracy of profile-based prediction, but in JIT it is useful to first compile function speculatively by using multiple threads.
I’m a little bit lost, from what I’m understanding, you’re arguing that with the profiling data (collected from runtime), we can make more precise prediction about callee candidates executed next. But in the baseline(i.e. first-time execution) stage, these profile data doesn’t exist, thus you want improve the prediction at that stage with the help of some static analysis.
Am I understanding correctly?
I have considered CFG as a higher level program representation, I maybe wrong here.
For example:
void f2() {}
void f3() {}
void z(){
if(/some condition/)
f2();f3();
else
fn();
}
Follow the control flow of z and compute probability that one of the paths[entry to exit] within the z that lead to a call f2, if the call to f2 occurs in many paths, then the probability that it will execute next is high. It will require some control flow analysis.
Following my comment above, I think you have some misunderstandings on the power of static analysis in LLVM: LLVM of course can eliminate some trivial control flow structure like
if(true) {
// do something
}
Or
if(1 + 2 < 6)
For these cases, you don’t even need to analyze the probability of those branches, cause LLVM will just eliminate and optimize the entire control structures.
But other than that, it’s really hard for LLVM to evaluate the probability of certain branches statically without help from (dynamic) data like profiling data or certain programmer’s annotations like __builtin_expect.
The closest analysis I can come up with are probably LazyValueInfo and ScalarEvolution. You can see if they fit your need.
Best,
Bekket