We have some customers reporting expression evaluation for template function not working on Linux. It is the first time I am looking into this area, based on my brief testing, the experience seems to be not ideal. Taking a hello world example below:
template <typename T>
T addTemplate(T num1, T num2) {
return (num1 + num2);
}
void runTemplateCode() {
int result1;
double result2;
// calling with int parameters
result1 = addTemplate<int>(2, 3);
std::cout << result1 << std::endl;
// calling with double parameters
result2 = addTemplate<double>(2.2, 3.3);
std::cout << result2 << std::endl;
}
If I try to evaluate “p (int)addTemplate(3, 5)”, I got
error: expression failed to parse:
error: <user expression 13>:1:6: 'addTemplate' does not name a template but is followed by template arguments
(int)addTemplate<int>(3, 5)
^ ~~~~~
note: non-template declaration found by name lookup
On Mac, I got:
(lldb) p (int)addTemplate<int>(3, 5)
error: expression failed to parse:
error: <user expression 1>:1:6: 'addTemplate' does not name a template but is followed by template arguments
(int)addTemplate<int>(3, 5)
^ ~~~~~
note: non-template declaration found by name lookup
Omit the template parameter works, “”
p (int)addTemplate(3, 45)
(int) 48
Is this expected that we do not allow evaluating function calls with template argument? Any design doc talking about the rationale/limitation around this? Thanks.
The problem with evaluating templates is that DWARF doesnt currently encode the generic definitions of templates, but only instantiations. So when clang::Sema asks LLDB for what Foo is, LLDB doesnt even find it in the DWARF index because that name apprars nowhere in DWARF; rather we index by Foo<int> etc. Even if we were to get the symbol out of the index, we’d need to reconstruct the generic ClassTemplateDecl AST node to pass to Clang for evaluation. But we dont have enough info in DWARF to construct one properly at the moment.
One solution is to use precompiled modules. LLDB can just deserialise the AST and pass that to clang, instead of constructing the AST from debuginfo. This feature is supported for the std module using the setting: target.import-std-module
Of course that doesnt solve the problem for user-defined templates.
Thanks for sharing. I understand the general template instantiation issue, and the C++ module proposed by the paper.
However, I am not sure how this specific example should fail:
The expressions we try to evaluate (addTemplate<int>/addTemplate<double> ) have been instantiated in the code and dwarf contains these two DW_TAG_subprogram for them:
In theory, lldb should be able to locate these two DW_TAG_subprogram and evaluate.
The fact the non-template auto-deduction expression (addTemplate(3, 5)) succeeds seems to indicates this is a parser issue instead of lacking information?
Basically, what I am saying is, even without C++ module’s support, we should be able to make template instantiated function call succeeds because we have the function information in dwarf. Let me know if this makes sense or not.
As a datapoint, I find GDB works well for this example:
(gdb) p addTemplate<int>(2, 3)
$1 = 5
(gdb) p addTemplate<double>(2.0, 3.3)
$2 = 5.2999999999999998
I understand GDB may have used a custom parser for this while we use Clang parser which insistently asking for addTemplate existence before continue parsing? Since C++ module generic solution is still a bit far away in master yet, maybe we should add some fallback hooking to make basic scenarios working similar as GDB? I heard there are some open diffs achieving this? cc @clayborg
As a datapoint, I find GDB works well for this example:
(gdb) p addTemplate<int>(2, 3)
$1 = 5
(gdb) p addTemplate<double>(2.0, 3.3)
$2 = 5.2999999999999998
I understand GDB may have used a custom parser for this while we use Clang parser which insistently asking for addTemplate existence before continue parsing? Since C++ module generic solution is still a bit far away in master yet, maybe we should add some fallback hooking to make basic scenarios working similar as GDB? I heard there are some open diffs achieving this? cc @clayborg
There’s an RFC for an extension to the “frame variable” data language to encompass some more operations, maybe even some very simple function calling, but I am pretty sure creating templated entities is out of scope for this operation. There’s no such affordance in the expression parser, and adding one seems way more complicated (and fragile) than getting modules to work for other C++ frameworks beyond std…
It’s quite hacky, but worked fine for our use cases. Maybe it’s worth coming back to it and re-evaluate the tradeoffs.
Maybe it won’t be too hard to support some of these cases. The template instantiations are in the debug info, so if the user explicitely writes the arguments and the corresponding type exists in the program, then the data language interpreter should be able to handle it just fine.
The fact the non-template auto-deduction expression (addTemplate(3, 5)) succeeds seems to indicates this is a parser issue instead of lacking information?
Just to describe the failure in more detail:
AFAICT in a somewhat roundabout way the “parser issue” stems from the “lacking information”. When we evaluate addTemplate<int>(3, 5), clang tries to parse this as a template. As part of this, clang::Sema, with the help of LLDB, tries to find an appropriate Decl that describes this function. LLDB constructs the following, which is what clang finds during the lookup:
This is not a template and thus removed from consideration (see clang::Sema::isTemplateName and clang::Sema::FilterAcceptableTemplateNames). Since LLDB couldn’t pass clang an appropriately constructed template decl, the expression stops getting parsed as a template and we run into these parser errors (at least the ones I’m seeing locally). LLDB does parse the DW_AT_subprogram as a template and adds a FunctionTemplateDecl to the AST but it isn’t what cla ng finds during lookup. Last time I looked at this we do this deliberately to work around the DWARF limitation. Revisiting this flow could be a useful exercise
It’s quite hacky, but worked fine for our use cases. Maybe it’s worth coming back to it and re-evaluate the tradeoffs.
jingham:
There’s an RFC for an extension to the “frame variable” data language to encompass some more operations, maybe even some very simple function calling, but I am pretty sure creating templated entities is out of scope for this operation.
Maybe it won’t be too hard to support some of these cases. The template instantiations are in the debug info, so if the user explicitely writes the arguments and the corresponding type exists in the program, then the data language interpreter should be able to handle it just fine.
Calling functions with arbitrary parameters and return types is pretty tricky. I don’t think the current data language plans are particularly ambitious in this regard. That’s what the expression parser is for.
D69309 Support template instantiation in the expression evaluator
jingham: