Where do we really need mangled names

Clang Frontend
1

So when comparing Clang’s debug info strings to GCC’s I came across a couple of disparities hinging on the inclusion of linkage names on certain functions. Here are a few differences:

  • Clang includes linkage names on file-local (static or anon namespace) functions. GCC does not.
  • Clang does not include the linkage name of member functions of function-local classes. GCC does, if the function is non-static/non-anonymous namespace and inline (ie: the member function has linkonce-odr linkage, not internal linkage)
  • Clang does not include the linkage name for constructors and destructors - this may be necessary due to the difference (GCC duplicates, Clang has one version call the other) in implementations, but I doubt it. I assume we still emit multiple member functions, one to describe each version of the ctor/dtor we’re emitting.

It looks like at least for the first case, this may’ve been deliberate ( http://llvm.org/viewvc/llvm-project?view=revision&revision=154570 which doesn’t explain why and points to rdar://11079003 which Jim Grosbach told didn’t have a great deal more context) but I don’t have enough context to understand why and whether it’s just a GCC bug that they don’t emit it, or something tools should handle better, etc.

So - any thoughts on the disparity and if/why it’s necessary?

  • Dave
2

My thought is that it is basically there to assist in lookup of
symbols. I put the earlier patch in because it was seen as useful
there. In general I find that mangled names are helpful on any entity
that we might want to put into a lookup table since we might have a
qualified name that we'd like to look up. I doubt that any
discrepancies here are intentional, but that any time we have a
mangled name we probably want to add it.

-eric

3

It is necessary if you want your debugger to be able to evaluate an expression that fully qualifies the name of the static:

(lldb) expr a::b::g_foo

If you don't have the mangled name, you can only do:

(lldb) expr g_foo

Then you might get many many many different versions if more than one file contains a static named "g_foo".

Also, users never tend to fully qualify names and might try to execute:

(lldb) expr b::g_foo

expecting to see "a::b::g_foo". LLDB is able to find these variables by first looking for all the variable base names that match "g_foo", then removing any whose demangled named doesn't contain "b::g_foo".

So the mangled names on statics allows debuggers to do the right thing and be able to correctly display qualified static variables and is very important for good debugging.

Greg

4

And regarding the change r154570, the summary of that radar is that there was a mangled name for a static function in the symbol table, but the debugger could not look it up in the DWARF because the mangled name was not mentioned there.

-- adrian

5

> So when comparing Clang's debug info strings to GCC's I came across a
couple of disparities hinging on the inclusion of linkage names on certain
functions. Here are a few differences:
>
> * Clang includes linkage names on file-local (static or anon namespace)
functions. GCC does not.
> * Clang does not include the linkage name of member functions of
function-local classes. GCC does, if the function is
non-static/non-anonymous namespace and inline (ie: the member function has
linkonce-odr linkage, not internal linkage)
> * Clang does not include the linkage name for constructors and
destructors - this may be necessary due to the difference (GCC duplicates,
Clang has one version call the other) in implementations, but I doubt it. I
assume we still emit multiple member functions, one to describe each
version of the ctor/dtor we're emitting.
>
> It looks like at least for the first case, this may've been deliberate (
http://llvm.org/viewvc/llvm-project?view=revision&revision=154570 which
doesn't explain why and points to rdar://11079003 which Jim Grosbach told
didn't have a great deal more context) but I don't have enough context to
understand why and whether it's just a GCC bug that they don't emit it, or
something tools should handle better, etc.
>
> So - any thoughts on the disparity and if/why it's necessary?

It is necessary if you want your debugger to be able to evaluate an
expression that fully qualifies the name of the static:

(lldb) expr a::b::g_foo

If you don't have the mangled name, you can only do:

(lldb) expr g_foo

A cursory experiment with GDB doesn't seem to exhibit this behavior. Have I
correctly captured the nature of your example here:

$ cat mang.cpp
namespace x {
static void func() {}
}

int main() { x::func(); }
$ g++-4.8.1 mang.cpp -g
$ gdb a.out
(gdb) start
Temporary breakpoint 1 at 0x40055a: file mang.cpp, line 5.
Starting program: /tmp/dbginfo/a.out

Temporary breakpoint 1, main () at mang.cpp:5
5 int main() { x::func(); }
(gdb) p x::func()
$1 = void
(gdb) exit
$ llvm-dwarfdump a.out | grep linkage_name
$

No mention of any linkage names yet the debugger appears to have been able
to identify the fully qualified name and call the function.

Then you might get many many many different versions if more than one file
contains a static named "g_foo".

I'm not quite sure what you're saying here - this is true of any static,
even those in namespaces - there may be (will be) multiple across a project
all meaning different things. The mangled name won't be unique to any
particular one.

Also, users never tend to fully qualify names and might try to execute:

(lldb) expr b::g_foo

From within namespace 'a' or just as a partial identifier that the user

expects the debugger to search for? I'm not sure if GDB offers this
functionality (I realize you're talking about LLDB, but I don't have it
setup to experiment with) so I can't compare, but I don't see any reason
GDB wouldn't be able to implement that without the mangled name too.

expecting to see "a::b::g_foo". LLDB is able to find these variables by
first looking for all the variable base names that match "g_foo", then
removing any whose demangled named doesn't contain "b::g_foo".

So the mangled names on statics allows debuggers to do the right thing and
be able to correctly display qualified static variables and is very
important for good debugging.

GDB /seems/ to be getting away without it and providing similar
functionality (I could be wrong - perhaps my examples aren't what you had
in mind).

Is it just that it's a performance optimization compared to having to walk
the DIE parent chain to build a fully qualified name? If that's the case,
can we quantify that perf/size tradeoff? (though at that point it's a fair
question about why have the mangled name at all - I'm not really sure what
GDB uses it for when it is present (on externally visible functions))

- Dave

6

Could you try a similar experiment with a constructor/destructor (we don't
emit linkage_name for them, but GCC does) and with member functions of
function-local classes (both in an inline/linkonce-odr function and an
external linkage function - GCC would only put the linkage_name on the
former, not the latter, I believe (because the former's member function has
linkonce-odr linkage and the latter should be internal linkage))?

It'd be nice to get these things consistent.

- David

7

Is it just that it's a performance optimization compared to having to walk
the DIE parent chain to build a fully qualified name? If that's the case,
can we quantify that perf/size tradeoff? (though at that point it's a fair
question about why have the mangled name at all - I'm not really sure what
GDB uses it for when it is present (on externally visible functions))

As a side note here it's pretty hard to do this. At least, afaik, no
debugger actually has a performance suite and few people seem
interested in writing one sadly.

-eric

8

Yeah, I realize that's a bit presumptuous/idealistic on my part. Seems hard
to reason about/justify tradeoffs like this without some kind of analysis.

I guess accelerator tables (just the biggest/most obvious debug info
size/debugger speed tradeoff that springs to mind) were just such such a
substantial win to speed and sufficiently small size that they were
'obvious' without very fine-grained experimentation?

- David

9

What exactly would you like me to test?
What I tried so far (setting breakpoints on those symbols in lldb) confirms what Greg said earlier — qualified lookups for functions that don’t have a linkage name fail:

(lldb) b A
Breakpoint 8: 2 locations.

(lldb) b ~A
Breakpoint 7: 2 locations.
(lldb) b A::A

Breakpoint 6: no locations (pending).
WARNING: Unable to resolve breakpoint to any actual locations.

(lldb) b A::~A()
Breakpoint 3: no locations (pending).
WARNING: Unable to resolve breakpoint to any actual locations.

(lldb) b A::getVal()
Breakpoint 1: where = MipsLinkageA::getVal() + 12 at MipsLinkage.cpp:6, address = 0x0000000100000e1c (lldb) b FunctionLocal::foo Breakpoint 9: no locations (pending). WARNING: Unable to resolve breakpoint to any actual locations. (lldb) b foo Breakpoint 10: where = MipsLinkagefoo + 16 at MipsLinkage.cpp:11, address = 0x0000000100000dd0

– adrian

MipsLinkage.cpp (355 Bytes)

10

From a performance standpoint, we do win when things are in the accelerator tables. LLDB doesn't grub around in the DWARF unless our accelerator tables are _not_ there (in which case we must manually create the accelerator tables by reading all DWARF very slowly). If they are there, they are assumed to be valid and we have been trying to hold clang to that standard. clang must put the plain and mangled name in the DWARF for it to be emitted in the accelerator tables, so we have been slowly improving clang as much as we can to get the accelerator tables accurate as. So LLDB won't even look in the DWARF unless it sees something in the accelerator tables (the __apple_XXX accelerator tables that I create and Eric Christopher then implemented in clang and in the DWARF committee).

Greg

11

From a performance standpoint, we do win when things are in the
accelerator tables. LLDB doesn't grub around in the DWARF unless our
accelerator tables are _not_ there (in which case we must manually create
the accelerator tables by reading all DWARF very slowly). If they are
there, they are assumed to be valid and we have been trying to hold clang
to that standard. clang must put the plain and mangled name in the DWARF
for it to be emitted in the accelerator tables, so we have been slowly
improving clang as much as we can to get the accelerator tables accurate
as. So LLDB won't even look in the DWARF unless it sees something in the
accelerator tables (the __apple_XXX accelerator tables that I create and
Eric Christopher then implemented in clang and in the DWARF committee).

OK - so rather than incrementally improving this by having users file bugs
about missing data, I'm trying to understand the underlying principles so
we can strive to implement this correctly. (While also not including data
we don't really need because people seem to care about debug info size)

Though I wasn't actually suggesting that the accelerator tables were
related to this, just another example of size/perf tradeoff - if they are
related, then that's another piece of the puzzle I would like to understand
so I can better implement these requirements in Clang/LLVM.

So how does the linkage name relate to the accelerator tables? I haven't
really looked into them at all, but I thought they only contained "public"
names (externally visible), but maybe that's just a limitation/misfeature
of the GNU pubnames stuff that will be address in the DWARF 5 accelerator
tables feature/proposal.

Eric - not sure if this is easier just to discuss in person if you have
enough/all the context here.

12

From a performance standpoint, we do win when things are in the
accelerator tables. LLDB doesn't grub around in the DWARF unless our
accelerator tables are _not_ there (in which case we must manually create
the accelerator tables by reading all DWARF very slowly). If they are there,
they are assumed to be valid and we have been trying to hold clang to that
standard. clang must put the plain and mangled name in the DWARF for it to
be emitted in the accelerator tables, so we have been slowly improving clang
as much as we can to get the accelerator tables accurate as. So LLDB won't
even look in the DWARF unless it sees something in the accelerator tables
(the __apple_XXX accelerator tables that I create and Eric Christopher then
implemented in clang and in the DWARF committee).

OK - so rather than incrementally improving this by having users file bugs
about missing data, I'm trying to understand the underlying principles so we
can strive to implement this correctly. (While also not including data we
don't really need because people seem to care about debug info size)

Though I wasn't actually suggesting that the accelerator tables were related
to this, just another example of size/perf tradeoff - if they are related,
then that's another piece of the puzzle I would like to understand so I can
better implement these requirements in Clang/LLVM.

So how does the linkage name relate to the accelerator tables? I haven't
really looked into them at all, but I thought they only contained "public"
names (externally visible), but maybe that's just a limitation/misfeature of
the GNU pubnames stuff that will be address in the DWARF 5 accelerator
tables feature/proposal.

The public part isn't really related, however, the content part of the
spec is still listed pretty accurately here:

http://llvm.org/docs/SourceLevelDebugging.html#name-accelerator-tables

which includes parts about linkage names.

Eric - not sure if this is easier just to discuss in person if you have
enough/all the context here.

Sure, we can do that.

-eric

13

>> So when comparing Clang's debug info strings to GCC's I came across a
couple
>> of disparities hinging on the inclusion of linkage names on certain
>> functions. Here are a few differences:
>>
>> * Clang includes linkage names on file-local (static or anon namespace)
>> functions. GCC does not.
>> * Clang does not include the linkage name of member functions of
>> function-local classes. GCC does, if the function is
>> non-static/non-anonymous namespace and inline (ie: the member function
has
>> linkonce-odr linkage, not internal linkage)
>> * Clang does not include the linkage name for constructors and
destructors -
>> this may be necessary due to the difference (GCC duplicates, Clang has
one
>> version call the other) in implementations, but I doubt it. I assume we
>> still emit multiple member functions, one to describe each version of
the
>> ctor/dtor we're emitting.
>>
>> It looks like at least for the first case, this may've been deliberate
(
>> http://llvm.org/viewvc/llvm-project?view=revision&revision=154570which
>> doesn't explain why and points to rdar://11079003 which Jim Grosbach
told
>> didn't have a great deal more context) but I don't have enough context
to
>> understand why and whether it's just a GCC bug that they don't emit
it, or
>> something tools should handle better, etc.
>>
>> So - any thoughts on the disparity and if/why it's necessary?
>>
>
> My thought is that it is basically there to assist in lookup of
> symbols. I put the earlier patch in because it was seen as useful
> there. In general I find that mangled names are helpful on any entity
> that we might want to put into a lookup table since we might have a
> qualified name that we'd like to look up. I doubt that any
> discrepancies here are intentional, but that any time we have a
> mangled name we probably want to add it.

And regarding the change r154570, the summary of that radar is that there
was a mangled name for a static function in the symbol table, but the
debugger could not look it up in the DWARF because the mangled name was not
mentioned there.

Could you try a similar experiment with a constructor/destructor (we don't
emit linkage_name for them, but GCC does) and with member functions of
function-local classes (both in an inline/linkonce-odr function and an
external linkage function - GCC would only put the linkage_name on the
former, not the latter, I believe (because the former's member function has
linkonce-odr linkage and the latter should be internal linkage))?

It'd be nice to get these things consistent.

What exactly would you like me to test?
What I tried so far (setting breakpoints on those symbols in lldb)
confirms what Greg said earlier -- qualified lookups for functions that
don't have a linkage name fail:

(lldb) b A
Breakpoint 8: 2 locations.

I do wonder what happens if there are multiple As. Is LLDB nice enough to
describe the set of candidates?

If LLDB happily accepts partially and unqualified names, you guys might
gain some debug info size by disabling imported_modules and
imported_declarations - they wouldn't serve any purpose to your debugger.
(though I'm not sure how big they are - there might be no real space
savings there)

Explains why no one complained that they were missing, though... :slight_smile:

(lldb) b ~A
Breakpoint 7: 2 locations.
(lldb) b A::A
Breakpoint 6: no locations (pending).
WARNING: Unable to resolve breakpoint to any actual locations.
(lldb) b A::~A()
Breakpoint 3: no locations (pending).
WARNING: Unable to resolve breakpoint to any actual locations.

Adrian - I'm assuming "A" is just a class at global scope? (not in a
namespace or function, etc)

So this is consistent with Greg's description that the simple name (since
that's the DW_AT_name on the member subprogram) works, but qualified names
don't without the mangled name.

LLDB doesn't form the qualified name from the scopes in which the DIE
appears. GCC does. I don't know what this space/perf tradeoff looks like.

Even if this is a matter for accelerator tables, that doesn't necessarily
mean the mangled name should also be in the DIE, though - right? It could
just appear in the accelerator table.

(lldb) b A::getVal()
Breakpoint 1: where = MipsLinkage`A::getVal() + 12 at MipsLinkage.cpp:6,
address = 0x0000000100000e1c
(lldb) b FunctionLocal::foo
Breakpoint 9: no locations (pending).
WARNING: Unable to resolve breakpoint to any actual locations.
(lldb) b foo
Breakpoint 10: where = MipsLinkage`foo + 16 at MipsLinkage.cpp:11, address
= 0x0000000100000dd0

foo is a member function of a class inside FunctionLocal? Or is it the
implicit ctor? If it's the latter then this example overlaps with the first
one. Perhaps try the getVal example except with a nested class instead of a
global one.

14

the source file was attached to my previous message :slight_smile:

– adrian

15

From a performance standpoint, we do win when things are in the accelerator tables. LLDB doesn't grub around in the DWARF unless our accelerator tables are _not_ there (in which case we must manually create the accelerator tables by reading all DWARF very slowly). If they are there, they are assumed to be valid and we have been trying to hold clang to that standard. clang must put the plain and mangled name in the DWARF for it to be emitted in the accelerator tables, so we have been slowly improving clang as much as we can to get the accelerator tables accurate as. So LLDB won't even look in the DWARF unless it sees something in the accelerator tables (the __apple_XXX accelerator tables that I create and Eric Christopher then implemented in clang and in the DWARF committee).

OK - so rather than incrementally improving this by having users file bugs about missing data, I'm trying to understand the underlying principles so we can strive to implement this correctly. (While also not including data we don't really need because people seem to care about debug info size)

Agreed. Our main concern with old DWARF accelerator tables is that they didn't accelerate anything, they provided a vague way to find the info you are looking for using the base name. Any additional information requires ingesting all DWARF from a compile unit in order to be able to reconstruct decl context information.

So our approach with the accelerator tables is to make them useful as accelerator tables. They share the strings in the .debug_str table for example unlike the old accelerator tables. They are pre-sorted, and they contain both the base name and the mangled name for all things we need to lookup.

Though I wasn't actually suggesting that the accelerator tables were related to this, just another example of size/perf tradeoff - if they are related, then that's another piece of the puzzle I would like to understand so I can better implement these requirements in Clang/LLVM.

So how does the linkage name relate to the accelerator tables?

We place both the base name (DW_AT_name) and the linkage name (either of DW_AT_MIPS_linkage_name or DW_AT_linkage_name) in the accelerator tables. This allows for lookups to happen from both the base name and also allows us to find the exact information given a linkage name. LLDB expressions use the MCJIT which often looks for things via linkage name. Also when users set breakpoints they sometimes might use a demangled name. We can then correlate this with the linkage name and set the breakpoint efficiently.

I haven't really looked into them at all, but I thought they only contained "public" names (externally visible),

They currently do and that is why they are less useful for debugging. They might be useful for a dynamic linker, but they really don't serve the needs debuggers where users will want to set breakpoints on any functions (internal, private, public, external, etc).

but maybe that's just a limitation/misfeature of the GNU pubnames stuff that will be address in the DWARF 5 accelerator tables feature/proposal.

Eric and I both believe that the new tables really do address and solve this issue in an efficient and effective way.

The benefits of the tables at a high level include:
- allow debuggers to use these and trust that a certain level of content are there
- they share strings in the .debug_str table so the strings can be used in .debug_info and all accelerator tables with minimal cost
- they can be mmap'ed in and used as is for efficient searches and don't require an initial parse + sort like all other DWARF tables do
- they are extensible so they can be used for any by name lookup for any kind of table

With LLDB we do many lookups:
- base name lookup
- fully qualified demangled name lookups (user entered breakpoint strings)
- mangled name lookups (from MCJIT and other JIT relocations)

Since DWARF is not a great format for partial parsing we use these accelerator tables to vastly decrease the amount of data we need to parse when doing lookups. The accelerator tables help us to quickly find something by base name, and then using the mangled/demangled name quickly eliminate it from contention before we need to pull in all the DWARF for a compile unit. This saves a lot of memory when debugging.

So our concern does include .o file size, but also includes how that information is later used when consumed by debuggers and symbolication tools.

Greg