# Making a new symbol provider

**URL:** https://discourse.llvm.org/t/making-a-new-symbol-provider/39920
**Category:** LLDB
**Created:** [February 11, 2016, 11:41pm UTC](https://discourse.llvm.org/t/making-a-new-symbol-provider/39920 "2016-02-11T23:41:21Z")
**Posts on this page:** 15
**Page:** 1

<div class="post-metadata">

### Author: ![Zachary\_Turner1](https://avatars.discourse-cdn.com/v4/letter/z/90db22/32.png) [@Zachary\_Turner1](https://discourse.llvm.org/u/Zachary_Turner1)
#### Post date: [February 11, 2016, 11:41pm UTC](https://discourse.llvm.org/t/making-a-new-symbol-provider/39920/1 "2016-02-11T23:41:21Z")

</div>

Hi,

I want to make a new symbol provider to teach LLDB to understand microsoft PDB files. I’ve been looking over the various symbol APIs, and I have a few questions.

1. Under what circumstances do I need a custom SymbolVendor? The way pdb works is that generally there is 1 file that contains all the debug info needed for a single binary (so or executable). Given a list of paths, we can then determine if there is a matching PDB in one of those paths. Is it better to do this in the CalculateAbilities() function of the symbol file plugin (by just returning 0 if we don’t find a match) or do we need to do something more complicated?

2. Why is there a function called ParseCompileUnitLanguage? The CompileUnit class already stores the language when ParseCompileUnit is called, and ParseCompileUnitLanguage is implemented by just getting that value out. What is the poitn of this function?

3. There’s a function called ParseCompileUnitDebugMacros. Is this referring to C / C++ macros? Like #define FOO 7? What is that used for? I don’t believe info about preprocessor definitions are stored in PDB. Is this going to cause problems?

4. ParseCompileUnitSupportFiles. What are “support files”? Given a file “foo.cpp” is this supposed to be header files etc?

5. ParseCompileUnitLineTable. On the LineTable class you can add “line sequences” or individual entries. What’s the difference here? Is there any disadvantage to adding every single line entry in the line table using the InsertLineEntry instead of building a line sequence and inserting the sequence?

I will probably have some more questions as I continue down this path. For now I’m planning to implement the minimum amount of functionality required just to make LLDB locate and open a PDB for an executable without actually returning anything useful from it. So when I start filling out types, functions, etc I may have some more questions.

---

<div class="post-metadata">

### Author: ![Greg\_Clayton1](https://avatars.discourse-cdn.com/v4/letter/g/838e76/32.png) [@Greg\_Clayton1](https://discourse.llvm.org/u/Greg_Clayton1)
#### Post date: [February 12, 2016, 1:35am UTC](https://discourse.llvm.org/t/making-a-new-symbol-provider/39920/2 "2016-02-12T01:35:16Z")

</div>

> Hi,
> 
> I want to make a new symbol provider to teach LLDB to understand microsoft PDB files. I've been looking over the various symbol APIs, and I have a few questions.
> 
> 1. Under what circumstances do I need a custom SymbolVendor? The way pdb works is that generally there is 1 file that contains all the debug info needed for a single binary (so or executable). Given a list of paths, we can then determine if there is a matching PDB in one of those paths. Is it better to do this in the CalculateAbilities() function of the symbol file plugin (by just returning 0 if we don't find a match) or do we need to do something more complicated?

I would suggest make a SymbolVendorPDB that only enables itself if you are able to find the PDB files for your COFF file. So look at your COFF file, and I presume somewhere in there there is a pointer to one or more PDB files inside that file? CalculateAbililties is the correct place to see if a COFF file has pointers to PDB files and making sure those files exist before you say that you can provide any abilities.

> 2. Why is there a function called ParseCompileUnitLanguage? The CompileUnit class already stores the language when ParseCompileUnit is called, and ParseCompileUnitLanguage is implemented by just getting that value out. What is the poitn of this function?

If we are constructing CompileUnit instances with a valid language, we will never need to call the ParseCompileUnitLanguage function on SymbolVendor/SymbolFile, but it it is eLanguageTypeInvalid, we will lazily populate this later.

> 3. There's a function called ParseCompileUnitDebugMacros. Is this referring to C / C++ macros? Like #define FOO 7? What is that used for? I don't believe info about preprocessor definitions are stored in PDB. Is this going to cause problems?

Nope, just don't implement. Hopefully there is a default implementation that does nothing. We should imply that by having a default implementation for this that there is nothing wrong with not filling it in.

> 4. ParseCompileUnitSupportFiles. What are "support files"? Given a file "foo.cpp" is this supposed to be header files etc?

This is largely mirroring how DWARF structures its data, but in general a compile unit might have files that it uses for line tables and decl file for things like variables.

So any files in your line table should be in here. In DWARF the line tables use file indexes in their line tables to save space. Also any DWARF info that says "I am declared on line 12 of file 'Foo.c'" will use an index to refer to 'Foo.c'. We use the compile unit support files for this:

&nbsp;&nbsp;&nbsp;&nbsp;case DW\_AT\_decl\_file:  
&nbsp;&nbsp;decl.SetFile(sc.comp\_unit-\>GetSupportFiles().GetFileSpecAtIndex(file\_idx));  
&nbsp;&nbsp;break;

The macro support you mention above also uses file indexes when referring to files.

So the support files should be a list of files that make sense to your PDB parser in case your PDB uses file indexes when referring to files. Since LLDB uses a partial parsing style of debug info, we only expand debug info into agnostic LLDB info lazily as the information is needed. All symbol files also get to pick their own identifiers for everything. For DWARF, we use the DIE offset as the identifier. So say you parse DWARF that looks like:

0x0000000b: TAG\_compile\_unit [1] \*  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AT\_producer( "Apple LLVM version 7.0.0 (clang-700.1.72)" )  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AT\_language( DW\_LANG\_C99 )  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AT\_name( "main.c" )  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AT\_stmt\_list( 0x00000000 )  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AT\_comp\_dir( "/Volumes/work/gclayton/Documents/src/args" )  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AT\_low\_pc( 0x0000000100000cf0 )  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AT\_high\_pc( 0x0000000100000e9b )

0x0000002e: TAG\_subprogram [2] \*  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AT\_low\_pc( 0x0000000100000cf0 )  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AT\_high\_pc( 0x0000000100000e9b )  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AT\_frame\_base( rbp )  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AT\_name( "main" )  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AT\_decl\_file( "main.c" )  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AT\_decl\_line( 9 )  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AT\_prototyped( 0x01 )  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AT\_type( {0x000000c6} ( int ) )  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AT\_external( 0x01 )

0x0000004d: TAG\_formal\_parameter [3]  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AT\_location( fbreg -1048 )  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AT\_name( "argc" )  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AT\_decl\_file( "main.c" )  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AT\_decl\_line( 9 )  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AT\_type( {0x000000c6} ( int ) )

0x0000005c: TAG\_formal\_parameter [3]  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AT\_location( fbreg -1056 )  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AT\_name( "argv" )  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AT\_decl\_file( "main.c" )  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AT\_decl\_line( 9 )  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AT\_type( {0x000000cd} ( const char\*\* ) )

The ID of the compile unit is 0x0000000b since that is the DIE offset for the compile unit. If we ask the compile unit any questions through the lldb\_private::CompileUnit, we can always extract the ID from the compile unit so we know how to dig up the original DWARF info so we can parse more info lazily and only as needed.

Likewise, the TAG\_subprogram represents a function. We might parse only the function "main" at 0x0000002e, and then later be asked to parse the blocks and variables inside of it. If we use 0x0000002e for the ID of the function, we can quickly find the DWARF for it and parse its child variables and blocks.

So be sure to pick identifiers that make sense for PDB. Hopefully this will be easy.

> 5. ParseCompileUnitLineTable. On the LineTable class you can add "line sequences" or individual entries. What's the difference here? Is there any disadvantage to adding every single line entry in the line table using the InsertLineEntry instead of building a line sequence and inserting the sequence?

The rule follows DWARF line tables: line sequences must be an array of line entries whose addresses are always increasing. You can add every line in sequence as long as the line entries are in increasing address order. We are going to sort the line entries into an array that is sorted for quick lookups.

> I will probably have some more questions as I continue down this path. For now I'm planning to implement the minimum amount of functionality required just to make LLDB locate and open a PDB for an executable without actually returning anything useful from it. So when I start filling out types, functions, etc I may have some more questions.

I am the person you will need to ask as I implemented everything in the symbols so far. If you have any questions, feel free to ask and I will get back to your as quickly as I can. If I am not around, you can take a look at the DWARF spec, or talk to someone that is familiar with DWARF, and you can probably bet we are very similar to DWARF in many respects since it is a very powerful and complete format.

Let me know what questions you have! I look forward to seeing the PDB plug-in make it into LLDB.

Greg Clayton

---

<div class="post-metadata">

### Author: ![Zachary\_Turner1](https://avatars.discourse-cdn.com/v4/letter/z/90db22/32.png) [@Zachary\_Turner1](https://discourse.llvm.org/u/Zachary_Turner1)
#### Post date: [February 12, 2016, 2:56am UTC](https://discourse.llvm.org/t/making-a-new-symbol-provider/39920/3 "2016-02-12T02:56:50Z")

</div>

Currently we use the operating system to query the PDBs. This could change in the future, but for now that’s how we’re doing it. The operating system does all the work of finding, matching, and loading the PDB for us, and it does it all in one call. So if we put this in the symbol vendor, there’s no way to say “is there a PDB” without also saying “actually load all the data from the PDB” at the same time. So I’m not sure if there’s a solution to this in there, because obviously I dont’ want to load it twice.

One question I had about SymbolVendor, is that I looked at SymbolVendorELF.cpp and it seems to boil down to this notion of “symbol file representations”. All the logic in SymbolVendorELF exists just to add some object file representations. What is this supposed to represent? I’ve got an exe or something, what other “representation” is there other than the exe itself?

---

<div class="post-metadata">

### Author: ![Greg\_Clayton1](https://avatars.discourse-cdn.com/v4/letter/g/838e76/32.png) [@Greg\_Clayton1](https://discourse.llvm.org/u/Greg_Clayton1)
#### Post date: [February 12, 2016, 5:41pm UTC](https://discourse.llvm.org/t/making-a-new-symbol-provider/39920/4 "2016-02-12T17:41:13Z")

</div>

> \>  
> \> Hi,  
> \>  
> \> I want to make a new symbol provider to teach LLDB to understand microsoft PDB files. I've been looking over the various symbol APIs, and I have a few questions.  
> \>  
> \> 1. Under what circumstances do I need a custom SymbolVendor? The way pdb works is that generally there is 1 file that contains all the debug info needed for a single binary (so or executable). Given a list of paths, we can then determine if there is a matching PDB in one of those paths. Is it better to do this in the CalculateAbilities() function of the symbol file plugin (by just returning 0 if we don't find a match) or do we need to do something more complicated?
> 
> I would suggest make a SymbolVendorPDB that only enables itself if you are able to find the PDB files for your COFF file. So look at your COFF file, and I presume somewhere in there there is a pointer to one or more PDB files inside that file? CalculateAbililties is the correct place to see if a COFF file has pointers to PDB files and making sure those files exist before you say that you can provide any abilities.  
> Currently we use the operating system to query the PDBs. This could change in the future, but for now that's how we're doing it. The operating system does all the work of finding, matching, and loading the PDB for us, and it does it all in one call. So if we put this in the symbol vendor, there's no way to say "is there a PDB" without also saying "actually load all the data from the PDB" at the same time. So I'm not sure if there's a solution to this in there, because obviously I dont' want to load it twice.

Interesting. If you are on windows and you have a COFF file, you might just want to make a SymbolVendorCOFF. Does PDB info always and only get created for COFF files?

> One question I had about SymbolVendor, is that I looked at SymbolVendorELF.cpp and it seems to boil down to this notion of "symbol file representations". All the logic in SymbolVendorELF exists just to add some object file representations. What is this supposed to represent? I've got an exe or something, what other "representation" is there other than the exe itself?

In SymbolVendoerMacOSX, we have the executable and then the DWARF debug info in a stand alone dSYM bundle. So MacOSX you have a.out as the main ObjectFile (a.out) for a Module, but the symbols are in a different ObjectFile (a.out.dSYM). For ELF I believe there is information in the ELF file that \_might\_ point to a separate debug info file, but it also might just contain the DWARF in the executable. So for ELF you have 1 file (exec ELF that contains DWARF) or two files (exe ELF with no DWARF + debug info ELF with DWARF).

A symbol vendor's only job is to take an executable and and then use it plus any other files (its job is to locate these extra debug files) to make a single coherent view of the symbols for a lldb\_private::Module. So the SymbolVendor::FindTypes(...) might look into the executable file and one or more other files to get the information. The information must be retrieved from one or more SymbolFile instances. A SymbolFile uses one ObjectFile to do its job. So there is a one to one mapping between SymbolFile and ObjectFile instances. The SymbolFile can use the same ObjectFile as the main executable if the data is in there. The SymbolVendor is the one that figures this out.

So some mappings might help show. The addresses before the object names are the address of the class in the LLDB address space. For a simple a.out ELF file that contains DWARF we would have:

0x1000: Module ("/tmp/a.out")  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;m\_obj\_file = 0x2000  
0x2000: ObjectFile ("/tmp/a.out")  
0x3000: SymbolVendorELF  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;m\_sym\_file = 0x4000  
0x4000: SymbolFile  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;m\_obj\_file = 0x2000

For a a.out ELF file that contains an external debug file "/var/debug/a.out"

0x1000: Module ("/tmp/a.out")  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;m\_obj\_file = 0x2000  
0x2000: ObjectFile ("/tmp/a.out")  
0x2200: ObjectFile ("/var/debug/a.out")  
0x3000: SymbolVendorELF  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;m\_sym\_file = 0x4000  
0x4000: SymbolFile  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;m\_obj\_file = 0x2200

Same goes for MacOSX where we have "a.out" and "a.out.dSYM" except the SymbolVendorMacOSX is used since it knows how to locate the dSYM files.

If there are multiple ObjectFile objects that represent the debug info, they must share the same section list. So ObjectFiles and SymbolFiles work to make a single section list within lldb\_private::Module that is used for all objects used to represent the symbol and debug info. That way the ObjectFile at 0x2000 and 0x2200 above both use the same section for ".text", ".data", etc. If one ObjectFile has sections (like .debug\_info for DWARF) where the other ObjectFile doesn't, then each ObjectFile adds sections as needed. Also if executable object file has no symbols, or a reduced amount of symbols, since it might have been stripped, the two ObjectFiles can combine their symbol tables to make a better symbol table. On MacOSX if we strip a.out and it has no symbols, we can get the symbols from the dSYM file (if we find one) since dSYM files always have fully unstripped symbol tables.

So think of SymbolVendor as the class that knows how to locate the symbol file for a given executable (possibly even fetch the symbols from your build system!!!) and put together one or more files to provide a coherent view of the debug info (grab debug info from the executable itself or a stand alone file) and object file (combine symbol tables from one or more object files, combine all sections from all ObjectFiles used for a Module/SymbolVendor) so the use doesn't ever need to worry about the underlying details, clients just ask the module for stuff and we provide it to them.

Greg

---

<div class="post-metadata">

### Author: ![Zachary\_Turner1](https://avatars.discourse-cdn.com/v4/letter/z/90db22/32.png) [@Zachary\_Turner1](https://discourse.llvm.org/u/Zachary_Turner1)
#### Post date: [February 12, 2016, 6:25pm UTC](https://discourse.llvm.org/t/making-a-new-symbol-provider/39920/5 "2016-02-12T18:25:59Z")

</div>

> > > Hi,
> > > 
> > > I want to make a new symbol provider to teach LLDB to understand microsoft PDB files. I’ve been looking over the various symbol APIs, and I have a few questions.
> > > 
> > > 1. Under what circumstances do I need a custom SymbolVendor? The way pdb works is that generally there is 1 file that contains all the debug info needed for a single binary (so or executable). Given a list of paths, we can then determine if there is a matching PDB in one of those paths. Is it better to do this in the CalculateAbilities() function of the symbol file plugin (by just returning 0 if we don’t find a match) or do we need to do something more complicated?
> > 
> > I would suggest make a SymbolVendorPDB that only enables itself if you are able to find the PDB files for your COFF file. So look at your COFF file, and I presume somewhere in there there is a pointer to one or more PDB files inside that file? CalculateAbililties is the correct place to see if a COFF file has pointers to PDB files and making sure those files exist before you say that you can provide any abilities.  
> > Currently we use the operating system to query the PDBs. This could change in the future, but for now that’s how we’re doing it. The operating system does all the work of finding, matching, and loading the PDB for us, and it does it all in one call. So if we put this in the symbol vendor, there’s no way to say “is there a PDB” without also saying “actually load all the data from the PDB” at the same time. So I’m not sure if there’s a solution to this in there, because obviously I dont’ want to load it twice.
> 
> Interesting. If you are on windows and you have a COFF file, you might just want to make a SymbolVendorCOFF. Does PDB info always and only get created for COFF files?

Yes. What’s the disadvantage to just using the default SymbolVendor implementation, and just having the SymbolFilePDB plugin, when it’s created, attempt to locate the PDB and just return 0 abilities if it can’t find a match?

> > One question I had about SymbolVendor, is that I looked at SymbolVendorELF.cpp and it seems to boil down to this notion of “symbol file representations”. All the logic in SymbolVendorELF exists just to add some object file representations. What is this supposed to represent? I’ve got an exe or something, what other “representation” is there other than the exe itself?
> 
> In SymbolVendoerMacOSX, we have the executable and then the DWARF debug info in a stand alone dSYM bundle. So MacOSX you have a.out as the main ObjectFile (a.out) for a Module, but the symbols are in a different ObjectFile (a.out.dSYM). For ELF I believe there is information in the ELF file that _might_ point to a separate debug info file, but it also might just contain the DWARF in the executable. So for ELF you have 1 file (exec ELF that contains DWARF) or two files (exe ELF with no DWARF + debug info ELF with DWARF).

Why does AddSymbolFileRepresentation take an ObjectFile though? A PDB is not an object file, so if we went and looked for a matching PDB, downloaded it from our build server, and wanted to use it, we couldn’t exactly call AddSymbolFileRepresentation(pdb) because we wouldn’t have an ObjectFile, we’d have a PDB. Maybe we’d need an overload of this function that just takes a SymbolFile\* directly so that the vendor could add the SymbolFile it finds.

> A symbol vendor’s only job is to take an executable and and then use it plus any other files (its job is to locate these extra debug files) to make a single coherent view of the symbols for a lldb\_private::Module. So the SymbolVendor::FindTypes(…) might look into the executable file and one or more other files to get the information. The information must be retrieved from one or more SymbolFile instances. A SymbolFile uses one ObjectFile to do its job. So there is a one to one mapping between SymbolFile and ObjectFile instances. The SymbolFile can use the same ObjectFile as the main executable if the data is in there. The SymbolVendor is the one that figures this out.

So in this case, there are no extra files. One object file (COFF) is all you need to locate the 1 PDB file. So we should be able to assume a 1-to-1 mapping between modules and SymbolFile instances.

> So think of SymbolVendor as the class that knows how to locate the symbol file for a given executable (possibly even fetch the symbols from your build system!!!)

We will definitely need something like this later, so I could see this being useful down the road. For now it seems like we can get by with just having a SymbolFilePDB class and using the default vendor implementation.

---

<div class="post-metadata">

### Author: ![Zachary\_Turner1](https://avatars.discourse-cdn.com/v4/letter/z/90db22/32.png) [@Zachary\_Turner1](https://discourse.llvm.org/u/Zachary_Turner1)
#### Post date: [February 12, 2016, 7:01pm UTC](https://discourse.llvm.org/t/making-a-new-symbol-provider/39920/6 "2016-02-12T19:01:18Z")

</div>

Just to make sure I understand, semantically here, there is nothing special about a line sequence, it’s just an optimization to let LineTable know you’re giving it sorted values? So any lines you add via a LineSequence, could also be added individually with insertLine, but it would be slower? And aside from that everything else would still work as expected?

---

<div class="post-metadata">

### Author: ![Zachary\_Turner1](https://avatars.discourse-cdn.com/v4/letter/z/90db22/32.png) [@Zachary\_Turner1](https://discourse.llvm.org/u/Zachary_Turner1)
#### Post date: [March 1, 2016, 1:09am UTC](https://discourse.llvm.org/t/making-a-new-symbol-provider/39920/7 "2016-03-01T01:09:13Z")

</div>

Suppose you’ve got two line sequences.

First sequence:  
4198512  
4198544  
4198547  
4198562

Second sequence:  
4198528  
4198531  
4198537  
4198552

These two line sequences overlap, and will not be inserted correctly into a LineTable. This sounds like a bug to me, because as far as I can tell there is nothing preventing LineSequences being organized this way, but LineTable::InsertSequence assumes that this cannot happen.

---

<div class="post-metadata">

### Author: ![Greg\_Clayton1](https://avatars.discourse-cdn.com/v4/letter/g/838e76/32.png) [@Greg\_Clayton1](https://discourse.llvm.org/u/Greg_Clayton1)
#### Post date: [March 1, 2016, 1:29am UTC](https://discourse.llvm.org/t/making-a-new-symbol-provider/39920/8 "2016-03-01T01:29:26Z")

</div>

Are these addresses or line numbers? If PDB can has its line tables randomly ordered, you will need to read all line entries out of the PDB file first into one big collection of all line entries, and remember if any are line termination entries and then make sequences out of the large collection you end up with.

We only expect to get one line entry for a given load address and I believe that is reasonable. The registration process for line entries is very DWARF centric right now, but I think that the end goal if having line table sequences in ascending order that have a single file and line for a given load address is a valid assumption.

So it sounds like you just need to read all of your stuff into one large buffer and then figure out how you want to register those sequences so they make sense.

In DWARF if you have a line tables like:

0x1000: foo.c line 1  
0x1010: foo.c line 2  
0x1030: foo.c line 3  
0x1040: termination entry

0x2000: foo.c line 11  
0x2010: foo.c line 12  
0x2020: foo.c line 13  
0x2050: termination entry

These are both sequences that define address ranges and we always know the address range of each line entry because the last entry in a contiguous line sequence always has a last entry to define the size of the last valid (non termination entry) line entry.

How does the PDB file format emit it line table entries? Would it be equivalent to the DWARF with no termination entries? Something like:

0x1000: foo.c line 1  
0x1010: foo.c line 2  
0x1030: foo.c line 3

0x2000: foo.c line 11  
0x2010: foo.c line 12  
0x2020: foo.c line 13

If that is the case, how do you deal with large gaps like the gap between 0x1040 and 0x2000?

And if I read what you are saying correctly you are saying your line tables might come out like:

0x1000: foo.c line 1  
0x2000: foo.c line 11  
0x1010: foo.c line 2  
0x2010: foo.c line 12  
0x1030: foo.c line 3  
0x2020: foo.c line 13

Questions:

- Does PDB emit ranges for each line or just a single address?  
- Does PDB have termination entries for the equivalent of address 0x1040 and 0x2050?

---

<div class="post-metadata">

### Author: ![Zachary\_Turner1](https://avatars.discourse-cdn.com/v4/letter/z/90db22/32.png) [@Zachary\_Turner1](https://discourse.llvm.org/u/Zachary_Turner1)
#### Post date: [March 1, 2016, 1:49am UTC](https://discourse.llvm.org/t/making-a-new-symbol-provider/39920/9 "2016-03-01T01:49:35Z")

</div>

Those are addresses. Here’s the situation I was encountering this on:

// foo.h  
#include “bar.h”  
inline int f(int n)  
{  
return g(n) + 1;  
}

// bar.h  
inline int g(int n)  
{  
return n+1;  
}

// foo.cpp  
#include “foo.h”  
int main(int argc, char\*\* argv)  
{  
return f(argc);  
}

PDB gives me back line numbers and address range grouped by file. So I get all of foo.h’s lines, all of bar.h’s lines, and all of foo.cpp’s lines. In sorted form, the lines for g will appear inside the sequence of lines for f. So that’s how the situation was arising.

I’ll upload a patch tomorrow morning, and along with it a test (which is how I found this to begin with). If you look at the test you will see the exact source code and the line / address sequences that are generated.

I think I have everything working, and I wrote some test cases to validate my assumptions. Which is good because every test was broken at first, so I wouldn’t have been able to fix things without them. They shoudl also help verify that my assumptions are correct to begin with, which hopefully makes reviewing easier.

---

<div class="post-metadata">

### Author: ![Zachary\_Turner1](https://avatars.discourse-cdn.com/v4/letter/z/90db22/32.png) [@Zachary\_Turner1](https://discourse.llvm.org/u/Zachary_Turner1)
#### Post date: [March 1, 2016, 1:51am UTC](https://discourse.llvm.org/t/making-a-new-symbol-provider/39920/10 "2016-03-01T01:51:20Z")

</div>

Just to clarify here. When I was encountering this problem, I would create one LineSequence for foo.h’s lines, one LineSequence for bar.h’s lines, and one for foo.cpp’s. And each one is monotonically increasing, but the ranges can overlap as per the previous explanation, which was causing InsertLineSequence to fail.

---

<div class="post-metadata">

### Author: ![Greg\_Clayton1](https://avatars.discourse-cdn.com/v4/letter/g/838e76/32.png) [@Greg\_Clayton1](https://discourse.llvm.org/u/Greg_Clayton1)
#### Post date: [March 1, 2016, 6:33pm UTC](https://discourse.llvm.org/t/making-a-new-symbol-provider/39920/11 "2016-03-01T18:33:19Z")

</div>

I understand now. Yes, you will need to parse all line entries one big buffer, sort them by address, and then figure out what sequences to submit after this.

Is there a termination entry for the last line entry in a function? Lets say there were 4096 byte gaps between "f" and "g" and "main"? Are there termination entries for the last '}' in each function so that when you put all of the line entries into one large collection and sort them by address, that you know there is a gap between the line entries? This is very important to get right. If there aren't termination entries, you will need to add them manually by looking up each line entry address and find the address range of the function (which you can cache at the time of making the line sequences from the sorted PDB line entries) and add termination entries for the ends of functions. So lets say f starts at 0x1000 and the "inline int f" is on line 3, g starts at 0x2000 and main starts at 0x3000, you don't want you line table looking like a single sequence:

0x1000: foo.cpp line 4 // {  
0x1010: foo.cpp line 5 // return g(n) + 1;  
0x1020: foo.cpp line 6 // }  
0x2000: foo.cpp line 10 // {  
0x2010: foo.cpp line 11 // return n+1;  
0x2020: foo.cpp line 12 // }  
0x3000: foo.cpp line 17 // {  
0x3010: foo.cpp line 18 // return f(argc);  
0x3020: foo.cpp line 19 // }

If you don't have termination entries, we will think foo.cpp:6 goes from [0x1020-0x2000) which is probably now what we want.

There should be termination entries between the functions so that the line entries do not contain gaps between functions in their address ranges. So you should actually have 3 sequences in the line table:

0x1000: foo.cpp line 4 // {  
0x1010: foo.cpp line 5 // return g(n) + 1;  
0x1020: foo.cpp line 6 // }  
0x1030: END

0x2000: foo.cpp line 10 // {  
0x2010: foo.cpp line 11 // return n+1;  
0x2020: foo.cpp line 12 // }  
0x2030: END

0x3000: foo.cpp line 17 // {  
0x3010: foo.cpp line 18 // return f(argc);  
0x3020: foo.cpp line 19 // }  
0x3030: END

0x1030, 0x2030 and 0x3030 are the end addresses of the functions f, g and main respectively. So if your line table only contains start addresses, you will need to inject these correctly otherwise source level single step can do the wrong thing since it uses line entry address ranges to implement the steps.

Greg

---

<div class="post-metadata">

### Author: ![Zachary\_Turner1](https://avatars.discourse-cdn.com/v4/letter/z/90db22/32.png) [@Zachary\_Turner1](https://discourse.llvm.org/u/Zachary_Turner1)
#### Post date: [March 1, 2016, 7:30pm UTC](https://discourse.llvm.org/t/making-a-new-symbol-provider/39920/12 "2016-03-01T19:30:56Z")

</div>

We do know the last line of a function. In the review i posted, you can see the condition where i set is\_epilogue to true. That is the last line of a function corresponding to the } (although the function may contain additional bytes, since that only refers to the first byte of the epilogue.

But I don’t know if it’s appropriate to set is\_terminal\_entry to true here because that’s a valid line with a valid address. Terminal entries are ignored when doing address lookups so this line would never be found when looking up that address.

What i might be able to do is figure out the size of the epilogue and inject a new entry with address=epilogue\_addr+epilogue\_size and make that the termination entry does that work? If so what should i set for its line number?

Just to make sure I understand, does “terminal entry” specifically mean the end of a _function_? Reading the code I thought it meant the end of a LineSequence

---

<div class="post-metadata">

### Author: ![Greg\_Clayton1](https://avatars.discourse-cdn.com/v4/letter/g/838e76/32.png) [@Greg\_Clayton1](https://discourse.llvm.org/u/Greg_Clayton1)
#### Post date: [March 1, 2016, 7:36pm UTC](https://discourse.llvm.org/t/making-a-new-symbol-provider/39920/13 "2016-03-01T19:36:52Z")

</div>

> We do know the last line of a function. In the review i posted, you can see the condition where i set is\_epilogue to true. That is the last line of a function corresponding to the } (although the function may contain additional bytes, since that only refers to the first byte of the epilogue.

I don't believe any compilers set is\_eqilogue correctly yet.

> But I don't know if it's appropriate to set is\_terminal\_entry to true here because that's a valid line with a valid address. Terminal entries are ignored when doing address lookups so this line would never be found when looking up that address.

The "is\_terminal\_entry" are to provide the address range for the last line in a sequence. It's line number doesn't mean anything, but it is typically the same as the previous one.

> What i might be able to do is figure out the size of the epilogue and inject a new entry with address=epilogue\_addr+epilogue\_size and make that the termination entry does that work? If so what should i set for its line number?

The line number can just be the same as the previous one. We need to make sure we cover every byte of a function with a valid line entry. Anywhere the user can actually stop should have a valid line entry when possible.

> Just to make sure I understand, does "terminal entry" specifically mean the end of a \*function\*? Reading the code I thought it meant the end of a LineSequence

No, it just is there to indicate that it terminates the previous line entry since line entries are stored with start address only. If a function is discontiguous, or if it has data in the middle, a function might have multiple sequences. So the terminal entry is just to provide an address range for the last line entry in a contiguous address range of line entries.

---

<div class="post-metadata">

### Author: ![Zachary\_Turner1](https://avatars.discourse-cdn.com/v4/letter/z/90db22/32.png) [@Zachary\_Turner1](https://discourse.llvm.org/u/Zachary_Turner1)
#### Post date: [March 1, 2016, 7:45pm UTC](https://discourse.llvm.org/t/making-a-new-symbol-provider/39920/14 "2016-03-01T19:45:50Z")

</div>

Alright I think i get it. Basically terminal entry means “everything from this address onwards is not part of any function”

---

<div class="post-metadata">

### Author: ![Greg\_Clayton1](https://avatars.discourse-cdn.com/v4/letter/g/838e76/32.png) [@Greg\_Clayton1](https://discourse.llvm.org/u/Greg_Clayton1)
#### Post date: [March 1, 2016, 8:38pm UTC](https://discourse.llvm.org/t/making-a-new-symbol-provider/39920/15 "2016-03-01T20:38:59Z")

</div>

Yes
