[RFC] New ClangDebuggerSupport Library

Hello cfe-dev,

I would like to propose a new Clang library for implementing functionality that is used by LLDB. I see this as the first step in a long process of refactoring the language interfaces for LLDB.

The short-term goal is for this library is to be a place for us to rebuild functionality that exists in LLDB today and relies heavily on the implementation of Clang. As we rebuild the functionality we will build a suite of testing tools in Clang that exercise this library and more general Clang functionality in the same ways that LLDB will.

As bits of functionality become fully implemented and tested, we will migrate LLDB to using the Clang implementations, allowing LLDB to remove its own copies. This will provide the Clang community with a higher confidence that changes in Clang do not break LLDB, and it will provide LLDB with better test coverage of the Clang functionality.

The long-term goal of this library is to provide the implementation for what could some day become a defined debugger<->frontend interface for providing modularized (maybe even plugin-based) language debugging support in LLDB. In the distant future I could see us being able to tell people building new frontends that we have a defined interface they need to implement for the debugger, and once implemented the debugger should “Just Work”.

The first bit of functionality that I would like to build up into the ClangDebuggerSupport library is materialization of Clang AST types from DWARF. To support this development I intend to add a new tool in Clang that reads DWARF types, generates a Clang AST, and prints the AST. I will also add DWARF support to obj2yaml and yaml2obj, so we will be able to write YAML LIT tests for the functionality.

If people are in favor of this general approach I’ll begin working in this direction, and I’ll probably add the new library sometime next month.

Thoughts?
-Chris

From: cfe-dev [mailto:cfe-dev-bounces@lists.llvm.org] On Behalf Of Chris
Bieneman via cfe-dev
Sent: Wednesday, November 09, 2016 2:26 PM
To: cfe-dev
Subject: [cfe-dev] [RFC] New ClangDebuggerSupport Library

Hello cfe-dev,

I would like to propose a new Clang library for implementing functionality
that is used by LLDB. I see this as the first step in a long process of
refactoring the language interfaces for LLDB.

The short-term goal is for this library is to be a place for us to rebuild
functionality that exists in LLDB today and relies heavily on the
implementation of Clang. As we rebuild the functionality we will build a
suite of testing tools in Clang that exercise this library and more
general Clang functionality in the same ways that LLDB will.

As bits of functionality become fully implemented and tested, we will
migrate LLDB to using the Clang implementations, allowing LLDB to remove
its own copies. This will provide the Clang community with a higher
confidence that changes in Clang do not break LLDB, and it will provide
LLDB with better test coverage of the Clang functionality.

The long-term goal of this library is to provide the implementation for
what could some day become a defined debugger<->frontend interface for
providing modularized (maybe even plugin-based) language debugging support
in LLDB. In the distant future I could see us being able to tell people
building new frontends that we have a defined interface they need to
implement for the debugger, and once implemented the debugger should “Just
Work”.

The first bit of functionality that I would like to build up into the
ClangDebuggerSupport library is materialization of Clang AST types from
DWARF. To support this development I intend to add a new tool in Clang
that reads DWARF types, generates a Clang AST, and prints the AST. I will
also add DWARF support to obj2yaml and yaml2obj, so we will be able to
write YAML LIT tests for the functionality.

I am not familiar with obj2yaml and yaml2obj, and I'm not finding any real
documentation for them on the website. But I can guess something about
them, and if I'm guessing anywhere close, having them understand DWARF
seems like a useful testing tool for debug info, independent of all the
rest of your proposal.

Would you be using LLVM's existing DWARF reader? Or replacing it with
something from LLDB? Adding a DWARF reader to Clang doesn't seem like
the right choice, the object-file details really belong to LLVM.

To achieve your longer-term goal, LLDB would need to be completely
isolated from AST and have no "native" understanding of the format.
And yet, a library capable of evaluating a source-language expression
would need to be able to call back into the debugger in order to read
and write process memory. An interesting design challenge for sure.
--paulr

From: cfe-dev [mailto:cfe-dev-bounces@lists.llvm.org] On Behalf Of Chris
Bieneman via cfe-dev
Sent: Wednesday, November 09, 2016 2:26 PM
To: cfe-dev
Subject: [cfe-dev] [RFC] New ClangDebuggerSupport Library

Hello cfe-dev,

I would like to propose a new Clang library for implementing functionality
that is used by LLDB. I see this as the first step in a long process of
refactoring the language interfaces for LLDB.

The short-term goal is for this library is to be a place for us to rebuild
functionality that exists in LLDB today and relies heavily on the
implementation of Clang. As we rebuild the functionality we will build a
suite of testing tools in Clang that exercise this library and more
general Clang functionality in the same ways that LLDB will.

As bits of functionality become fully implemented and tested, we will
migrate LLDB to using the Clang implementations, allowing LLDB to remove
its own copies. This will provide the Clang community with a higher
confidence that changes in Clang do not break LLDB, and it will provide
LLDB with better test coverage of the Clang functionality.

The long-term goal of this library is to provide the implementation for
what could some day become a defined debugger<->frontend interface for
providing modularized (maybe even plugin-based) language debugging support
in LLDB. In the distant future I could see us being able to tell people
building new frontends that we have a defined interface they need to
implement for the debugger, and once implemented the debugger should “Just
Work”.

The first bit of functionality that I would like to build up into the
ClangDebuggerSupport library is materialization of Clang AST types from
DWARF. To support this development I intend to add a new tool in Clang
that reads DWARF types, generates a Clang AST, and prints the AST. I will
also add DWARF support to obj2yaml and yaml2obj, so we will be able to
write YAML LIT tests for the functionality.

I am not familiar with obj2yaml and yaml2obj, and I’m not finding any real
documentation for them on the website. But I can guess something about
them, and if I’m guessing anywhere close, having them understand DWARF
seems like a useful testing tool for debug info, independent of all the
rest of your proposal.

obj2yaml and yaml2obj are testing tools in LLVM used for testing binary files. I agree having DWARF support would improve testability for DWARF all across LLVM & Clang.

Would you be using LLVM’s existing DWARF reader? Or replacing it with
something from LLDB? Adding a DWARF reader to Clang doesn’t seem like
the right choice, the object-file details really belong to LLVM.

Greg Clayton is working to move LLDB to using LLVM’s DWARF reader, so we would use LLVM’s reader in the Clang APIs as well. Part of why I’m proposing this be in a new library is so that Clang itself doesn’t need to pull in the DWARF reader from LLVM, but we won’t be adding any dependencies from Clang on LLDB.

To achieve your longer-term goal, LLDB would need to be completely
isolated from AST and have no “native” understanding of the format.
And yet, a library capable of evaluating a source-language expression
would need to be able to call back into the debugger in order to read
and write process memory. An interesting design challenge for sure.

I’ve been talking with Lang Hames about ways to abstract the remote JIT APIs to support this end goal. Hopefully we won’t need the language support library to have any access to the remote process. That said, this is all just my pipe dream at the moment.

-Chris

The first bit of functionality that I would like to build up into the
ClangDebuggerSupport library is materialization of Clang AST types from
DWARF. To support this development I intend to add a new tool in Clang that
reads DWARF types, generates a Clang AST, and prints the AST. I will also
add DWARF support to obj2yaml and yaml2obj, so we will be able to write
YAML LIT tests for the functionality.

I'd be happy to see this. I have C/C++ support in the julia debugger, but
it relies on re-parsing the headers from source, instead of reconstructing
things from DWARF. I don't care enough to actually split the DWARF->AST
functionality out of LLDB myself, but I'd be happy to use it and
collaborate it it's packaged in a reasonably reusable way.

To achieve your longer-term goal, LLDB would need to be completely
isolated from AST and have no "native" understanding of the format.
And yet, a library capable of evaluating a source-language expression
would need to be able to call back into the debugger in order to read
and write process memory. An interesting design challenge for sure.
                        
I’ve been talking with Lang Hames about ways to abstract the remote JIT
APIs to support this end goal. Hopefully we won’t need the language
support library to have any access to the remote process. That said, this
is all just my pipe dream at the moment.

My impression (not having ever used it) is that the JIT basically is
premised on having complete functions executing in their own context,
rather than expressions or individual statements executing within an
existing call frame and likely to need to access variables optimized
into registers. That's a hefty chunk of context LLDB would have to
produce at each stopping point, if it doesn't want to understand
anything about the source-language constructs you're expecting your
users to type in. I'm going to stop rambling about it at this point,
because it's really straying from your original question (and I have
nothing further to contribute there), but it would be kind of fun to
hack away at this during a social sometime.
--paulr

Hello cfe-dev,

I would like to propose a new Clang library for implementing functionality that is used by LLDB. I see this as the first step in a long process of refactoring the language interfaces for LLDB.

The short-term goal is for this library is to be a place for us to rebuild functionality that exists in LLDB today and relies heavily on the implementation of Clang. As we rebuild the functionality we will build a suite of testing tools in Clang that exercise this library and more general Clang functionality in the same ways that LLDB will.

As bits of functionality become fully implemented and tested, we will migrate LLDB to using the Clang implementations, allowing LLDB to remove its own copies. This will provide the Clang community with a higher confidence that changes in Clang do not break LLDB, and it will provide LLDB with better test coverage of the Clang functionality.

The long-term goal of this library is to provide the implementation for what could some day become a defined debugger<->frontend interface for providing modularized (maybe even plugin-based) language debugging support in LLDB. In the distant future I could see us being able to tell people building new frontends that we have a defined interface they need to implement for the debugger, and once implemented the debugger should “Just Work”.

I think this is an overall great idea.

The first bit of functionality that I would like to build up into the ClangDebuggerSupport library is materialization of Clang AST types from DWARF. To support this development I intend to add a new tool in Clang that reads DWARF types, generates a Clang AST, and prints the AST. I will also add DWARF support to obj2yaml and yaml2obj, so we will be able to write YAML LIT tests for the functionality.

My understanding is that yaml2obj is a tool that allows us to generate (malformed) object files from a textual description so we can test tools like llvm-objdump. For a subset of DWARF, we already have a tool that translates DWARF from a textual description into an object file: the assembler. Currently it only supports the line table via .loc directives, but I imagine we could quite naturally add assembler directives for debug info DW_TAGs as well. Have you put any thought into this option?

-- adrian

Hello cfe-dev,

I would like to propose a new Clang library for implementing functionality that is used by LLDB. I see this as the first step in a long process of refactoring the language interfaces for LLDB.

The short-term goal is for this library is to be a place for us to rebuild functionality that exists in LLDB today and relies heavily on the implementation of Clang. As we rebuild the functionality we will build a suite of testing tools in Clang that exercise this library and more general Clang functionality in the same ways that LLDB will.

As bits of functionality become fully implemented and tested, we will migrate LLDB to using the Clang implementations, allowing LLDB to remove its own copies. This will provide the Clang community with a higher confidence that changes in Clang do not break LLDB, and it will provide LLDB with better test coverage of the Clang functionality.

The long-term goal of this library is to provide the implementation for what could some day become a defined debugger<->frontend interface for providing modularized (maybe even plugin-based) language debugging support in LLDB. In the distant future I could see us being able to tell people building new frontends that we have a defined interface they need to implement for the debugger, and once implemented the debugger should “Just Work”.

The first bit of functionality that I would like to build up into the ClangDebuggerSupport library is materialization of Clang AST types from DWARF. To support this development I intend to add a new tool in Clang that reads DWARF types, generates a Clang AST, and prints the AST. I will also add DWARF support to obj2yaml and yaml2obj, so we will be able to write YAML LIT tests for the functionality.

If people are in favor of this general approach I’ll begin working in this direction, and I’ll probably add the new library sometime next month.

Thoughts?

My knee-jerk reaction is that this could just as well live in LLDB if there are no other users - though Keno points out he could use it in Julia, so that seems like potentially sufficient interest to demonstrate that it might be a reasonable/reusable abstraction.

I’ll leave it up to Richard or others to weigh in on whether the ongoing cost to the Clang project would be worth the benefit to these users/use cases.

Beyond that - as for testing infrastructure. I’d like to make sure we only have one piece of code for generating DWARF, as we are trying to have one piece of code for reading it (& perhaps in some far off future - we could have one piece of code that does both, to some degree - at least some more shared information about the size/representation of forms, etc) - so however we’re generating DWARF, it’d be good if that used the APIs in llvm’s lib/CodeGen/AsmPrinter (potentially refactored out to some more general purpose location & generalized somewhat).

Not sure exactly what sort of syntax (how low level - presumably not raw bytes written in hex or anything) you’re thinking of for writing DWARF - wonder if we could have a syntax that’s nice enough to read that it’s either the standard form llvm-dwarfdump could use, or at least a flag to enable it? (obviously dwarfdump wants to print things like byte offsets, etc - but a higher level symbolic rendering would probably be nice for many uses and be more likely what you’d want to write when writing a test case)

  • Dave

I think this functionality should live in the clang codebase. The AST has a
surprising number of invariants that are easy to violate. Right now LLDB is
super tightly coupled to clang because it has to carefully learn and
satisfy those invariants.

The approach taken in LLDB is to essentially "deserialize" DWARF into ASTs.
If we consider DWARF to be a stable serialization of the AST, then the
reading logic should live near the writing logic, which is currently in
clang/lib/CodeGen/CGDebugInfo.cpp.

I’m not sure having it in the Clang codebase is going to make that much/any easier to satisfy, it may shift the burden to the Clang project/developers which is why I wonder whether it’s a cost worth adding to the Clang project/gating Clang changes on. If it’s a service Clang (as a project) thinks is worth vending to users, sure enough.

I wouldn’t really think of it that way - for one thing, there’s lots of DWARF generated by things other than Clang (even for C++), so I don’t see the symmetry quite as much here.

Just my 2c.

  • Dave

(re-adding cfe-dev)

Hello cfe-dev,

I would like to propose a new Clang library for implementing functionality that is used by LLDB. I see this as the first step in a long process of refactoring the language interfaces for LLDB.

The short-term goal is for this library is to be a place for us to rebuild functionality that exists in LLDB today and relies heavily on the implementation of Clang. As we rebuild the functionality we will build a suite of testing tools in Clang that exercise this library and more general Clang functionality in the same ways that LLDB will.

As bits of functionality become fully implemented and tested, we will migrate LLDB to using the Clang implementations, allowing LLDB to remove its own copies. This will provide the Clang community with a higher confidence that changes in Clang do not break LLDB, and it will provide LLDB with better test coverage of the Clang functionality.

The long-term goal of this library is to provide the implementation for what could some day become a defined debugger<->frontend interface for providing modularized (maybe even plugin-based) language debugging support in LLDB. In the distant future I could see us being able to tell people building new frontends that we have a defined interface they need to implement for the debugger, and once implemented the debugger should “Just Work”.

I think this is an overall great idea.

The first bit of functionality that I would like to build up into the ClangDebuggerSupport library is materialization of Clang AST types from DWARF. To support this development I intend to add a new tool in Clang that reads DWARF types, generates a Clang AST, and prints the AST. I will also add DWARF support to obj2yaml and yaml2obj, so we will be able to write YAML LIT tests for the functionality.

My understanding is that yaml2obj is a tool that allows us to generate (malformed) object files from a textual description so we can test tools like llvm-objdump. For a subset of DWARF, we already have a tool that translates DWARF from a textual description into an object file: the assembler. Currently it only supports the line table via .loc directives, but I imagine we could quite naturally add assembler directives for debug info DW_TAGs as well. Have you put any thought into this option?

The yaml<->obj tools allow generation of both structurally valid and malformed object files. Adding DWARF support would allow bit-for-bit identical round-tripping between YAML and DWARF which should make it easier to write and generate test cases that are human readable.

If the test is only concerned with the DWARF representation needing to use the assembler radically expands the surface area of LLVM code required to execute the tests. By contrast yaml2obj relies on very little of LLVM; the MachO side of yaml2obj only relies on libSupport and libObjectYAML, and we avoid complexity and code duplication by making the YAML format very explicit so the binary writer is simple.

Also, the binary representation of DWARF in .s files is kinda not great for fabricating test cases from anything other than sources run through clang. Using YAML we could actually hand craft DWARF to drive testing.

-Chris

I’m not sure having it in the Clang codebase is going to make that much/any easier to satisfy, it may shift the burden to the Clang project/developers which is why I wonder whether it’s a cost worth adding to the Clang project/gating Clang changes on. If it’s a service Clang (as a project) thinks is worth vending to users, sure enough.

I think that shifting the burden of keeping it working to Clang is good for anyone who wants LLDB to be a stable and reliable debugger, which I think is good for (the Royal) LLVM.

I wouldn’t really think of it that way - for one thing, there’s lots of DWARF generated by things other than Clang (even for C++), so I don’t see the symmetry quite as much here.

I agree with Dave here. We can’t really think of DWARF as a bi-directional serialization to ASTs. We really need to be able to generate reasonable ASTs from DWARF created by multiple versions of Clang, GCC, ICC, and any other C/C++/Obj-C(/Julia) compiler anyone wants LLDB to support.

-Chris

Hello cfe-dev,

I would like to propose a new Clang library for implementing functionality that is used by LLDB. I see this as the first step in a long process of refactoring the language interfaces for LLDB.

The short-term goal is for this library is to be a place for us to rebuild functionality that exists in LLDB today and relies heavily on the implementation of Clang. As we rebuild the functionality we will build a suite of testing tools in Clang that exercise this library and more general Clang functionality in the same ways that LLDB will.

As bits of functionality become fully implemented and tested, we will migrate LLDB to using the Clang implementations, allowing LLDB to remove its own copies. This will provide the Clang community with a higher confidence that changes in Clang do not break LLDB, and it will provide LLDB with better test coverage of the Clang functionality.

The long-term goal of this library is to provide the implementation for what could some day become a defined debugger<->frontend interface for providing modularized (maybe even plugin-based) language debugging support in LLDB. In the distant future I could see us being able to tell people building new frontends that we have a defined interface they need to implement for the debugger, and once implemented the debugger should “Just Work”.

I think this is an overall great idea.

The first bit of functionality that I would like to build up into the ClangDebuggerSupport library is materialization of Clang AST types from DWARF. To support this development I intend to add a new tool in Clang that reads DWARF types, generates a Clang AST, and prints the AST. I will also add DWARF support to obj2yaml and yaml2obj, so we will be able to write YAML LIT tests for the functionality.

My understanding is that yaml2obj is a tool that allows us to generate (malformed) object files from a textual description so we can test tools like llvm-objdump. For a subset of DWARF, we already have a tool that translates DWARF from a textual description into an object file: the assembler. Currently it only supports the line table via .loc directives, but I imagine we could quite naturally add assembler directives for debug info DW_TAGs as well. Have you put any thought into this option?

The yaml<->obj tools allow generation of both structurally valid and malformed object files. Adding DWARF support would allow bit-for-bit identical round-tripping between YAML and DWARF which should make it easier to write and generate test cases that are human readable.

If the test is only concerned with the DWARF representation needing to use the assembler radically expands the surface area of LLVM code required to execute the tests. By contrast yaml2obj relies on very little of LLVM; the MachO side of yaml2obj only relies on libSupport and libObjectYAML, and we avoid complexity and code duplication by making the YAML format very explicit so the binary writer is simple.

Ok.

Also, the binary representation of DWARF in .s files is kinda not great for fabricating test cases from anything other than sources run through clang. Using YAML we could actually hand craft DWARF to drive testing.

That is why I said that we could add higher-level assembler directives to represent DWARF TAGs (similar to how we have .loc directives for describing the line table which are translated into the DWARF state machine by the assembler).

– adrian

Adding assembler directives means the tests run only for those targets that use the integrated assembler?

I think a custom tool would be preferable, although as David said it should reuse LLVM’s DIE classes if at all possible.

–paulr

Better join this thread late than never.

I have a testing tool under development (up for preliminary review in the next few days!) that would use the functionality you propose. The tool parses expressions that have references to external types, does some user-configurable logging, and optionally emits the resulting LLVM IR. It is intended to provide better coverage for ExternalASTSource and ASTImporter, and to generate more representative uses of the code paths that LLDB exercises. Once it is cleaned up and works the way we want it to, the heart of it ("Parse expression E with respect to DeclContexts D1, D2, D3, ... in ASTContexts A1, A2, A3, ...") would be very much at home in a library like the one you describe.

Currently the testing tool is intended to generate external types by feeding source files to separate CompilerInstances. It then uses the ExternalASTSource and ASTImporter to pull them in as necessary to the main expression's ASTContext. If your functionality can provide a pair of [ASTContext, FileManager] (so that source locations could actually come from files!) then those could just be plugged into my tool, no problem. I could see the DWARF->AST function and the (Expression, [AST]) -> IR function acting as "eval" for Clang, allowing Clang to provide embedded expression parsing functionality independent of LLDB. We'd also need to figure out how to JIT the IR in a way that makes it actually access the variables where they are in the target program. That's a problem I intend to tackle in the future.

LLDB could then adopt all of these functions as a client, and the interface to LLDB would be – as you've mentioned – significantly simplified. I see a bigger benefit to Clang's clients, though: this provides the beginnings of a more dynamic compilation model, allowing both new runtime features and separate analysis tools.

Sean

>
> Hello cfe-dev,
>
> I would like to propose a new Clang library for implementing
functionality that is used by LLDB. I see this as the first step in a long
process of refactoring the language interfaces for LLDB.
>
> The short-term goal is for this library is to be a place for us to
rebuild functionality that exists in LLDB today and relies heavily on the
implementation of Clang. As we rebuild the functionality we will build a
suite of testing tools in Clang that exercise this library and more general
Clang functionality in the same ways that LLDB will.
>
> As bits of functionality become fully implemented and tested, we will
migrate LLDB to using the Clang implementations, allowing LLDB to remove
its own copies. This will provide the Clang community with a higher
confidence that changes in Clang do not break LLDB, and it will provide
LLDB with better test coverage of the Clang functionality.
>
> The long-term goal of this library is to provide the implementation for
what could some day become a defined debugger<->frontend interface for
providing modularized (maybe even plugin-based) language debugging support
in LLDB. In the distant future I could see us being able to tell people
building new frontends that we have a defined interface they need to
implement for the debugger, and once implemented the debugger should “Just
Work”.

I think this is an overall great idea.

>
> The first bit of functionality that I would like to build up into the
ClangDebuggerSupport library is materialization of Clang AST types from
DWARF. To support this development I intend to add a new tool in Clang that
reads DWARF types, generates a Clang AST, and prints the AST. I will also
add DWARF support to obj2yaml and yaml2obj, so we will be able to write
YAML LIT tests for the functionality.

My understanding is that yaml2obj is a tool that allows us to generate
(malformed) object files from a textual description so we can test tools
like llvm-objdump.

Small correction / hostorical anecdote:

That is true for the COFF and Mach-O formats of yaml2obj. For historical
reasons though, the ELF part of yaml2obj is different and makes it quite
difficult actually to create malformed object files (e.g. wild offsets or
section numbers). The design goal for the ELF part of yaml2obj was to serve
as a testing tool for LLD: you would take an interesting object file, run
obj2yaml, then pare it down in your editor and use that as a test input by
yaml2obj'ing it in the test (so you had to have reasonable safety
guarantees when editing the YAML that you wouldn't create a corrupt object
file).

This was used extensively for testing the Atom ELF LLD, but now that it is
gone, there are very few users of the ELF yaml2obj (early in lld/ELF's
development it was decided that .s files would be used instead of
yaml2obj). Arguably, it might be better to change the ELF part of yaml2obj
to be consistent with the other two formats now that its original design
purpose is mostly gone. Or maybe just delete it.

-- Sean Silva

I realize work is already underway/being committed here, but figured discussing the following in this thread rather than on some random commit email might be better.

We now have two ways of generating DWARF, both committed in relation to a similar effort to integrate LLDB better with teh rest of the LLVM project.

There’s this YAML effort, to help test the library that will allow the generation of Clang ASTs from DWARF. (currently such code resides in LLDB, and it’s proposing to be rolled up into Clang here)

Then there’s Greg’s effort to provide a unit test API for generating DWARF for unit testing LLVM’s DWARF parsing APIs for use in LLDB (currently what LLVM has was a fork of LLDB’s, and Greg’s working on reconciling that, rolling in LLDB’s post-fork features, then migrating LLDB to use the fully featured LLVM version)

Why are these done in two different ways? They seem like really similar use cases - generating DWARF for the purpose of testing some (LLVM or Clang) API that consumes DWARF bytes.

Could we resolve this in favor of one approach or the other - I’m somewhat partial to the API approach & writing unit tests against the ClangDebuggerSupport library, myself.

  • David

David, the two approaches address very different problems.

The YAML tools are focused on a bit-for-bit identical round trip path for DWARF into and out of YAML. The goal with that work is to be able to generate a test suite from the output of many different versions of many different compilers. This is specifically with the goal of creating LIT-style tests that read DWARF and operate on it.

-Chris

David, the two approaches address very different problems.

The YAML tools are focused on a bit-for-bit identical round trip path for DWARF into and out of YAML. The goal with that work is to be able to generate a test suite from the output of many different versions of many different compilers. This is specifically with the goal of creating LIT-style tests that read DWARF and operate on it.

Ah, thanks for explaining.

These tests wouldn’t appear in LLVM/Clang’s test suite, then, right? So normal regression tests for the ClangDebuggerSupport library would be written as unit tests using Greg’s DWARF-generation library?

Large tests generated from other compilers on raw source I would expect to appear in something like the test-suite, rather than in an LLVM project’s regression or unit test suite.

Why the need for round tripping, then? Would it be sufficient for the test-suite to have binaries checked in next to info about what compiler generated them? (& why not just have the source checked in & run a variety of buildbot configurations (or one meta-configuration that could enumerate a variety of compilers) with different host compilers to test the behavior? That’s how GDB’s test suite works (for better and worse, don’t get me wrong - there are things that could be improved from that position))

  • Dave

David, the two approaches address very different problems.

The YAML tools are focused on a bit-for-bit identical round trip path for DWARF into and out of YAML. The goal with that work is to be able to generate a test suite from the output of many different versions of many different compilers. This is specifically with the goal of creating LIT-style tests that read DWARF and operate on it.

Ah, thanks for explaining.

These tests wouldn’t appear in LLVM/Clang’s test suite, then, right? So normal regression tests for the ClangDebuggerSupport library would be written as unit tests using Greg’s DWARF-generation library?

My goal is actually to have reduced test cases based on the YAML tools in the clang test suite. LLDB’s use of clang APIs with dwarf generated by mismatched compilers is the source of many issues for the debugger, so having basic testing of DWARF generated by alternate compilers in Clang is highly desirable.

Large tests generated from other compilers on raw source I would expect to appear in something like the test-suite, rather than in an LLVM project’s regression or unit test suite.

Large tests will certainly not be included in the clang test suite. YAML representations of DWARF should enable us to make reduced test cases in many situations, and where we cannot we will put the test in an external suite.

Why the need for round tripping, then? Would it be sufficient for the test-suite to have binaries checked in next to info about what compiler generated them?

The benefit of supporting round tripping in and out of a text-based format is that we may be able to reduce the test cases to things that we can include in the Clang test suite.

(& why not just have the source checked in & run a variety of buildbot configurations (or one meta-configuration that could enumerate a variety of compilers) with different host compilers to test the behavior? That’s how GDB’s test suite works (for better and worse, don’t get me wrong - there are things that could be improved from that position))

This is actually basically how the LLDB test suite works. There is one huge drawback to this. Not everyone has access to every compiler we want to support, and certainly most people don’t have them all installed. As a result having source-based tests means that many people may not be able to reproduce test failures locally. Using YAML encodings to generate the binary DWARF removes the compiler from the picture, and allows everyone to test every compiler’s output.

-Chris