GSoC Proposal: Virtual Tables and RTTI in the Microsoft Visual C++ ABI

Hi,

Here is another of my GSoC proposals (the first was on LLVMdev). This seems somewhat small for a GSoC project. Is there something I'm missing?

Chip

Project Title: Virtual Tables and RTTI in the Microsoft Visual C++ ABI

Abstract:
Two years ago, I performed work to support multiple C++ ABIs in Clang, and started work to support the Microsoft Visual C++ ABI. This project is a continuation of that work, adding support for virtual tables in the Microsoft ABI. This is necessary before certain other work (RTTI, dynamic_cast, exceptions) can be performed.

Proposal:
Two years ago, in a Summer of Code project, I added support for multiple C++ ABIs to Clang, and began work on adding a second ABI to exercise this functionality: the Microsoft Visual C++ ABI. Unfortunately, implementing the whole ABI is far too much for one project. Therefore, this project proposes implementing a very important piece: emission of virtual tables.

Some pieces of this are already in place. A contributor, Dmitriy "r4start" Sokolov, kindly added some code to lay out the virtual table pointers in each class. Unfortunately, that work is unfinished, because he has been busy with his own life. This project will pick up where he left off.

In the Microsoft C++ ABI, there are actually two virtual tables: a function table and a base table. The function table is used to call virtual methods; the base table is used to access virtual bases. This project will implement both kinds of v-table.

One of the difficulties in this is ensuring the right table is used for a lookup. Currently, Clang still assumes it is compiling for the Itanium C++ ABI when it compiles C++ code with virtual methods/bases--which has a very different idea of how to implement support for v-bases and virtual methods. The various operations that are performed--v-table method pointer lookup, virtual base lookup--must therefore be abstracted out into the C++ ABI class I added in the first project.

Another is that, for ABI compatibility, the v-table entries must be emitted in the correct order. This will require research into how the Microsoft compiler lays out v-tables. It may also require another method in the C++ ABI object to lay out the virtual tables.

A third might be the use and emission of hidden "vtordisp" fields, which are used when a virtual method in a virtual base is called from a 'structor. Dmitriy has performed work in this area; his patch may help here.

Like the Itanium ABI, the Microsoft ABI sometimes requires the use of virtual thunks which adjust the 'this' pointer before calling the method proper. This proposal will also implement emission of such thunks.

If there is time, I may also begin work on emitting RTTI structures. These are officially undocumented, but luckily the Wine developers have worked out their definitions for their implementation of the Visual C++ runtime. We can use this to implement emitting the RTTI structures. Dmitriy also says he has patches to implement this himself, but he has not sent them to the list yet.

Proposed Timeline:
Week 1-2: Implement laying out Microsoft ABI v-tables and mangling of symbols.
Week 3: Implement virtual method calls.
Week 4: Implement virtual thunks.
Week 5-6: Implement virtual base loads and conversions.
Week 7-8: Finish vtordisp support.
Week 9-10: Implement emission of RTTI descriptors.
Week 11-12: Implement typeid() and dynamic_cast<> support.

Contact:
email: cdavis (at) mymail (dot) mines (dot) edu
phone: (seven-one-nine) 963-4781
IRC: cdavis5x@freenode, cdavis5x@oftc

Interest:
I have long been interested in figuring out how the Microsoft ABI works, and also in compiling libraries that need to use the Microsoft ABI for Wine.

Use to LLVM:
Supporting the Microsoft ABI will increase adoption of LLVM and Clang, particularly on Windows, where a certain compiler is most commonly used. In addition, supporting the Microsoft ABI will also make Clang more useful to projects like Wine that need to support the ABI, and might even spur GCC into supporting this.

Experience:
I have worked with LLVM for over two and a half years. I implemented mangler support for the Microsoft C++ ABI back in GSoC 2010. I have also implemented force_align_arg_pointer support, fixed a number of small bugs in Clang with attributes and calling conventions, and began work to support the Win64 exception-handling model. I have also contributed patches to Wine, mostly in the area of support for Mac OS.

I think Timur was investigating Microsoft ABI recently.

I think Timur was investigating Microsoft ABI recently.

Thanks Alexander!

Hi,

Here is another of my GSoC proposals (the first was on LLVMdev). This seems
somewhat small for a GSoC project. Is there something I'm missing?

Chip

Project Title: Virtual Tables and RTTI in the Microsoft Visual C++ ABI

Abstract:
Two years ago, I performed work to support multiple C++ ABIs in Clang, and
started work to support the Microsoft Visual C++ ABI. This project is a
continuation of that work, adding support for virtual tables in the
Microsoft ABI. This is necessary before certain other work (RTTI,
dynamic_cast, exceptions) can be performed.

Proposal:
Two years ago, in a Summer of Code project, I added support for multiple C++
ABIs to Clang, and began work on adding a second ABI to exercise this
functionality: the Microsoft Visual C++ ABI. Unfortunately, implementing the
whole ABI is far too much for one project. Therefore, this project proposes
implementing a very important piece: emission of virtual tables.

Some pieces of this are already in place. A contributor, Dmitriy "r4start"
Sokolov, kindly added some code to lay out the virtual table pointers in
each class. Unfortunately, that work is unfinished, because he has been busy
with his own life. This project will pick up where he left off.

In the Microsoft C++ ABI, there are actually two virtual tables: a function
table and a base table. The function table is used to call virtual methods;
the base table is used to access virtual bases. This project will implement
both kinds of v-table.

One of the difficulties in this is ensuring the right table is used for a
lookup. Currently, Clang still assumes it is compiling for the Itanium C++
ABI when it compiles C++ code with virtual methods/bases--which has a very
different idea of how to implement support for v-bases and virtual methods.
The various operations that are performed--v-table method pointer lookup,
virtual base lookup--must therefore be abstracted out into the C++ ABI class
I added in the first project.

Another is that, for ABI compatibility, the v-table entries must be emitted
in the correct order. This will require research into how the Microsoft
compiler lays out v-tables. It may also require another method in the C++
ABI object to lay out the virtual tables.

A third might be the use and emission of hidden "vtordisp" fields, which are
used when a virtual method in a virtual base is called from a 'structor.
Dmitriy has performed work in this area; his patch may help here.

Like the Itanium ABI, the Microsoft ABI sometimes requires the use of
virtual thunks which adjust the 'this' pointer before calling the method
proper. This proposal will also implement emission of such thunks.

If there is time, I may also begin work on emitting RTTI structures. These
are officially undocumented, but luckily the Wine developers have worked out
their definitions for their implementation of the Visual C++ runtime. We can
use this to implement emitting the RTTI structures. Dmitriy also says he has
patches to implement this himself, but he has not sent them to the list yet.

Proposed Timeline:
Week 1-2: Implement laying out Microsoft ABI v-tables and mangling of
symbols.
Week 3: Implement virtual method calls.
Week 4: Implement virtual thunks.
Week 5-6: Implement virtual base loads and conversions.
Week 7-8: Finish vtordisp support.
Week 9-10: Implement emission of RTTI descriptors.
Week 11-12: Implement typeid() and dynamic_cast<> support.

Contact:
email: cdavis (at) mymail (dot) mines (dot) edu
phone: (seven-one-nine) 963-4781
IRC: cdavis5x@freenode, cdavis5x@oftc

Interest:
I have long been interested in figuring out how the Microsoft ABI works, and
also in compiling libraries that need to use the Microsoft ABI for Wine.

Use to LLVM:
Supporting the Microsoft ABI will increase adoption of LLVM and Clang,
particularly on Windows, where a certain compiler is most commonly used. In
addition, supporting the Microsoft ABI will also make Clang more useful to
projects like Wine that need to support the ABI, and might even spur GCC
into supporting this.

Hi Charles,

Do you have any specific Win projects or test suites in mind?

I think Chromium would be very much interested in having Clang build
for Windows [ see http://crbug.com/82385 ]
I suppose starting with Chromium is too much though
so you can start with something smaller, e.g.
googletest samples (Chromium code is based on googletest),
ThreadSanitizer tests, AddressSanitizer tests (these use googletest as well).

See
http://code.google.com/p/googletest/
http://code.google.com/p/data-race-test/wiki/RacecheckUnittest
http://code.google.com/p/address-sanitizer/wiki/HowToBuild
Please note that ASan tests are not fully ported to Windows, that's
work in progress.

I have filed a meta bug and a few specific bugs based on the problems
I've encountered while trying to build the above stuff,
see http://llvm.org/bugs/show_bug.cgi?id=12477

P.s.
I'm interested in the Microsoft C++ ABI support because I work on
AddressSanitizer-for-Chromium-on-Windows.
You can take a look here
http://code.google.com/p/address-sanitizer/wiki/WindowsPort

Thanks,
Timur Iskhodzhanov,
Google Russia

Hi all!

I have some code that:
  - build MS RTTI descriptors;
  - generates vb-tables;
  - mangle vf-table, vb-table and thunks;
  - initialize virtual tables in ctor;
  - generates vf-table.

I don`t publish this code because work doesn`t complete yet and code isn`t good.
At now I don`t have enough time to work on it, but I think I will return to implementing MS ABI at June(I hope).
If you need some help with implementing MS ABI I can send you this code.

  - Dmitry.

I think it'd be valuable to have this code at least for the reference.

Ok.

I take pause for 2 days and then publish it.
I could broke it and I need some time for relocation it for newer revision(my revision is dated 17.02).

  - Dmitry.

All right then.

@list: Since the deadline is looming, I've posted my application on Melange. You can see it here:

http://google-melange.appspot.com/gsoc/proposal/review/google/gsoc2012/cdavis/30002

Chip

Forgot to mention: RTTI is not that important for Chromium since it's
forbidden by the coding style guide.

That said, I'd rather have something like "fix other minor Microsoft
C++ ABI issues" instead of RTTI support if the latter is not trivial.

Thanks!

Hi!

I attached patch with all I have.
Unfortunately I don't have enough time to cleanup it.
Also, 2 test is broken: ms-rtti2 & ms-rtti4. It is normal.
If you have any questions about code you can write me directly or in this mail list.

  - Dmitry.

clang.patch (209 KB)