[GSoC] "Microsoft Visual C++ ABI support in Clang" proposal

Here's one of my proposals for this year's Google Summer of Code. (The
other is on LLVMdev.)

Title: Microsoft Visual C++ ABI support in Clang

Abstract:

Both GCC and Clang default to using the Itanium C++ ABI. However, the de
facto standard C++ ABI on Windows is the one provided by Microsoft in
their Visual C++ product. If Clang could generate code against the
Microsoft Visual C++ ABI, it would be more compatible with C++ libraries
that were built with Visual C++. This should also ease development of
C++ DLLs for use with Wine (http://www.winehq.org).

Content:

1. Proposal

Currently, both GCC and Clang default to generating C++ code against the
so-called Itanium C++ ABI. This is fine for Unix, where there is no real
standard, but is a problem on Windows. There, the de facto standard is
the ABI provided by Microsoft in their Visual C++ product. Most people
on Windows use Visual C++--in part, because the compiler itself is
available for no charge from Microsoft--and, as a result, many other
compilers for Windows support to some extent the Microsoft Visual C++
ABI. This ABI is, in fact, integrated into Windows itself: several
system libraries are built against it.

It is therefore imperative that Clang on Windows support the Microsoft
Visual C++ ABI. By supporting this de facto standard, Clang could
improve its compatibility with many C++ libraries built on Windows.

In this project, I will implement enough of the Visual C++ ABI to get
simple programs working, such as the C++ "Hello, world" program. If time
permits, which I admit it likely will, I will implement additional
features. Features covered by the scope of this project include
everything outside of exception handling, including name mangling
(already partially implemented), class layout, vtable layout, calls to
virtual functions, and accesses of members in virtual bases. Proper EH
support requires SEH support and is thus beyond the scope of this project.

This would not just benefit Windows users. This support could
potentially be used in combination with Wine (http://www.winehq.org) to
ease the porting and/or development of DLLs written in C++ to Wine.

2. Interest

My interest is primarily in the second benefit (to Wine), since I do not
primarily run Windows. However, I realize that my work in this area
would benefit the many Windows users we now have, as well.

3. Usefulness to LLVM

By adding support for the Microsoft Visual C++ ABI, Clang could expand
its user-base, particularly on Windows, where this ABI is prevalent.
Clang would also become more compatible with a vast collection of
software libraries that were built against VC++.

4. Prior knowledge in compilers and LLVM in particular

Last year's project involved laying out the necessary support for this.
I have been working with LLVM since the fall of 2009. I am familiar with
Clang and its inner workings, and have added several new features to
Clang since I started working with it (such as forced stack alignment,
GNU-compatible extern inline, and of course, extensible C++ ABI support).

5. Academic, industry, etc. experience

I am a student at the Colorado School of Mines studying in Computer
Science. I am a Senior by year and credits, and I am set to graduate in
December of 2011. In addition, I have contributed to several open-source
projects, including Wine and LLVM.

6. Contact information

- E-mail: cdavis@mymail.mines.edu
- IRC nick: cdavis5x

Chip

Charles Davis <cdavis@mymail.mines.edu> writes:

Title: Microsoft Visual C++ ABI support in Clang

Abstract:

Both GCC and Clang default to using the Itanium C++ ABI. However, the de
facto standard C++ ABI on Windows is the one provided by Microsoft in
their Visual C++ product.

Long time ago I used a lot of Windows C++ compilers and the only one
that was compatible with VC++ was Intel. Are there any others that
switched to the MS C++ ABI? Just curious.

[snip]

In this project, I will implement enough of the Visual C++ ABI to get
simple programs working, such as the C++ "Hello, world" program. If time
permits, which I admit it likely will, I will implement additional
features. Features covered by the scope of this project include
everything outside of exception handling, including name mangling
(already partially implemented), class layout, vtable layout, calls to
virtual functions, and accesses of members in virtual bases. Proper EH
support requires SEH support and is thus beyond the scope of this project.

LLVM needs some non-trivial enhancements. See PR5058 and PR5064. Also
note the claims about MS C++ ABI being undocumented on the commentaries
to the last PR.

Did you investigate the legal status of the pieces of VC++ ABI?

[snip]

Charles Davis <cdavis@mymail.mines.edu> writes:

Title: Microsoft Visual C++ ABI support in Clang

Abstract:

Both GCC and Clang default to using the Itanium C++ ABI. However, the de
facto standard C++ ABI on Windows is the one provided by Microsoft in
their Visual C++ product.

Long time ago I used a lot of Windows C++ compilers and the only one
that was compatible with VC++ was Intel. Are there any others that
switched to the MS C++ ABI? Just curious.

The Windows version of Metrowerks CodeWarrior, when it still supported
x86 code-gen. (It was my very first compiler...)

Not for recent versions of Windows. C++ EH and SEH have always been slightly orthogonal (although VC++ has supported generating SEH cleanup information for C++ code, as well as pure C++ EH metadata).

With x86-64, the EH model on Windows is a lot simpler. It's actually fairly similar to the Itanium model (unwind library, personality function, code in functions for branching on different handles based on value installed by the personality function). I think it should be possible to support this exception model without any changes / extensions to LLVM IR, although it will obviously require changes in the back end.

If anyone's interested in working this, Microsoft has actually bothered to document the ABI pretty well this time:

I don't have a Windows machine, but if anyone is interested in making this work then let me know - I'd like for Objective-C exceptions to be able to use this mechanism on Win64...

David

This one should not be ignored. If any of these pieces are proprietary (in a legal sense) then that is a potential major impediment to putting any of this work into the open source tree.

I looked around and couldn't find anything. I even looked on the USPTO
website. Still nothing.

Looks right now like we are go for this.

Chip

Interesting WikiPedia article of Visual C++ name mangling :-

http://en.wikipedia.org/wiki/Microsoft_Visual_C%2B%2B_Name_Mangling

Aaron