Windows .exe from clang + COFF w/o gcc - success with issues

Hi all,

I have been experimenting with the creation of Win32 executables using only MSVC SDK headers, clang, llc and link.exe from MS Visual Studio 2008. This has uncovered a few issues:

1) VisualStudioWindowsX86_32TargetInfo needs to define _STDCALL_SUPPORTED. Otherwise the Win32 API does not get dllimport declarations and linking fails. Patch attached.

2) The SDK include directory search fails for me. InitHeaderSearch.cpp:getWindowsSDKDir() uses key "InstallationFolder" whose contents lacks the final "\include" needed for SDK 6.0a.

3) Compiling the simple test program below causes the resulting llvm assembly to include definitions of the functions Int64ShllMod32, Int64ShraMod32 and Int64ShrlMod32 although these are neither used or referenced in the code. This in turn causes problems in 4):

define x86_stdcallcc i64 @Int64ShllMod32(i64 %Value, i32 %ShiftCount) nounwind inlinehint {
entry:
   %retval = alloca i64, align 8 ; <i64*> [#uses=1]
   %Value.addr = alloca i64, align 8 ; <i64*> [#uses=1]
   %ShiftCount.addr = alloca i32, align 4 ; <i32*> [#uses=1]
   store i64 %Value, i64* %Value.addr
   store i32 %ShiftCount, i32* %ShiftCount.addr
   call void asm sideeffect "FIXME: not done", "~{dirflag},~{fpsr},~{flags}"() nounwind, !srcloc !0
   %0 = load i64* %retval ; <i64> [#uses=1]
   ret i64 %0
}

4) Parser::FuzzyParseMicrosoftAsmStatement() emits "FIXME: not done" in the llvm code above when microsoft assembly is encountered. Changing it to emit "/*"FIXME: not done*/" makes the resulting file compile. Patch attached.

DefineSTDCALLSUPPORTED.patch (424 Bytes)

StopEmittingFIXME.patch (539 Bytes)

Ping? Does anyone feel like taking a look at these patches? Should I submit any bugs for the issues below?

Hey,

I’m new on this list, but earlier I caught a thread about similar issues. Check out the thread “CLang compiling windows.h from the Windows SDK” from Francois Pichet.

If I understand this correctly, there are a lot of things about the windows headers that are not going to be easy fixes. However, I’m interested in doing the same thing; getting CLang to compile a windows executable. Personally, I prefer Doug’s recommended way of dealing with other companies or projects that are not compliant, by telling them to correct their headers, but who knows how long that will take until MS finally get around to fixing them.

I’ll check out this patch later tonight and see what results I get and post my results back here. Can someone with commit rights check out the patch as well?

Thanks,
-Ken Noland

2010/8/16 per@lumai.se <per@lumai.se>

Also, it appears to help if you put the word [PATCH] in the title

Cheers,
-Ken Noland

Actually, clang is very close to be able to compile Windows SDK headers.
One of the feature needed is support for the __uuidof operator. I am
preparing a patch for this issue.
The only unclean thing we'll have to do is to inject a foward struct
declaration for IUnknow type.

Yes, I am well aware that compiling SDK headers is possible given some tricks. I have been doing that for a while now. The issues below are about _linking_ the compiled files and actually creating a working executable without resorting to Mingw32/gcc. As I stated; when I fixed these issues locally this worked. Has anyone of you accomplished this without hitting the issues I mentioned? In that case they can be ignored, of course...

Yes, I am well aware that compiling SDK headers is possible given some tricks. I have been doing that for a while now. The issues below are about _linking_ the compiled files and actually creating a working executable without resorting to Mingw32/gcc. As I stated; when I fixed these issues locally this worked. Has anyone of you accomplished this without hitting the issues I mentioned? In that case they can be ignored, of course...

Sorry if I have posted this message twice, but I'm having some problems with newsgroup posting right now.

Thanks Per, I applied your two patches. It would be nice if someone would compare the list of macros predefined by MSVC with the list of macros predefined by clang (when targeting windows). Instead of trusting the MS docs, it would be good to find a way to get MSVC to dump its macro table.

-Chris

The problem is how do we deal with predefined macros that depend on
cl.exe specific compiler switches? Almost all MSVC predefined macros
fit this category.

What I am planning to do is to write a cl2clang driver that would act
as a cl.exe (MSVC) drop-in replacement. That driver would accept the
same set of options as cl.exe
(http://msdn.microsoft.com/en-us/library/fwkeyyhe(VS.71).aspx).
Translate those options (including setting predefined macros with -D)
to be clang compatible and spawn a new clang process.

Since Visual studio 2010 is now using msbuild to build C++ projects,
I think it would be possible to do something like:
msbuild /p:VCBuildToolPath=cl2lang.exe mysolution.sln

And use the whole MSVC build system and just have clang do the compilation.

Anybody think it would be useful to put that new cl2lang driver into
the mainline repository?
It is just an idea for now.. I haven't started working on it.

Why not just add the cl.exe switches to clang? Try parsing anything
beginning with / as an option, and if that fails treat it like a path
to a file.

- Michael Spencer

Anybody think it would be useful to put that new cl2lang driver into
the mainline repository?
It is just an idea for now.. I haven't started working on it.

I definitely think that having a cl.exe-compatible driver, similar to clang being a gcc-compatible driver, would be useful.

Why not just add the cl.exe switches to clang? Try parsing anything
beginning with / as an option, and if that fails treat it like a path
to a file.

Because that could create option conflicts - cl.exe accepts unix-style options as well, i.e. you can pass options as -option to cl.exe. Then what do you do if you have a gcc and a cl option that have the same spelling, but mean something subtly different?

Sebastian

Anybody think it would be useful to put that new cl2lang driver into
the mainline repository?
It is just an idea for now.. I haven't started working on it.

I definitely think that having a cl.exe-compatible driver, similar to clang being a gcc-compatible driver, would be useful.

Yes, for sure, I would love to have a cl.exe compatible driver. And
this is definitely something we would want in the tree.

I haven't thought much about it, but it would be nice if it could be
integrated into the Driver library in such a way that we could accept
both syntaxes, and perhaps would just change what the default expected
syntax was based on the host / default target.

- Daniel

Thanks Per, I applied your two patches. It would be nice if someone would compare the list of macros predefined by MSVC with the list of macros predefined by clang (when targeting windows). Instead of trusting the MS docs, it would be good to find a way to get MSVC to dump its macro table.

-Chris

Using IDA I got the compiler to print out every macro it defines, I
haven't gotten it to spit out the expansion, but that's easy to get
without IDA.

__FILE__
__DATE__
__TIME__
__TIMESTAMP__
__VA_ARGS__
__FUNCTION__
__FUNCDNAME__
__FUNCSIG__
__FSTREXP
__LPREFIX
__APREFIX
__COUNTER__
_MSC_EXTENSIONS
_INTEGRAL_MAX_BITS
_MSC_VER
_MSC_FULL_VER
_MSC_BUILD
_WIN32
_M_IX86
_M_IX86_FP
_MT

Interestingly, _STDCALL_SUPPORTED is not in this list. I expected this
due to the string not being in any of the binaries and it not
expanding to anything using the compiler normally.

MSVC has two separate binaries for c and c++, so I'll have to
duplicate this in the c++ case, but it should be easy now that I know
the global variable they use to pass identifiers around. (For anyone
that wants to take a look. They are in c1.dll and the function is
<.text:4F057EC0 ?InstallDefine@@YIXPAEHH_N@Z proc near> and the
variable is <.data:4F0B5800 ?pCurId@@3PAUId_t@@A dd>).

?pCurId@@3PAUId_t@@A is a pointer to a struct where the second dword
is a pointer to the identifier string. You can use
GetString(Dword(Dword(LocByName("?pCurId@@3PAUId_t@@A"))+4), -1,
ASCSTR_C) while debugging a program in IDA to get the string.

I'll collect all the defines for c and c++ with the various compiler
options and put them all up someplace.

- Michael Spencer

Does that version support X86_64? I would've expected to see _M_AMD64...

Does that version support X86_64? I would've expected to see _M_AMD64...

I have not gotten around to disassembling the 64 bit compiler yet
(yes, it's a separate compiler), but _M_X86 and _M_AMD64 are defined
when compiling with the x64 compiler.

I've attached a patch for review that fixes most of the predefined
macros on windows. I can compile hello world and parts of Lua.

- Michael Spencer

commit-fix-predefines (7.13 KB)

Michael Spencer wrote:

Interestingly, _STDCALL_SUPPORTED is not in this list. I expected
this due to the string not being in any of the binaries and it not
expanding to anything using the compiler normally.

Considering that it appears in the windows headers in tests like

#if (_MSC_VER >= 800) || defined(_STDCALL_SUPPORTED)

we could guess that it was important in MS C versions before 8.0 (aka Visual C++ 1.0, 1993 :-).

http://support.microsoft.com/kb/65472

Nowadays it is always supported, as "standard call" is the ... eh ... standard calling convention for the win32 API.

Bo Persson

can you remove the 2 non predefined macros "WINNT" and
"_CRT_SECURE_CPP_OVERLOAD_SECURE_NAMES" currently defined by clang?

Nice work!

Yes, _STDCALL_SUPPORTED is a temporary definition to get MSVC-header-base code to link until clang defines _MSC_VER _and_ handles the (nasty) consequences of that...

Right now there are a few parallell discussions about MSVC+clang. Perhaps some information could be centralized in a wiki page or such? A few topics as I see them:

* Compilation problems in the various SDK headers
* cl.exe command line wrapper/drop-in replacement ("llvm-cl"?)
* Supporting different MSVC versions using clang options
* Correct compiler predefines (without breaking compilation completely for the next 6 months...)