Windows strategy?

Hi,

[Synopsis: Mikael thinks Windows support is overlooked and that this is a strategic error. You can skip to the next mail now, if you’d like :slight_smile: Long, long, long mail follows.]

Before I go on, I’d like to informally apologize for my many postings to this mailing list. I have no intentions of spamming it, but I still feel a need to grasp certain things that seem to be implied by the documentation or perhaps not described at all. Also, as a self-appointed LLVM/Clang Windows Evangelist, I’d like to keep Windows in your mental horizon so that we one day have full support for at the very least both 32-bit and 64-bit variants of Clang using the respective version of MinGW.

Very briefly put, I came from a background of Windows tools: OpenWatcom C++ (used to be the best optimizer on Windows), Microsoft Visual C++, and back in the ancient days of early DOS and NT, JPI, Borland, and Metaware C++. I had next to no experience with the MinGW toolchain and had only very little experience with actual use of GCC. So everything was pretty overwhelming at first. I started using C++ in 1990 so I am not exactly a newbie. I have been off the C++ path for a decade, though, because better languages have popped up (Python, then C#). I knew nothing about CMake and was, like most Windows users are going to be, a stranger to the whole ordeal of Unix-on-Windows (incl. Cygwin and MinGW).

I have so far tried pretty much every path in the maze that LLVM’s support of Windows is: I*ve tried building with MSVC, I’ve tried building with Mingw64, I’ve tried building with a 64-bit edition of Clang, and now I am down to the last desirable choice: 32-bit MinGW. My 32-bit Clang++ is building as we speak. I tried with a 64-bit build of Clang, built using MinGW64, but it fails on some linker errors (the report about that erronously sent to this mailing list earlier today), so I hope, pray, and expect that at the very least, the 32-bit Clang++ built from MinGW32 will work as expected (that is, it is possible to compile and link and execute a non-trivial program using Clang32).

But the past two weeks have been spent on setting up a Windows 7 x64 MinGW64 buildbot slave and then working a bit on my own project (I’m translating 25 KLOC of C# to C++ so as to be able to use LLVM as my backend) and then playing around with Clang.

My view is that the aspiring Clang++ Windows user is in for a nightmare reverse walk through a night time minefield with both eyes shut.

So, I am wondering if there is anything that can be done to help this situation? I’d love great ideas from you guys, even though I am getting rather used to getting no responses to my mails :wink: I am willing to invest considerable amount of time into helping with this, I just cannot do serious C++ coding at this point because I am yet too rusty in C++ and because I am busy, busy, busy with my own project (a compiler that uses LLVM as its backend).

I propose these things:

  1. Priority is put into making MinGW64 work, if the problem I reported earlier today is related to LLVM/Clang (link time errors related to some section). This is not that important, though.
  2. If nobody (including licenses) object to it, I’d like to make two unofficial Windows distributions of Clang: clang-3.2-win32 and clang-3.2-win64. Both would make use of the MinGW toolchains and both would be shipped as ready-made 7-zip archives that could be extracted anywhere and used directly from there without further installations. I am thinking of including the MinGW header files (once the warnings reported when they are included are gone) so as to produce a complete retail product that can be used without any technical knowledge of the LLVM and Clang components. I think this would bring lots and lots of attention to LLVM and Clang, and then hopefully pay off in skilled developers joining the cause. I am also thinking of setting up an automated build system so that they can be built once a day and automatically be published on my own website.
  3. I will eagerly hunt down every opportunity to document Windows specifics a bit more. I have received no feedback, whatsoever, on my Windows build document so I take it that it is not to the dev group’s liking. So be it. Life goes on. But how about a Windows section in the FAQ? The word Windows isn’t even mentioned?! Most of the stuff I’ve ran into ought to be easy to describe over the course of a couple of hours and then newcomers would save days and weeks of frustrations, dead ends, and mines exploding around them.
  4. Please give me your ideas for how I can contribute to the Windows support without actually hacking on LLVM and Clang (yet)!

I know that you guys are awesomely skilled and are really great at what you are doing. No doubt about that. But I think you lack marketing/strategic thinking. Think of it a bit like I do:

  1. 97.7 percent of the world’s computer users are Windows users.
  2. 2 percent of the world’s computer users are Linux users.
  3. 0.3 percent of the world’s computer users use other Unices.

The numbers are grasped out of thin air so as to illustrate my point: Windows ought to be a very high priority because it will eventually pay off handsomely by increasing the size of the developer group by a factor of ten, if only you play the cards right and offer neat, easy-to-use, convenient Windows support. Besides, it is my impression that the non-Windows support is in general pretty mature. Clang could very well step in as THE alternative to Microsoft Visual C++. OpenWatcom is slowly dying (very poor ISO C++ support) and Intel’s C++ compiler is only used by specialists, as far as I can tell.

I suspect that some of you might hold the view that Windows users are noobs (something I’ve worked hard to confirm :wink: and that the really great guys are all doing Unix. Not so. Statistically speaking, the vast majority of coders out there are on Windows, so statistically speaking, the vast majority of the truly brilliant dev people are on Windows. So it makes a lot of sense trying to attract them! Not that I am implying that I myself are anywhere near brilliant. I’m just saying that overlooking Windows might be the greatest strategical debt you have incurred yet.

P.S. Please think of the above as a wake-up call and my dedication to the great future that LLVM/Clang should have.

Cheers,
Mikael
– Love Thy Frog (yes, frogs are worth loving, even though most people don’t seem to grok that).

Hi,

Again, this is more of a topic for cfe-dev (copied). LLVM itself works very well on Windows; it’s Clang that has issues.

[Synopsis: Mikael thinks Windows support is overlooked and that this is a strategic error. You can skip to the next mail now, if you’d like :slight_smile: Long, long, long mail follows.]

Generally, I too would like to see much greater Windows support in Clang. However, most developers here are centered around Mac/Linux. It will take contributions from Windows people to improve this support (and also watch out for legal quagmires). It’s not so much that LLVM/Clang developers actively choose not to support Windows, it’s more that there are just not enough resources to improve the support.

Before I go on, I’d like to informally apologize for my many postings to this mailing list. I have no intentions of spamming it, but I still feel a need to grasp certain things that seem to be implied by the documentation or perhaps not described at all. Also, as a self-appointed LLVM/Clang Windows Evangelist, I’d like to keep Windows in your mental horizon so that we one day have full support for at the very least both 32-bit and 64-bit variants of Clang using the respective version of MinGW.

Very briefly put, I came from a background of Windows tools: OpenWatcom C++ (used to be the best optimizer on Windows), Microsoft Visual C++, and back in the ancient days of early DOS and NT, JPI, Borland, and Metaware C++. I had next to no experience with the MinGW toolchain and had only very little experience with actual use of GCC. So everything was pretty overwhelming at first. I started using C++ in 1990 so I am not exactly a newbie. I have been off the C++ path for a decade, though, because better languages have popped up (Python, then C#). I knew nothing about CMake and was, like most Windows users are going to be, a stranger to the whole ordeal of Unix-on-Windows (incl. Cygwin and MinGW).

I don’t know if I’d call MinGW “unix-on-windows.” It’s just a port of GCC with Windows headers/libraries. You don’t need to use bash or any other unix programs.

I have so far tried pretty much every path in the maze that LLVM’s support of Windows is: I*ve tried building with MSVC, I’ve tried building with Mingw64, I’ve tried building with a 64-bit edition of Clang, and now I am down to the last desirable choice: 32-bit MinGW. My 32-bit Clang++ is building as we speak. I tried with a 64-bit build of Clang, built using MinGW64, but it fails on some linker errors (the report about that erronously sent to this mailing list earlier today), so I hope, pray, and expect that at the very least, the 32-bit Clang++ built from MinGW32 will work as expected (that is, it is possible to compile and link and execute a non-trivial program using Clang32).

MSVC ABI support in Clang is very incomplete as this point. MinGW32 support works fairly well, at least in my experience. MinGW64 is a bit of a problem at the moment because of GCC versions. Clang is hard-coded to search for headers in GCC version-specific locations. For the dgn version of MinGW64, GCC 4.7.1 is used (in the latest release), but Clang only looks up for up to 4.7.0 at the moment. That’s why you’re running into issues. If I manually setup paths, I can build C++ programs using Clang using the MinGW64 headers/libraries:

$ clang++ foo.cpp -o foo.exe -I “c:\mingw64\include\c++\4.7.1” -I “c:\mingw64\include\c++\4.7.1\x86_64-w64-mingw32” -I c:\mingw64\mingw\include

But the past two weeks have been spent on setting up a Windows 7 x64 MinGW64 buildbot slave and then working a bit on my own project (I’m translating 25 KLOC of C# to C++ so as to be able to use LLVM as my backend) and then playing around with Clang.

My view is that the aspiring Clang++ Windows user is in for a nightmare reverse walk through a night time minefield with both eyes shut.

Unfortunately, yes.

So, I am wondering if there is anything that can be done to help this situation? I’d love great ideas from you guys, even though I am getting rather used to getting no responses to my mails :wink: I am willing to invest considerable amount of time into helping with this, I just cannot do serious C++ coding at this point because I am yet too rusty in C++ and because I am busy, busy, busy with my own project (a compiler that uses LLVM as its backend).

I propose these things:

  1. Priority is put into making MinGW64 work, if the problem I reported earlier today is related to LLVM/Clang (link time errors related to some section). This is not that important, though.
  2. If nobody (including licenses) object to it, I’d like to make two unofficial Windows distributions of Clang: clang-3.2-win32 and clang-3.2-win64. Both would make use of the MinGW toolchains and both would be shipped as ready-made 7-zip archives that could be extracted anywhere and used directly from there without further installations. I am thinking of including the MinGW header files (once the warnings reported when they are included are gone) so as to produce a complete retail product that can be used without any technical knowledge of the LLVM and Clang components. I think this would bring lots and lots of attention to LLVM and Clang, and then hopefully pay off in skilled developers joining the cause. I am also thinking of setting up an automated build system so that they can be built once a day and automatically be published on my own website.

As long as you do not violate any licenses, this should be fine. I would love to see such a distribution!

  1. I will eagerly hunt down every opportunity to document Windows specifics a bit more. I have received no feedback, whatsoever, on my Windows build document so I take it that it is not to the dev group’s liking. So be it. Life goes on. But how about a Windows section in the FAQ? The word Windows isn’t even mentioned?! Most of the stuff I’ve ran into ought to be easy to describe over the course of a couple of hours and then newcomers would save days and weeks of frustrations, dead ends, and mines exploding around them.

To be honest, I just haven’t looked at it in detail yet. I plan to, but most developers here are very busy and it takes time to get reviews. Please bump the topic after a couple days of no replies.

  1. Please give me your ideas for how I can contribute to the Windows support without actually hacking on LLVM and Clang (yet)!

File bugs for what doesn’t work.

  1. Full support for libc++ on Win32/Win64.

I know that you guys are awesomely skilled and are really great at what you are doing. No doubt about that. But I think you lack marketing/strategic thinking. Think of it a bit like I do:

  1. 97.7 percent of the world’s computer users are Windows users.
  2. 2 percent of the world’s computer users are Linux users.
  3. 0.3 percent of the world’s computer users use other Unices.

And the vast majority of those Windows users will use MSVC instead of Clang. Most will get by just fine with the Microsoft compiler, and they have no reason to really choose Clang, unless they want it for better ISO C++ compliance, or cross-compiling.

The numbers are grasped out of thin air so as to illustrate my point: Windows ought to be a very high priority because it will eventually pay off handsomely by increasing the size of the developer group by a factor of ten, if only you play the cards right and offer neat, easy-to-use, convenient Windows support. Besides, it is my impression that the non-Windows support is in general pretty mature. Clang could very well step in as THE alternative to Microsoft Visual C++. OpenWatcom is slowly dying (very poor ISO C++ support) and Intel’s C++ compiler is only used by specialists, as far as I can tell.

Clang could definitely fill a niche market here, but it needs the support of talented Windows developers.

I suspect that some of you might hold the view that Windows users are noobs (something I’ve worked hard to confirm :wink: and that the really great guys are all doing Unix. Not so. Statistically speaking, the vast majority of coders out there are on Windows, so statistically speaking, the vast majority of the truly brilliant dev people are on Windows. So it makes a lot of sense trying to attract them! Not that I am implying that I myself are anywhere near brilliant. I’m just saying that overlooking Windows might be the greatest strategical debt you have incurred yet.

I doubt that’s the general feeling…

Once again, I forgot about the cfe-dev list. I’m moving the thread over there for those interested.

Cheers,
Mikael

Hi,

[Synopsis: Mikael thinks Windows support is overlooked and that this is a strategic error. …]

Synopsis of my response: It is not overlooked, it has some technical, non-technical, and manpower limitations.

Allow me to briefly summarize these issues:

Technical: Currently, we have a shortage of build infrastructure and test infrastructure of high quality running windows. This limits the amount of testing that target gets, and limits the ability of non-windows-hosted developers to recognize when their patch causes a problem.

Non-technical: Currently, there is an unofficial policy that is in place limiting what parts of Windows support we’re pursuing. Feel free to reach out to me directly for details, but here is a summary of the strategy that I discussed with Doug Gregor and several others:

  • MingGW and Cygwin in all of their various forms are fine. Patches extremely welcome here, these should be first-class supported platforms.

  • Documentation is always welcome, and much needed. Please contribute more here. =]

  • Win64 ABI, mangling, etc. compatibility is fine. Patches are quite welcome here, and please cite the win64 spec[1] for what your patch fixes.

  • We are not planning to do more work on Win32 ABI. Some bug fixing (name mangling, etc) is fine, but I would encourage development efforts to focus on Win64, and get that working instead. We may decide to support some subset of Win32, but that’s in the future.

  • Supporting well specified Windows language extensions is welcome. See the declspec work for great examples of this.

  • We are working to figure out the best way to support parsing the Windows headers, but this is a complex issue. Expect some patches to clear the way in the not-too-distant future.

[1] (http://msdn.microsoft.com/en-us/library/7kcdt6fy(v=vs.110)

My view is that the aspiring Clang++ Windows user is in for a nightmare reverse walk through a night time minefield with both eyes shut.

Yes. See documentation point above.

So, I am wondering if there is anything that can be done to help this situation? I’d love great ideas from you guys, even though I am getting rather used to getting no responses to my mails :wink: I am willing to invest considerable amount of time into helping with this, I just cannot do serious C++ coding at this point because I am yet too rusty in C++ and because I am busy, busy, busy with my own project (a compiler that uses LLVM as its backend).

Patches are likely to be necessary here, but patches to things like documentation can be done even w/o much C++ expertise.

  1. Priority is put into making MinGW64 work

Please understand, each developer has their own priorities. Yours are Windows support, and that is excellent, but other developers cannot simply shift their priorities around. We need new contributors (like yourself) to prioritize Windows because of their interest and investment in the platform.

  1. If nobody (including licenses) object to it, I’d like to make two unofficial Windows distributions of Clang: clang-3.2-win32 and clang-3.2-win64.

These would, I suspect, not be “official” releases. Please join in the release testing and qualification process for the next Clang release though! We need people to help test and qualify the release (and build binariy packages) for platforms like Windows.

There is documentation on the website about how to join this effort for the 3.2 release cycle.

  1. I will eagerly hunt down every opportunity to document Windows specifics a bit more. I have received no feedback, whatsoever, on my Windows build document so I take it that it is not to the dev group’s liking. So be it. Life goes on.

Wait, what document? I missed it. =[ Can you ping the thread? We are all dealing with a very large email load, but I assure you there is interest here. You may have to help us keep track of the email thread and get you feedback, but it’s not a lack of interest.

  1. Please give me your ideas for how I can contribute to the Windows support without actually hacking on LLVM and Clang (yet)!

I hope I’ve done so…

One point I have to make however is that this:

  1. 97.7 percent of the world’s computer users are Windows users.

is not an effective argument. ;] All of the developers on Clang/LLVM are working because they have some specific itch to scratch. Some of the developers are employed by companies that have a corporate itch to scratch with Clang. These often represent significant user bases, and are investing time to directly support them. It’s not about Windows vs. Linux vs. other. It’s about users that I am directly responsible for supporting, and their needs trump.

Just be aware of that. ;] Also, there are a lot of Mac users out there. Just saying.

Hi Justin,

Thanks for your positive reply. I imagined I’d be banned from the list or something :wink:

I don’t know if I’d call MinGW “unix-on-windows.” It’s just a port of GCC with Windows headers/libraries. You don’t need to use bash or any other unix programs.

I’m thinking of the beast “ld” here. As a former linker developer (Link&Locate 386/XLINK386, an absolute/embedded linker from the 80ties and 90ties) and a long-time DOS/Windows user, I simply cannot get used to the archaic notion that the order of the input libraries should matter the least (as long as there are no duplicate symbols). The DOS/Windows platform gave up that notion some 20-25 years ago. Yet, on most Unices, people live with it every day and are accustomed to having to shuffle libraries around every now and then to get the linker to link once again (I’ve already done it two or three times with my tiny project - suddenly out of the blue, it refuses to link and you have to shuffle a bit to get it working again).

MSVC ABI support in Clang is very incomplete as this point. MinGW32 support works fairly well, at least in my experience. MinGW64 is a bit of a problem at the moment because of GCC versions. Clang is hard-coded to search for headers in GCC version-specific locations. For the dgn version of MinGW64, GCC 4.7.1 is used (in the latest release), but Clang only looks up for up to 4.7.0 at the moment. That’s why you’re running into issues. If I manually setup paths, I can build C++ programs using Clang using the MinGW64 headers/libraries:

$ clang++ foo.cpp -o foo.exe -I “c:\mingw64\include\c++\4.7.1” -I “c:\mingw64\include\c++\4.7.1\x86_64-w64-mingw32” -I c:\mingw64\mingw\include

Oh, that’s wonderful! I am actually thinking of doing something about the problem of the static library locations, somehow, once I get some time to look at it. Because there MUST be a better way than maintaining a static list of library search directories. Perhaps two external files containing lists of directories to search (one for headers and one for libraries)?

Ohh, now I remember: I actually have the code below in my master CMakeLists.txt file and yet the link fails:

if("${CMAKE_CXX_COMPILER_ID}" MATCHES “Clang”)
include_directories("/Mingw64/include")
include_directories("/Mingw64/include/c++/4.7.1")
include_directories("/Mingw64/include/c++/4.7.1/x86_64-w64-mingw32")
include_directories("/Mingw64/x86_64-w64-mingw32/include")
elseif("${CMAKE_CXX_COMPILER_ID}" MATCHES “GNU”)
endif()

So, I guess I’ve run into some kind of problem after all. I’ll try to file a bug report on it.

As long as you do not violate any licenses, this should be fine. I would love to see such a distribution!

Great, that’s a deal then! I’ll check the GNU/MinGW licenses and see if it isn’t okay to do it. Talk to the GNU guys, if need be.

To be honest, I just haven’t looked at it in detail yet. I plan to, but most developers here are very busy and it takes time to get reviews. Please bump the topic after a couple days of no replies.

Okay, I guess I also have to be a bit more patient and get to know the process flow around here, before I jump to conclusions. It contains a deliberate provocation that’s supposed to cause reactions (the part about GDB being mostly for nostalgic people).

File bugs for what doesn’t work.

I’m still at the “is it me or is it the tool?” stage, so I am a bit reluctant to file bugs. But, hey, bug reports can be closed if incorrect.

  1. Full support for libc++ on Win32/Win64.

Yup, that would be neat.

And the vast majority of those Windows users will use MSVC instead of Clang. Most will get by just fine with the Microsoft compiler, and they have no reason to really choose Clang, unless they want it for better ISO C++ compliance, or cross-compiling.

I think many more people are upset at Microsofts exorbitant prices than you imagine. After all, Visual Studio Professional is an expensive purchase and Microsoft never really excelled at anything as far as I can tell. I think the popularity of MinGW, which is quite high I believe (over a million downloads) speaks for itself. The Express editions of VS are a joke because they can’t load most VS Pro solutions.

I doubt that’s the general feeling…

Okay, I withdraw that thesis then.

Thank you for replying!

Cheers,
Mikael
– Love Thy Frog!

Thanks for your extensive answer! I now better understand the processes involving LLVM and Clang. I figured you were like most FOSS projects: A group of dudes working more or less together towards a common objective and therefore wanted to try to tune that objective.

I have contributed a Mingw64 Windows buildbot slave that fails predictably in 29 tests on each build, and I will happily offer more CPU power down the road. In fact, I was looking at AMD FX 8150 (eight core) processors for yet another Windows buildbot slave earlier today, but it gets poor reviews. I’ll figure something out later.

I no longer see support of the Microsoft C++ ABI as a decisive feature - people can simply recompile their sources and that’s it. In fact, I think it is much more important to do a full binary release of Clang32/Clang64 using MinGW. Besides, there’s the patent issues with supporting the Microsoft C++ ABI (I posted links to some reverse engineering articles on Microsoft C++ and they listed a whole slew of Microsoft patents in the area). Also, the whole idea of using Clang and Microsoft Visual Studio together is sort of like using a vacuum cleaner and a broom together.

I understand things much better now, thank you for that. And I’ll study the 3.2 docs first thing tomorrow.

2012/6/11 Chandler Carruth <chandlerc@google.com>

Please refrain from discussing patents, reverse engineering, or other specific legal issues on public LLVM and Clang mailing lists. Such conversations are better had with a lawyer.

Will do. Not that I ever had any reverse engineering in mind. And patents are best left for Americans - after all, they have already patented more than 50 percent of the human DNA, however that ever became possible :wink:

2012/6/11 Chandler Carruth <chandlerc@google.com>

Except for exception handling.

- Paul

Hi Chandler,

We need new contributors (like yourself) to prioritize Windows because of their interest and investment in the platform.

I am a Windows developer interested in contributing after watching your nice talk at MS. If a couple of hours a day after my 2-year-old goes to bed can result in useful work. :slight_smile:

Hacking is nice but I imagine a Windows roadmap would help, so newcomers can pick tasks and work on them and measure progress. I still don’t know what I should start working on, and how I could fit the other Windows contributors. Sorry if this is already documented, any pointers are greatly appreciated.

I work full time with unrelated stuff and my main interest is in learning from a big and beautiful C++ project like this. Of course having a complete BSD C++ compiler stack fully supported on Windows would be great.

We are working to figure out the best way to support parsing the Windows headers, but this is a complex issue. Expect some patches to clear the way in the not-too-distant future.

I don’t understand what headers you are talking about, because my first test writing a small Win32 program worked nicely and I just #included <windows.h> with no problems.

Thanks!

Best regards
Mello

Hi Cesar,

I don’t understand what headers you are talking about, because my first test writing a small Win32 program worked nicely and I just #included <windows.h> with no problems.

The headers Chandler are referring to are the Microsoft Visual C++ headers. I suppose you are using the MinGW headers if you can compile cleanly. There’s an issue with some Microsoft construct, force_inline or some such, that Clang does not yet support.

Cheers,
Mikael