RFC: "Building with MinGW on Windows" (DOC, NEW)

Here’s the first part of my Windows set up documentation. This one only covers the process of setting up a normal box for building LLVM and Clang. A later document, which will follow in a few days, will cover how to set up a buildbot slave.

As usual: I’m not a psychic, so if you have opinions regarding or suggestions on this document, please feel free to share them with me (on this mailing list).

To see the document in its full glory, as it is going to look on the website, do this (I am NOT bragging about my document, I am just very happy about the whole switch to Sphinx - the color coding of the examples is awesome thanks to Sphinx and Pygments):

  1. Ensure you have Python and Sphinx installed.
  2. Copy the attached file to llvm/docs.
  3. Run “make html”.
  4. Open up the HTML document at “llvm/docs/_build/html/Building with MinGW on Windows.html”.

I’m especially interested in tips and tricks that Windows users should or must know. Things that can make it difficult for them to succeed. Things that can scare them away (my writing style?). Things that can confuse them. And so on.

Cheers,
Mikael

For those interested in the actual file, it is now attached.

Building with MinGW on Windows.rst (24.9 KB)

Mikael,

Thanks for working on.

My gross comments...

- Canonical URL of LLVM project is; http://llvm.org/ , not www.llvm.org.

- "test-suite and cygwin" could be split out. It would be special thing.

- Memory/storage requirements would be moved to "A.B Debugging". Less
memory would be enough without debug build.

- Python (x64 binary) is available, too. I am using.
  Python 2.7.1 (r271:86832, Nov 27 2010, 17:19:03) [MSC v.1500 64 bit
(AMD64)] on win32

- Distinguish "test-suite" and "{llvm|clang} tests". (Installing GnuWin32)

- Don't recommend to delete MSYS sh.exe in "Installing CMake". Use
MSYS Makefiles instead with msys.

- gnuwin32 is not needed on %PATH% if gnuwin32 is dedicated to tests.
Use LLVM_LIT_TOOLS_DIR instead then.
  (gnuwin32 is not needed when person is working on MSYS bash shell, FYI)

- cmake and python are needed on %PATH%, as long as they are not
invoked explicitly.
  cmake-generated Makefiles know where they are.
  eg. "make edit_cache", "make check"

- If the target of this would be for newbies, you may mention "cmake-gui", too.
  FYI, I am usually use cmake-gui.exe, because I am n00b.

- (Building the Sources), In my experience, make.exe -jN tends to be
choked. I don't know where is -jN-stable gnu make on mingw. (Lemme
know if you knew better version of make!)

- Why don't you mention "make {check|clang-test|check-all}", despite
of introducing gnuwin32?

- You forgot to mention a few important cmake variables.

  - CMAKE_BUILD_TYPE=Release (for most purpose)
  - CMAKE_INSTALL_PREFIX (Who would be happy if clang were installed
onto C:\PROGRA~1\?)
  - LLVM_LIT_TOOLS_DIR, to use gnuwin32 dedicated to testing llvm and clang.
  - LLVM_EXTERNAL_CLANG_SOURCE_DIR, to check out clang outside from
llvm source tree
  - PYTHON_EXECUTABLE, if %PATH% does not contain python.

- Consider to introduce ninja?

Anyway, this could be integrated to "GettingStartedVS.html" :wink:

...Takumi

ps. feel free to visit and ask me on the irc, oftc#llvm.

Hi Takumi,

THANK YOU for your excellent review! It makes all the work worthwhile in itself.

It is getting “late” here (it is early morning) so I am about to call it a day. But I will contemplate and incorporate your suggestions tomorrow, and send you a full reply. I think I originally wrote a document where I addressed both Visual Studio and MinGW, but then decided that there were way too many “if this” and "if that"s to continue on that path.

Thanks again :slight_smile:

Cheers,
Mikael

2012/6/16 NAKAMURA Takumi <geek4civic@gmail.com>

Hi Takumi,

THANK YOU for your excellent review! It makes all the work worthwhile in itself.

It is getting “late” here (it is early morning) so I am about to call it a day. But I will contemplate and incorporate your suggestions tomorrow, and send you a full reply. I think I originally wrote a document where I addressed both Visual Studio and MinGW, but then decided that there were way too many “if this” and "if that"s to continue on that path.

Thanks again :slight_smile:

Under Python installation, you mention that Python is only needed for the test suite. This is not true, as the normal LLVM build still requires Python.

I would seriously consider splitting this up into “building LLVM/Clang” and then “getting and running the test suite.” At first glance, a Windows user may get overwhelmed by the number of steps that are listed in this document. But only a handful of them are actually needed if you just want to build LLVM/Clang and do not care about the test suite. Or at the very least include a Windows quick-start guide at the beginning which just lists Python, CMake, and MinGW. And then introduce Subversion if they want the latest sources, then everything else if they want to run the test suite.

I missed one sentence;

- cmake and python are needed on %PATH%, as long as they are not
invoked explicitly.
cmake-generated Makefiles know where they are.
eg. "make edit_cache", "make check"

"CMake and Python are *not* needed on %PATH%, as long as they are not
invoked explicitly (manually)."

...Why I missed the word "not". :stuck_out_tongue:

FYI, cmake-gui.exe is needed to launch at first time with essential
environment variables.
eg. on MSYS Shell,

/path/to/msys/build $ /c/Program\ Files\ \(x86\)/CMake\ 2.8/bin/cmake-gui.exe

After generating makefiles, we can invoke cmake-gui with "make edit_cache".

...Takumi

Under Python installation, you mention that Python is only needed for the
test suite. This is not true, as the normal LLVM build still requires
Python.

That's right. Now, python>=2.4 is essential.
(In contrast, perl is not required, IIRC)

I would seriously consider splitting this up into "building LLVM/Clang" and
then "getting and running the test suite." At first glance, a Windows user
may get overwhelmed by the number of steps that are listed in this document.
But only a handful of them are actually needed if you just want to build
LLVM/Clang and do not care about the test suite. Or at the very least
include a Windows quick-start guide at the beginning which just lists
Python, CMake, and MinGW. And then introduce Subversion if they want the
latest sources, then everything else if they want to run the test suite.

As long as building llvm and clang with cmake on win32,
We may say, "Choose your compiler, mingw32-gcc.exe or ms cl.exe (or,
mingw32-clang.exe)"
Processes among compilers would be similar.

Suggested sections;

  - Install CMake
  - Install Python
    - Install VS Express (optional)
    - Install mingw32 (optional)
    - Install MSYS (optional)
  - Configure your build with cmake
    - Build and install on VS
    - Build and install on MSYS (or mingw32 cmd.exe)
  - Inter-platform testing with Lit
  - Run test-suite with Cygwin (optional! Also Takumi has not tried ever!)

BTW, shall we introduce "msbuild" here, too?

msbuild -m -nologo -v:m tools\clang\test\clang-test.vcxproj -p:Configuration=%1

...Takumi

No problem. Perhaps some of it can be converted into mere patches to existing documentation?

That leaves us with three docs so far:

  1. Testing.
  2. Windows build.
  3. Windows buildbot slave.

2012/6/16 Justin Holewinski <justin.holewinski@gmail.com>

As for memory/storage, I think you want these presented as early as possible so people don’t go setting up a box only to find out, half a week later, that they can never do what they intended to do (build Clang in debug mode).

The reason I use 32-bit Python is that it is what you need, if you want to make a Windows buildbot slave because the Win32 bindings are needed and I don’t think Buildbot works with the 64-bit Win64 bindings yet. So I am trying to share paths here at the cost of having the user install a 32-bit version of Python even on a 64-bit system. The document actually started out as a guide for how to set up a Windows buildbot slave, but then it got split up.

No, you’re working on LLVM and/or Clang, so you can’t possibly be a n00b :slight_smile:

The only result I’ve seen with make is that there’s no difference between make -j2 and j-4: it only uses 50 percent CPU load in both cases (also on a quad-core box).

The “make check” omission is simply an error on my part.

About integrating into VS document: I am using a new path here, it seems, cmake + mingw32. I don’t even need to have MSVC installed to build this way.

Ninja: great idea. I tried pulling it yesterday with the intent of building it, but there are some errors. Perhaps they’ve been fixed today.

2012/6/16 NAKAMURA Takumi <geek4civic@gmail.com>

The whole purpose of the document is to teach the user how to build with MinGW - without Microsoft Visual Studio. As far as I know, MSVS cannot be used together with Clang and I think many people are interested in trying out Clang. That’s what I am trying to make them able to do without requiring the installation of MSVS.

2012/6/17 NAKAMURA Takumi <geek4civic@gmail.com>

The reason I use 32-bit Python is that it is what you need, if you want to
make a Windows buildbot slave because the Win32 bindings are needed and I
don't think Buildbot works with the 64-bit Win64 bindings yet. So I am
trying to share paths here at the cost of having the user install a 32-bit
version of Python even on a 64-bit system. The document actually started
out as a guide for how to set up a Windows buildbot slave, but then it got
split up.

My buildslave (mingw and msvc) is running on python x64.
http://bb.pgr.jp/buildslaves/win7
I don't remember how it was harder to set up the slave.

The only result I've seen with make is that there's no difference between
make -j2 and j-4: it only uses 50 percent CPU load in both cases (also on a
quad-core box).

I suspect your make.exe might not be capable of -jN.
gussing "mingw32-make.exe"
For me, -j4 shows me much more improvement than -j2. (and tends to
hang up more earlier)

About integrating into VS document: I am using a new path here, it seems,
cmake + mingw32. I don't even need to have MSVC installed to build this
way.

Me too, though, I would not like to install and use MS development
environments any more. :stuck_out_tongue:

As long as cmake/mingw (not msys) is used, the difference against
cmake/msvc would not be so huge.
Extremely, only compiler differs each other.
(You know, we are developing 3rd compiler on win32, cl.exe, gcc.exe,
... and clang.exe!)

I won't let you install and describe how to build with MS cl.exe, for now.
I will tweak either (or both) of documents in future.
Go on now :wink:

...Takumi

As for Subversion, I think the general strategy of the LLVM project is that people should build from the trunk. When people pop up with questions regarding release builds, they are usually referred to using the trunk version. So I think Subversion is mandatory.

2012/6/16 Justin Holewinski <justin.holewinski@gmail.com>

As for Ninja, I’d like to wait until it is officially part of the released CMake. As I understand it, Windows support has been checked into the CMake repository but no offical release supporting Ninja has been made yet.

I just retrieved Ninja from Github and this time it built without problems.

2012/6/16 NAKAMURA Takumi <geek4civic@gmail.com>

Now I can’t make CMake use “MSYS Makefiles”. I get an error about GNU v4.7.0 being broken as it can’t build a simple test program. Using “MinGW Makefiles”, everything works out of the box. And I am trying to give the user the most Windowsy experience (to avoid Cygwin, GnuWin32, and MSYS, if I can). So I think I’ll stick to my current angle - which is to let the user install only MinGW, CMake, and Python, and then leave out any mention of MSYS (until the Testing on Windows document).

P.S. There is one more compiler for Windows: The OpenWatcom compiler (my favorite for its marvellous debugger), but unfortunately it does not support newer C++ advances so it can’t be used to build LLVM and Clang as far as I recall. At the very least, it is not usable for Unicode programs as it does not support wstreams.

2012/6/16 NAKAMURA Takumi <geek4civic@gmail.com>

Sigh. It was just the PATH environment variable that was set up incorrectly. Now MSYS works too. Turns out you have to have MSYS before GnuWin32 in the path.

2012/6/17 Mikael Lyngvig <mikael@lyngvig.org>

MSbuild should go into the MSVC document, I think. I’ll probably look into converting it to Sphinx within not too long. I just have a few things I need to do first.

2012/6/17 NAKAMURA Takumi <geek4civic@gmail.com>

Oh, didn’t realize that there already is a semi-official build of CMake with Ninja support. I am trying it out right now and it seems very promising. So I’ll change the document to use Ninja instead of “MinGW Makefiles”. No need to waste energy on MinGW and MSYS Makefiles when Ninja outdoes them both. I thought people would have to build CMake themselves, something I’d rather avoid to keep things simple.

2012/6/17 Mikael Lyngvig <mikael@lyngvig.org>

Takumi, I’ve had to give up of the idea of presenting “make check” (ninja check), “make clang-test”, and so on. It needs GnuWin32 and I got some beating for having included that in the original document. So all checks and tests are presented in the “Testing on Windows” document or whichever name it’ll end up having.

Instead, Ninja is now the my prescribed build tool on Windows :slight_smile: No more funny “make -jN” issues. No more waiting forever on “make -j1”.

– Disclaimer: I am not arrogant in real life, so if you perceive me as being arrogant, you are to blame :wink:

2012/6/17 Mikael Lyngvig <mikael@lyngvig.org>