Using Clang 5.0.0 RC2 with MSVC dev builds

Clang Frontend
1

Hi Clang devs,

I'm trying to use Clang/LLVM with MSVC's STL in order to keep our code conformant and Clang-compatible, and I've encountered a problematic behavior change between Clang 4.0 and 5.0.

The only Visual Studio installation on my machine is VS 2013. (I work on msvcp140.dll, so my life is easier if I avoid anything that installs it system-wide like VS 2015/2017.)

I am NOT using a VS 2013 Developer Command Prompt.

I'm starting with a plain command prompt, using a development build of VS 2017 (somewhat beyond 15.3) by adding it to PATH, INCLUDE, and LIB. (I'm NOT running an installer for this development build, nor am I modifying the registry in any way.) The directory structure of our development build is simple, although it does not correspond to the publicly distributed installer. On my machine, I have S:\binaries\x86ret, with S:\binaries\x86ret\bin\i386\cl.exe containing the x86-native compiler, S:\binaries\x86ret\inc\vector containing my favorite header, and S:\binaries\x86ret\lib\i386\libcpmt.lib containing the STL's static lib. (The CRT's headers and libs are in different directories, but Clang isn't having trouble with INCLUDE and LIB.)

I've also used 7-Zip to extract Clang/LLVM 4.0.1 x64-hosted to a directory, WITHOUT running an installer.

In this scenario, everything works fine, and we've been testing our STL with Clang/LLVM since June (originally 4.0.0, now 4.0.1):

C:\Temp>echo %PATH%
S:\WCFB01\vctools\nonship\bin\x86;S:\binaries\x86ret\bin\i386;S:\WCFB01\vctools\Compiler\Regress\bin\AMD64\xperf;S:\WCFB01\vctools\NonShip\ClangLLVM\bin;C:\Temp\Exec;C:\Temp\git\cmd;C:\windows\system32;C:\windows;C:\windows\System32\Wbem;C:\windows\System32\WindowsPowerShell\v1.0\;C:\Program Files (x86)\Windows Kits\8.1\Windows Performance Toolkit\;C:\Program Files\Microsoft SQL Server\110\Tools\Binn\;C:\Program Files (x86)\Microsoft SDKs\TypeScript\1.0\;C:\Users\stl\AppData\Local\Microsoft\WindowsApps;;S:\WCFB01\vctools\Compiler\Regress\bin;S:\WCFB01\vctools\Compiler\Regress\bin\AMD64;;S:\WCFB01\tools\perl\bin

C:\Temp>echo %INCLUDE%
S:\binaries\x86ret\atlmfc\include;S:\binaries\x86ret\inc;S:\WCFB01\ExternalApis\Windows\10\sdk\inc;S:\WCFB01\ExternalApis\UnifiedCRT\inc;S:\WCFB01\ExternalAPIs\DirectX\Jun2010\Include;

C:\Temp>echo %LIB%
S:\binaries\x86ret\atlmfc\lib\i386;S:\binaries\x86ret\lib\i386;S:\WCFB01\ExternalApis\Windows\10\sdk\lib\x86;S:\WCFB01\ExternalApis\UnifiedCRT\lib\i386;S:\WCFB01\ExternalApis\NetFX\v4.5\lib\x86;S:\WCFB01\ExternalAPIs\DirectX\Jun2010\Lib\x86;

C:\Temp>type meow.cpp
#include <iostream>
int main() {
#ifdef __clang__
    std::cout << "I am Clang: " << __clang_version__ << "\n";
#elif defined(_MSC_VER)
    std::cout << "I am C1XX: " << _MSC_FULL_VER << "\n";
#else
    std::cout << "I am something else.\n";
#endif
}

C:\Temp>where cl
S:\binaries\x86ret\bin\i386\cl.exe

C:\Temp>where link
S:\binaries\x86ret\bin\i386\link.exe

C:\Temp>cl /EHsc /nologo /W4 /MT meow.cpp && meow
meow.cpp
I am C1XX: 191125614

C:\Temp>where clang-cl
S:\WCFB01\vctools\NonShip\ClangLLVM\bin\clang-cl.exe

C:\Temp>clang-cl -v -m32 /EHsc /nologo /W4 /MT meow.cpp && meow
clang version 4.0.1 (tags/RELEASE_401/final)
Target: i386-pc-windows-msvc
Thread model: posix
InstalledDir: S:\WCFB01\vctools\NonShip\ClangLLVM\bin
"S:\\WCFB01\\vctools\\NonShip\\ClangLLVM\\bin\\clang-cl.exe" -cc1 -triple i386-pc-windows-msvc19.11.25614 -emit-obj -mrelax-all -mincremental-linker-compatible -disable-free -main-file-name meow.cpp -mrelocation-model static -mthread-model posix -mdisable-fp-elim -relaxed-aliasing -fmath-errno -masm-verbose -mconstructor-aliases -target-cpu pentium4 -D_MT -flto-visibility-public-std --dependent-lib=libcmt --dependent-lib=oldnames -stack-protector 2 -fcxx-exceptions -fexceptions -fexternc-nounwind -fms-volatile -fdiagnostics-format msvc -v -dwarf-column-info -debugger-tuning=gdb -resource-dir "S:\\WCFB01\\vctools\\NonShip\\ClangLLVM\\bin\\..\\lib\\clang\\4.0.1" -internal-isystem "S:\\WCFB01\\vctools\\NonShip\\ClangLLVM\\bin\\..\\lib\\clang\\4.0.1\\include" -internal-isystem "S:\\binaries\\x86ret\\atlmfc\\include" -internal-isystem "S:\\binaries\\x86ret\\inc" -internal-isystem "S:\\WCFB01\\ExternalApis\\Windows\\10\\sdk\\inc" -internal-isystem "S:\\WCFB01\\ExternalApis\\UnifiedCRT\\inc" -internal-isystem "S:\\WCFB01\\ExternalAPIs\\DirectX\\Jun2010\\Include" -WCL4 -fdeprecated-macro -fdebug-compilation-dir "C:\\Temp" -ferror-limit 19 -fmessage-length 120 -fms-extensions -fms-compatibility -fms-compatibility-version=19.11.25614 -std=c++14 -fdelayed-template-parsing -fobjc-runtime=gcc -fdiagnostics-show-option -fcolor-diagnostics -o "C:\\Users\\stl\\AppData\\Local\\Temp\\1\\meow-291efe.obj" -x c++ meow.cpp
clang -cc1 version 4.0.1 based upon LLVM 4.0.1 default target x86_64-pc-windows-msvc
#include "..." search starts here:
#include <...> search starts here:
S:\WCFB01\vctools\NonShip\ClangLLVM\bin\..\lib\clang\4.0.1\include
S:\binaries\x86ret\atlmfc\include
S:\binaries\x86ret\inc
S:\WCFB01\ExternalApis\Windows\10\sdk\inc
S:\WCFB01\ExternalApis\UnifiedCRT\inc
S:\WCFB01\ExternalAPIs\DirectX\Jun2010\Include
End of search list.
"S:\\binaries\\x86ret\\bin\\i386\\link.exe" -out:meow.exe -nologo "C:\\Users\\stl\\AppData\\Local\\Temp\\1\\meow-291efe.obj"
I am Clang: 4.0.1 (tags/RELEASE_401/final)

Observe that my C1XX version, 19.11.25614, is detected by Clang which passes -fms-compatibility-version=19.11.25614 and defaults to -std=c++14. The linker from my development build is also used, S:\binaries\x86ret\bin\i386\link.exe.

Next, I deleted everything in S:\WCFB01\vctools\NonShip\ClangLLVM\, downloaded http://prereleases.llvm.org/5.0.0/rc2/LLVM-5.0.0-rc2-win64.exe (grinding my teeth at non-secure HTTP), and extracted 5.0.0-rc2 there:

S:\WCFB01\vctools\NonShip\ClangLLVM>dir /b

S:\WCFB01\vctools\NonShip\ClangLLVM>7z x C:\Temp\LLVM-5.0.0-rc2-win64.exe -x!Uninstall.exe -x!$PLUGINSDIR\*

7-Zip [64] 16.04 : Copyright (c) 1999-2016 Igor Pavlov : 2016-10-04

Scanning the drive for archives:
1 file, 131097761 bytes (126 MiB)

Extracting archive: C:\Temp\LLVM-5.0.0-rc2-win64.exe

2

2. Is there a simple way for me to get Clang 4.0's behavior back, where

having cl.exe and link.exe on my PATH makes Clang automatically detect the
correct -fms-compatibility-version and link.exe to be used?

I haven't figured out what changed, but that definitely seems like a
regression.

3

I think this change might be causing the problem you are seeing: https://reviews.llvm.org/D30758

4

You could use the prebuilt binaries at https://is.gd/chromeclang to narrow down the regression range. The change don linked to sounds likely though, so hopefully Zach can say if this was an intentional change :slight_smile:

5

As I read the current code, we search the environment, then we try the new
setup config API, and then we try the registry. That seems correct.

I think the change from 4.0 to 5.0 was that when we look at the
environment, we now try these variables in order:
1. VCToolsInstallDir: this is new
2. VCINSTALLDIR: we always tried this before PATH
3. PATH

Are either of those first two variables set to point to 2013?

Once clang finds cl.exe it calls GetFileVersionInfoW to work out the
version, which should work as intended.

6

Since we most likely cannot reproduce this without access to this same development environment, I think the easiest way to figure this out would be for Stephan to debug into clang itself. It definitely sounds like a problem, but I think only you will be able to figure out exactly what it is. Luckily it should be easy to figure out. Set a breakpoint in the constructor of MSVCToolChain in clang/lib/Driver/ToolChains/MSVC.cpp and see what’s going on in there. Most likely it’s the VCToolsInstallDir or VCINSTALLDIR as Reid mentioned.

7

I can’t test this since I don’t have windows, but I think the problem is the additional directory checks in findVCToolChainViaEnvironment() which don’t match his path to cl.exe, i.e., S:\binaries\x86ret\bin\i386\cl.exe.

8

[Reid Kleckner]

As I read the current code, we search the environment, then we try the
new setup config API, and then we try the registry. That seems correct.

Agreed.

I think the change from 4.0 to 5.0 was that when we look at the
environment, we now try these variables in order:
1. VCToolsInstallDir: this is new
2. VCINSTALLDIR: we always tried this before PATH
3. PATH
Are either of those first two variables set to point to 2013?

Neither are set:

C:\Temp>set VCToolsInstallDir
Environment variable VCToolsInstallDir not defined

C:\Temp>set VCINSTALLDIR
Environment variable VCINSTALLDIR not defined

[Zachary Turner]

Since we most likely cannot reproduce this
without access to this same development environment

I'll try to set up an imitation for you. We have NuGet packages of the toolset (which are basically ZIPs), and extracting them and rearranging the directories should be sufficient to replicate the structure of our dev builds.

I think the easiest way to figure this out would be for Stephan to
debug into clang itself. It definitely sounds like a problem, but I
think only you will be able to figure out exactly what it is.
Luckily it should be easy to figure out. Set a breakpoint in the
constructor of MSVCToolChain in clang/lib/Driver/ToolChains/MSVC.cpp
and see what's going on in there. Most likely it's the VCToolsInstallDir
or VCINSTALLDIR as Reid mentioned.

That'll be my last resort, since I think I'll need to build Clang with debug info, and I haven't gone through the process of building Clang yet.

[don hinton]

I can't test this since I don't have windows, but I think the problem
is the additional directory checks in findVCToolChainViaEnvironment()
which don't match his path to cl.exe, i.e.,
S:\binaries\x86ret\bin\i386\cl.exe.

I think you're correct (which is why I explained my directory structure). If I had an override option to say "you'll find cl.exe and link.exe on the path, and I've set up INCLUDE and LIB for you, there's no need to inspect the directory structure further", I think that would be sufficient.

Thanks,
STL

9

Unfortunately, it looks like the current test is based on the path.
Perhaps it would be better to actually run a simple test compile, a la,
configure/cmake, and spit out the value of _MSC_VER instead.

10

[don hinton]

Unfortunately, it looks like the current test is based on the path. Perhaps
it would be better to actually run a simple test compile, a la, configure/cmake,
and spit out the value of _MSC_VER instead.

My usage scenario involves running over 4,000 invocations of clang-cl.exe (one for each of libc++'s many small tests). Having each of those launch cl.exe would be highly undesirable.

As I understand it, if you can find cl.exe (which you can, it's right there on the path), you can get its version information from GetFileVersionInfoW() like Reid said, without invoking it. The issue seems to be the additional directory structure analysis to find the "root path" (is this to find where the libraries are?), which I believe is causing the cl.exe to be rejected as "not a tool chain".

I am continuing to set up my VM and should have a repro soon.

Thanks,
STL

11

We use the toolchain to find three different things:

  1. bin dir
    vs 2017:
    /bin/HostX64/x86
    /bin/HostX64/x64

/bin/HostX86/x86
/bin/HostX86/x64

pre-2017:
/bin
/bin/amd64
2) include dir
/include
3) lib dir
vs 2017:
/lib/x86
/lib/x64
pre-2017:
/lib
/lib/amd64

It sounds like if you had a way to configure each of these 3 paths independently, everything would work? Alternatively, is there any way you could organize your existing directory structure into this format?

12

[Zachary Turner]

We use the toolchain to find three different things:
1) bin dir
vs 2017:
<root>/bin/HostX64/x86
<root>/bin/HostX64/x64
<root>/bin/HostX86/x86
<root>/bin/HostX86/x64
pre-2017:
<root>/bin
<root>/bin/amd64
2) include dir
<root>/include
3) lib dir
vs 2017:
<root>/lib/x86
<root>/lib/x64
pre-2017:
<root>/lib
<root>/lib/amd64
It sounds like if you had a way to configure each of these 3 paths independently, everything would work?

I suspect so - as long as it were sufficiently general to avoid assuming the stuff in the middle like "HostX86" - although I think that'd be far more than I need, and more complicated to use in practice. I'm already setting up INCLUDE and LIB, and Clang seems to be using those fine (as long as the ms-compatibility-version and linker are okay). As for the binaries, I have either the x86-native or the x64-native toolset on the PATH, and that's the cl.exe whose version I want Clang to inspect, and the link.exe that I want Clang to use. (We always use x64-hosted Clang, and instruct it to target either x86 with -m32 or x64 with -m64.)

Clang 4.0 appeared to have this behavior, of trusting the PATH.

Alternatively, is there any way you could organize your existing directory structure into this format?

That wouldn't really be possible in practice (and I'm grinding my teeth while saying that, since I'm usually the vendor hearing a customer saying that). Our dev infrastructure assumes this internal layout and changing it would be highly disruptive and time-consuming. (In principle, I agree that the internal builds should match the publicly distributed structure, argh.)

Thanks,
STL

13

The problem is that must be in the form “…/VC”, which isn’t always the case:

lib/Driver/ToolChains/MSVC.cpp:135: if (IsBin) {
lib/Driver/ToolChains/MSVC.cpp:136: llvm::StringRef ParentPath = llvm::sys::path::parent_path(TestPath);
lib/Driver/ToolChains/MSVC.cpp:137: if (llvm::sys::path::filename(ParentPath) == “VC”) {
lib/Driver/ToolChains/MSVC.cpp:138: Path = ParentPath;
lib/Driver/ToolChains/MSVC.cpp:139: IsVS2017OrNewer = false; << Should this be set here?
lib/Driver/ToolChains/MSVC.cpp:140: return true;
lib/Driver/ToolChains/MSVC.cpp:141: }

14

Here's how to repro my directory structure in a clean VM. (I have to run an installer to extract the WinSDK, but then I just copy its files around.) I could prepare similar instructions to gather this directory structure from a real installation of VS, for copying to a clean VM, if you would prefer that.

1. Download (600 MB): https://www.nuget.org/api/v2/package/VisualCppTools.Community.D14Layout/14.10.25506

2. Rename visualcpptools.community.d14layout.14.10.25506.nupkg to nuget.zip .

3. Extract nuget.zip to C:\Temp , creating C:\Temp\nuget\lib\native\branchinfo.txt (for example).

4. In C:\Temp run:

robocopy nuget\lib\native\bin binaries\x86ret\bin\i386
robocopy nuget\lib\native\bin\1033 binaries\x86ret\bin\i386\1033 /s
robocopy nuget\lib\native\bin\amd64 binaries\amd64ret\bin\amd64 /s
robocopy nuget\lib\native\bin\x86_amd64 binaries\x86ret\bin\x86_amd64 /s
robocopy nuget\lib\native\bin\amd64_x86 binaries\amd64ret\bin\amd64_x86 /s
robocopy nuget\lib\native\include binaries\x86ret\inc /s
robocopy nuget\lib\native\include binaries\amd64ret\inc /s
robocopy nuget\lib\native\lib binaries\x86ret\lib\i386
robocopy nuget\lib\native\lib\amd64 binaries\amd64ret\lib\amd64 /s

5. We need the WinSDK:

"Download the .EXE"
"Install ... to this computer" is fine, Next.
Either Yes or No to anonymous usage data is fine, Next.
If you accept the EULA, Accept.
I'm too lazy to figure out the minimal set of features, so leave them all checked, Install.

6. In C:\Temp run:

robocopy "C:\Program Files (x86)\Windows Kits\10\Include\10.0.15063.0\um" ExternalApis\Windows\10\sdk\inc /s
robocopy "C:\Program Files (x86)\Windows Kits\10\Include\10.0.15063.0\shared" ExternalApis\Windows\10\sdk\inc /s
robocopy "C:\Program Files (x86)\Windows Kits\10\Include\10.0.15063.0\ucrt" ExternalApis\UnifiedCRT\inc /s
robocopy "C:\Program Files (x86)\Windows Kits\10\Lib\10.0.15063.0\um\x86" ExternalApis\Windows\10\sdk\lib\x86 /s
robocopy "C:\Program Files (x86)\Windows Kits\10\Lib\10.0.15063.0\um\x64" ExternalApis\Windows\10\sdk\lib\x64 /s
robocopy "C:\Program Files (x86)\Windows Kits\10\Lib\10.0.15063.0\ucrt\x86" ExternalApis\UnifiedCRT\lib\i386 /s
robocopy "C:\Program Files (x86)\Windows Kits\10\Lib\10.0.15063.0\ucrt\x64" ExternalApis\UnifiedCRT\lib\amd64 /s

7. Download LLVM-5.0.0-rc2-win64.exe and in C:\Temp use 7-Zip to extract it:

7z x LLVM-5.0.0-rc2-win64.exe -x!Uninstall.exe -x!$PLUGINSDIR\* -oClangLLVM

8. Now you can set up an x86 or an x64 command prompt (these are mutually exclusive options):

title x86 Dev Build
set PATH=C:\Temp\binaries\x86ret\bin\i386;C:\Temp\ClangLLVM\bin;%PATH%
set INCLUDE=C:\Temp\binaries\x86ret\inc;C:\Temp\ExternalApis\Windows\10\sdk\inc;C:\Temp\ExternalApis\UnifiedCRT\inc
set LIB=C:\Temp\binaries\x86ret\lib\i386;C:\Temp\ExternalApis\Windows\10\sdk\lib\x86;C:\Temp\ExternalApis\UnifiedCRT\lib\i386

OR:

title x64 Dev Build
set PATH=C:\Temp\binaries\amd64ret\bin\amd64;C:\Temp\ClangLLVM\bin;%PATH%
set INCLUDE=C:\Temp\binaries\amd64ret\inc;C:\Temp\ExternalApis\Windows\10\sdk\inc;C:\Temp\ExternalApis\UnifiedCRT\inc
set LIB=C:\Temp\binaries\amd64ret\lib\amd64;C:\Temp\ExternalApis\Windows\10\sdk\lib\x64;C:\Temp\ExternalApis\UnifiedCRT\lib\amd64

9. Let's try the "x86 Dev Build".

NOTE: this VM is totally clean and doesn't have VS 2013 installed, so the behavior is a little different from my original repro (Clang can't find VS, instead of erroneously finding VS 2013), but the behavior is similarly undesirable.

C:\Temp>echo %PATH%
C:\Temp\binaries\x86ret\bin\i386;C:\Temp\ClangLLVM\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit\;C:\Users\stl\AppData\Local\Microsoft\WindowsApps;

C:\Temp>echo %INCLUDE%
C:\Temp\binaries\x86ret\inc;C:\Temp\ExternalApis\Windows\10\sdk\inc;C:\Temp\ExternalApis\UnifiedCRT\inc

C:\Temp>echo %LIB%
C:\Temp\binaries\x86ret\lib\i386;C:\Temp\ExternalApis\Windows\10\sdk\lib\x86;C:\Temp\ExternalApis\UnifiedCRT\lib\i386

C:\Temp>type meow.cpp
#include <iostream>
int main() {
#ifdef __clang__
    std::cout << "I am Clang: " << __clang_version__ << "\n";
#elif defined(_MSC_VER)
    std::cout << "I am C1XX: " << _MSC_FULL_VER << "\n";
#else
    std::cout << "I am something else.\n";
#endif
    std::cout << "And I am " << sizeof(void *) * 8 << "-bit.\n";
}

C:\Temp>where cl
C:\Temp\binaries\x86ret\bin\i386\cl.exe

C:\Temp>where link
C:\Temp\binaries\x86ret\bin\i386\link.exe

C:\Temp>cl /EHsc /nologo /W4 /MT meow.cpp && meow
meow.cpp
I am C1XX: 191125506
And I am 32-bit.

C:\Temp>where clang-cl
C:\Temp\ClangLLVM\bin\clang-cl.exe

C:\Temp>clang-cl -v -m32 /EHsc /nologo /W4 /MT meow.cpp && meow
clang version 5.0.0 (tags/RELEASE_500/rc2)
Target: i386-pc-windows-msvc
Thread model: posix
InstalledDir: C:\Temp\ClangLLVM\bin
clang-cl.exe: warning: unable to find a Visual Studio installation; try running Clang from a developer command prompt [-Wmsvc-not-found]
"C:\\Temp\\ClangLLVM\\bin\\clang-cl.exe" -cc1 -triple i386-pc-windows-msvc18.0.0 -emit-obj -mrelax-all -mincremental-linker-compatible -disable-free -main-file-name meow.cpp -mrelocation-model static -mthread-model posix -mdisable-fp-elim -relaxed-aliasing -fmath-errno -masm-verbose -mconstructor-aliases -target-cpu pentium4 -D_MT -flto-visibility-public-std --dependent-lib=libcmt --dependent-lib=oldnames -stack-protector 2 -fcxx-exceptions -fexceptions -fexternc-nounwind -fms-volatile -fdiagnostics-format msvc -v -dwarf-column-info -debugger-tuning=gdb -resource-dir "C:\\Temp\\ClangLLVM\\lib\\clang\\5.0.0" -internal-isystem "C:\\Temp\\ClangLLVM\\lib\\clang\\5.0.0\\include" -internal-isystem "C:\\Temp\\binaries\\x86ret\\inc" -internal-isystem "C:\\Temp\\ExternalApis\\Windows\\10\\sdk\\inc" -internal-isystem "C:\\Temp\\ExternalApis\\UnifiedCRT\\inc" -WCL4 -fdeprecated-macro -fdebug-compilation-dir "C:\\Temp" -ferror-limit 19 -fmessage-length 120 -fms-extensions -fms-compatibility -fms-compatibility-version=18 -std=c++11 -fno-threadsafe-statics -fdelayed-template-parsing -fobjc-runtime=gcc -fdiagnostics-show-option -fcolor-diagnostics -o "C:\\Users\\stl\\AppData\\Local\\Temp\\2\\meow-952e26.obj" -x c++ meow.cpp
clang -cc1 version 5.0.0 based upon LLVM 5.0.0-rc2 default target x86_64-pc-windows-msvc
#include "..." search starts here:
#include <...> search starts here:
C:\Temp\ClangLLVM\lib\clang\5.0.0\include
C:\Temp\binaries\x86ret\inc
C:\Temp\ExternalApis\Windows\10\sdk\inc
C:\Temp\ExternalApis\UnifiedCRT\inc
End of search list.
In file included from meow.cpp:1:
In file included from C:\Temp\binaries\x86ret\inc\iostream:6:
In file included from C:\Temp\binaries\x86ret\inc\istream:6:
In file included from C:\Temp\binaries\x86ret\inc\ostream:6:
In file included from C:\Temp\binaries\x86ret\inc\ios:6:
In file included from C:\Temp\binaries\x86ret\inc\xlocnum:7:
In file included from C:\Temp\binaries\x86ret\inc\cmath:617:
In file included from C:\Temp\binaries\x86ret\inc\xtgmath.h:9:
C:\Temp\binaries\x86ret\inc\xtr1common(204,22) : error: use of undeclared identifier 'char16_t'
        struct _Is_integral<char16_t>
                            ^
[...more errors...]

C:\Temp>clang-cl -v -m32 /EHsc /nologo /W4 /MT -fms-compatibility-version=19.11.25506 meow.cpp && meow
clang version 5.0.0 (tags/RELEASE_500/rc2)
Target: i386-pc-windows-msvc
Thread model: posix
InstalledDir: C:\Temp\ClangLLVM\bin
clang-cl.exe: warning: unable to find a Visual Studio installation; try running Clang from a developer command prompt [-Wmsvc-not-found]
"C:\\Temp\\ClangLLVM\\bin\\clang-cl.exe" -cc1 -triple i386-pc-windows-msvc19.11.25506 -emit-obj -mrelax-all -mincremental-linker-compatible -disable-free -main-file-name meow.cpp -mrelocation-model static -mthread-model posix -mdisable-fp-elim -relaxed-aliasing -fmath-errno -masm-verbose -mconstructor-aliases -target-cpu pentium4 -D_MT -flto-visibility-public-std --dependent-lib=libcmt --dependent-lib=oldnames -stack-protector 2 -fcxx-exceptions -fexceptions -fexternc-nounwind -fms-volatile -fdiagnostics-format msvc -v -dwarf-column-info -debugger-tuning=gdb -resource-dir "C:\\Temp\\ClangLLVM\\lib\\clang\\5.0.0" -internal-isystem "C:\\Temp\\ClangLLVM\\lib\\clang\\5.0.0\\include" -internal-isystem "C:\\Temp\\binaries\\x86ret\\inc" -internal-isystem "C:\\Temp\\ExternalApis\\Windows\\10\\sdk\\inc" -internal-isystem "C:\\Temp\\ExternalApis\\UnifiedCRT\\inc" -WCL4 -fdeprecated-macro -fdebug-compilation-dir "C:\\Temp" -ferror-limit 19 -fmessage-length 120 -fms-extensions -fms-compatibility -fms-compatibility-version=19.11.25506 -std=c++14 -fdelayed-template-parsing -fobjc-runtime=gcc -fdiagnostics-show-option -fcolor-diagnostics -o "C:\\Users\\stl\\AppData\\Local\\Temp\\2\\meow-8771d8.obj" -x c++ meow.cpp
clang -cc1 version 5.0.0 based upon LLVM 5.0.0-rc2 default target x86_64-pc-windows-msvc
#include "..." search starts here:
#include <...> search starts here:
C:\Temp\ClangLLVM\lib\clang\5.0.0\include
C:\Temp\binaries\x86ret\inc
C:\Temp\ExternalApis\Windows\10\sdk\inc
C:\Temp\ExternalApis\UnifiedCRT\inc
End of search list.
"link.exe" -out:meow.exe -nologo "C:\\Users\\stl\\AppData\\Local\\Temp\\2\\meow-8771d8.obj"
clang-cl.exe: error: unable to execute command: program not executable
clang-cl.exe: error: linker command failed with exit code 1 (use -v to see invocation)

C:\Temp>clang-cl -v -m32 /EHsc /nologo /W4 /MT -fms-compatibility-version=19.11.25506 -fuse-ld=lld-link meow.cpp && meow
clang version 5.0.0 (tags/RELEASE_500/rc2)
Target: i386-pc-windows-msvc
Thread model: posix
InstalledDir: C:\Temp\ClangLLVM\bin
clang-cl.exe: warning: unable to find a Visual Studio installation; try running Clang from a developer command prompt [-Wmsvc-not-found]
"C:\\Temp\\ClangLLVM\\bin\\clang-cl.exe" -cc1 -triple i386-pc-windows-msvc19.11.25506 -emit-obj -mrelax-all -mincremental-linker-compatible -disable-free -main-file-name meow.cpp -mrelocation-model static -mthread-model posix -mdisable-fp-elim -relaxed-aliasing -fmath-errno -masm-verbose -mconstructor-aliases -target-cpu pentium4 -D_MT -flto-visibility-public-std --dependent-lib=libcmt --dependent-lib=oldnames -stack-protector 2 -fcxx-exceptions -fexceptions -fexternc-nounwind -fms-volatile -fdiagnostics-format msvc -v -dwarf-column-info -debugger-tuning=gdb -resource-dir "C:\\Temp\\ClangLLVM\\lib\\clang\\5.0.0" -internal-isystem "C:\\Temp\\ClangLLVM\\lib\\clang\\5.0.0\\include" -internal-isystem "C:\\Temp\\binaries\\x86ret\\inc" -internal-isystem "C:\\Temp\\ExternalApis\\Windows\\10\\sdk\\inc" -internal-isystem "C:\\Temp\\ExternalApis\\UnifiedCRT\\inc" -WCL4 -fdeprecated-macro -fdebug-compilation-dir "C:\\Temp" -ferror-limit 19 -fmessage-length 120 -fms-extensions -fms-compatibility -fms-compatibility-version=19.11.25506 -std=c++14 -fdelayed-template-parsing -fobjc-runtime=gcc -fdiagnostics-show-option -fcolor-diagnostics -o "C:\\Users\\stl\\AppData\\Local\\Temp\\2\\meow-67d57c.obj" -x c++ meow.cpp
clang -cc1 version 5.0.0 based upon LLVM 5.0.0-rc2 default target x86_64-pc-windows-msvc
#include "..." search starts here:
#include <...> search starts here:
C:\Temp\ClangLLVM\lib\clang\5.0.0\include
C:\Temp\binaries\x86ret\inc
C:\Temp\ExternalApis\Windows\10\sdk\inc
C:\Temp\ExternalApis\UnifiedCRT\inc
End of search list.
"C:\\Temp\\ClangLLVM\\bin\\lld-link" -out:meow.exe -nologo "C:\\Users\\stl\\AppData\\Local\\Temp\\2\\meow-67d57c.obj"
I am Clang: 5.0.0 (tags/RELEASE_500/rc2)
And I am 32-bit.

10. And the "x64 Dev Build":

C:\Temp>echo %PATH%
C:\Temp\binaries\amd64ret\bin\amd64;C:\Temp\ClangLLVM\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit\;C:\Users\stl\AppData\Local\Microsoft\WindowsApps;

C:\Temp>echo %INCLUDE%
C:\Temp\binaries\amd64ret\inc;C:\Temp\ExternalApis\Windows\10\sdk\inc;C:\Temp\ExternalApis\UnifiedCRT\inc

C:\Temp>echo %LIB%
C:\Temp\binaries\amd64ret\lib\amd64;C:\Temp\ExternalApis\Windows\10\sdk\lib\x64;C:\Temp\ExternalApis\UnifiedCRT\lib\amd64

C:\Temp>type meow.cpp
#include <iostream>
int main() {
#ifdef __clang__
    std::cout << "I am Clang: " << __clang_version__ << "\n";
#elif defined(_MSC_VER)
    std::cout << "I am C1XX: " << _MSC_FULL_VER << "\n";
#else
    std::cout << "I am something else.\n";
#endif
    std::cout << "And I am " << sizeof(void *) * 8 << "-bit.\n";
}

C:\Temp>where cl
C:\Temp\binaries\amd64ret\bin\amd64\cl.exe

C:\Temp>where link
C:\Temp\binaries\amd64ret\bin\amd64\link.exe

C:\Temp>cl /EHsc /nologo /W4 /MT meow.cpp && meow
meow.cpp
I am C1XX: 191125506
And I am 64-bit.

C:\Temp>where clang-cl
C:\Temp\ClangLLVM\bin\clang-cl.exe

C:\Temp>clang-cl -v -m64 /EHsc /nologo /W4 /MT meow.cpp && meow
clang version 5.0.0 (tags/RELEASE_500/rc2)
Target: x86_64-pc-windows-msvc
Thread model: posix
InstalledDir: C:\Temp\ClangLLVM\bin
clang-cl.exe: warning: unable to find a Visual Studio installation; try running Clang from a developer command prompt [-Wmsvc-not-found]
"C:\\Temp\\ClangLLVM\\bin\\clang-cl.exe" -cc1 -triple x86_64-pc-windows-msvc18.0.0 -emit-obj -mrelax-all -mincremental-linker-compatible -disable-free -main-file-name meow.cpp -mrelocation-model pic -pic-level 2 -mthread-model posix -relaxed-aliasing -fmath-errno -masm-verbose -mconstructor-aliases -munwind-tables -target-cpu x86-64 -D_MT -flto-visibility-public-std --dependent-lib=libcmt --dependent-lib=oldnames -stack-protector 2 -fcxx-exceptions -fexceptions -fexternc-nounwind -fms-volatile -fdiagnostics-format msvc -momit-leaf-frame-pointer -v -dwarf-column-info -debugger-tuning=gdb -resource-dir "C:\\Temp\\ClangLLVM\\lib\\clang\\5.0.0" -internal-isystem "C:\\Temp\\ClangLLVM\\lib\\clang\\5.0.0\\include" -internal-isystem "C:\\Temp\\binaries\\amd64ret\\inc" -internal-isystem "C:\\Temp\\ExternalApis\\Windows\\10\\sdk\\inc" -internal-isystem "C:\\Temp\\ExternalApis\\UnifiedCRT\\inc" -WCL4 -fdeprecated-macro -fdebug-compilation-dir "C:\\Temp" -ferror-limit 19 -fmessage-length 120 -fms-extensions -fms-compatibility -fms-compatibility-version=18 -std=c++11 -fno-threadsafe-statics -fdelayed-template-parsing -fobjc-runtime=gcc -fdiagnostics-show-option -fcolor-diagnostics -o "C:\\Users\\stl\\AppData\\Local\\Temp\\2\\meow-0e7e4d.obj" -x c++ meow.cpp
clang -cc1 version 5.0.0 based upon LLVM 5.0.0-rc2 default target x86_64-pc-windows-msvc
#include "..." search starts here:
#include <...> search starts here:
C:\Temp\ClangLLVM\lib\clang\5.0.0\include
C:\Temp\binaries\amd64ret\inc
C:\Temp\ExternalApis\Windows\10\sdk\inc
C:\Temp\ExternalApis\UnifiedCRT\inc
End of search list.
In file included from meow.cpp:1:
In file included from C:\Temp\binaries\amd64ret\inc\iostream:6:
In file included from C:\Temp\binaries\amd64ret\inc\istream:6:
In file included from C:\Temp\binaries\amd64ret\inc\ostream:6:
In file included from C:\Temp\binaries\amd64ret\inc\ios:6:
In file included from C:\Temp\binaries\amd64ret\inc\xlocnum:7:
In file included from C:\Temp\binaries\amd64ret\inc\cmath:617:
In file included from C:\Temp\binaries\amd64ret\inc\xtgmath.h:9:
C:\Temp\binaries\amd64ret\inc\xtr1common(204,22) : error: use of undeclared identifier 'char16_t'
        struct _Is_integral<char16_t>
                            ^
[...more errors...]

C:\Temp>clang-cl -v -m64 /EHsc /nologo /W4 /MT -fms-compatibility-version=19.11.25506 meow.cpp && meow
clang version 5.0.0 (tags/RELEASE_500/rc2)
Target: x86_64-pc-windows-msvc
Thread model: posix
InstalledDir: C:\Temp\ClangLLVM\bin
clang-cl.exe: warning: unable to find a Visual Studio installation; try running Clang from a developer command prompt [-Wmsvc-not-found]
"C:\\Temp\\ClangLLVM\\bin\\clang-cl.exe" -cc1 -triple x86_64-pc-windows-msvc19.11.25506 -emit-obj -mrelax-all -mincremental-linker-compatible -disable-free -main-file-name meow.cpp -mrelocation-model pic -pic-level 2 -mthread-model posix -relaxed-aliasing -fmath-errno -masm-verbose -mconstructor-aliases -munwind-tables -target-cpu x86-64 -D_MT -flto-visibility-public-std --dependent-lib=libcmt --dependent-lib=oldnames -stack-protector 2 -fcxx-exceptions -fexceptions -fexternc-nounwind -fms-volatile -fdiagnostics-format msvc -momit-leaf-frame-pointer -v -dwarf-column-info -debugger-tuning=gdb -resource-dir "C:\\Temp\\ClangLLVM\\lib\\clang\\5.0.0" -internal-isystem "C:\\Temp\\ClangLLVM\\lib\\clang\\5.0.0\\include" -internal-isystem "C:\\Temp\\binaries\\amd64ret\\inc" -internal-isystem "C:\\Temp\\ExternalApis\\Windows\\10\\sdk\\inc" -internal-isystem "C:\\Temp\\ExternalApis\\UnifiedCRT\\inc" -WCL4 -fdeprecated-macro -fdebug-compilation-dir "C:\\Temp" -ferror-limit 19 -fmessage-length 120 -fms-extensions -fms-compatibility -fms-compatibility-version=19.11.25506 -std=c++14 -fdelayed-template-parsing -fobjc-runtime=gcc -fdiagnostics-show-option -fcolor-diagnostics -o "C:\\Users\\stl\\AppData\\Local\\Temp\\2\\meow-a284f9.obj" -x c++ meow.cpp
clang -cc1 version 5.0.0 based upon LLVM 5.0.0-rc2 default target x86_64-pc-windows-msvc
#include "..." search starts here:
#include <...> search starts here:
C:\Temp\ClangLLVM\lib\clang\5.0.0\include
C:\Temp\binaries\amd64ret\inc
C:\Temp\ExternalApis\Windows\10\sdk\inc
C:\Temp\ExternalApis\UnifiedCRT\inc
End of search list.
"link.exe" -out:meow.exe -nologo "C:\\Users\\stl\\AppData\\Local\\Temp\\2\\meow-a284f9.obj"
clang-cl.exe: error: unable to execute command: program not executable
clang-cl.exe: error: linker command failed with exit code 1 (use -v to see invocation)

C:\Temp>clang-cl -v -m64 /EHsc /nologo /W4 /MT -fms-compatibility-version=19.11.25506 -fuse-ld=lld-link meow.cpp && meow
clang version 5.0.0 (tags/RELEASE_500/rc2)
Target: x86_64-pc-windows-msvc
Thread model: posix
InstalledDir: C:\Temp\ClangLLVM\bin
clang-cl.exe: warning: unable to find a Visual Studio installation; try running Clang from a developer command prompt [-Wmsvc-not-found]
"C:\\Temp\\ClangLLVM\\bin\\clang-cl.exe" -cc1 -triple x86_64-pc-windows-msvc19.11.25506 -emit-obj -mrelax-all -mincremental-linker-compatible -disable-free -main-file-name meow.cpp -mrelocation-model pic -pic-level 2 -mthread-model posix -relaxed-aliasing -fmath-errno -masm-verbose -mconstructor-aliases -munwind-tables -target-cpu x86-64 -D_MT -flto-visibility-public-std --dependent-lib=libcmt --dependent-lib=oldnames -stack-protector 2 -fcxx-exceptions -fexceptions -fexternc-nounwind -fms-volatile -fdiagnostics-format msvc -momit-leaf-frame-pointer -v -dwarf-column-info -debugger-tuning=gdb -resource-dir "C:\\Temp\\ClangLLVM\\lib\\clang\\5.0.0" -internal-isystem "C:\\Temp\\ClangLLVM\\lib\\clang\\5.0.0\\include" -internal-isystem "C:\\Temp\\binaries\\amd64ret\\inc" -internal-isystem "C:\\Temp\\ExternalApis\\Windows\\10\\sdk\\inc" -internal-isystem "C:\\Temp\\ExternalApis\\UnifiedCRT\\inc" -WCL4 -fdeprecated-macro -fdebug-compilation-dir "C:\\Temp" -ferror-limit 19 -fmessage-length 120 -fms-extensions -fms-compatibility -fms-compatibility-version=19.11.25506 -std=c++14 -fdelayed-template-parsing -fobjc-runtime=gcc -fdiagnostics-show-option -fcolor-diagnostics -o "C:\\Users\\stl\\AppData\\Local\\Temp\\2\\meow-0b315e.obj" -x c++ meow.cpp
clang -cc1 version 5.0.0 based upon LLVM 5.0.0-rc2 default target x86_64-pc-windows-msvc
#include "..." search starts here:
#include <...> search starts here:
C:\Temp\ClangLLVM\lib\clang\5.0.0\include
C:\Temp\binaries\amd64ret\inc
C:\Temp\ExternalApis\Windows\10\sdk\inc
C:\Temp\ExternalApis\UnifiedCRT\inc
End of search list.
"C:\\Temp\\ClangLLVM\\bin\\lld-link" -out:meow.exe -nologo "C:\\Users\\stl\\AppData\\Local\\Temp\\2\\meow-0b315e.obj"
I am Clang: 5.0.0 (tags/RELEASE_500/rc2)
And I am 64-bit.

Hope this helps,
STL

15

Currently, for pre-2017, the root directory must be in the form /VC.

Could you try renaming x86ret and amd64ret to VC and try it again? e.g.:

C:\Temp\binaries\x86ret\bin\i386 ==> C:\Temp\binaries\VC\bin\i386

C:\Temp\binaries\amd64ret\bin\amd64 ==> C:\Temp\binaries\VC\bin\amd64

16

[STL]

Here's how to repro my directory structure in a clean VM.

[don hinton]

Currently, for pre-2017, the root directory must be in the form <some path>/VC.

Note that these development builds (both my actual ones, and the repro constructed from the nuget package) are 2017-class. They just don't conform to 2017's normal installer.

Could you try renaming x86ret and amd64ret to VC and try it again? e.g.:
C:\Temp\binaries\x86ret\bin\i386 ==> C:\Temp\binaries\VC\bin\i386
C:\Temp\binaries\amd64ret\bin\amd64 ==> C:\Temp\binaries\VC\bin\amd64

Sure. x86 still misbehaves, but x64 works properly! Does this indicate that Clang just needs to do slightly less strict validation of the root directory (possibly controlled by an option)?

*** x86: ***

C:\Temp>echo %PATH%
C:\Temp\binaries\VC\bin\i386;C:\Temp\ClangLLVM\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit\;C:\Users\stl\AppData\Local\Microsoft\WindowsApps;

C:\Temp>echo %INCLUDE%
C:\Temp\binaries\VC\inc;C:\Temp\ExternalApis\Windows\10\sdk\inc;C:\Temp\ExternalApis\UnifiedCRT\inc

C:\Temp>echo %LIB%
C:\Temp\binaries\VC\lib\i386;C:\Temp\ExternalApis\Windows\10\sdk\lib\x86;C:\Temp\ExternalApis\UnifiedCRT\lib\i386

C:\Temp>type meow.cpp
#include <iostream>
int main() {
#ifdef __clang__
    std::cout << "I am Clang: " << __clang_version__ << "\n";
#elif defined(_MSC_VER)
    std::cout << "I am C1XX: " << _MSC_FULL_VER << "\n";
#else
    std::cout << "I am something else.\n";
#endif
    std::cout << "And I am " << sizeof(void *) * 8 << "-bit.\n";
}

C:\Temp>where cl
C:\Temp\binaries\VC\bin\i386\cl.exe

C:\Temp>where link
C:\Temp\binaries\VC\bin\i386\link.exe

C:\Temp>cl /EHsc /nologo /W4 /MT meow.cpp && meow
meow.cpp
I am C1XX: 191125506
And I am 32-bit.

C:\Temp>where clang-cl
C:\Temp\ClangLLVM\bin\clang-cl.exe

C:\Temp>clang-cl -v -m32 /EHsc /nologo /W4 /MT meow.cpp && meow
clang version 5.0.0 (tags/RELEASE_500/rc2)
Target: i386-pc-windows-msvc
Thread model: posix
InstalledDir: C:\Temp\ClangLLVM\bin
"C:\\Temp\\ClangLLVM\\bin\\clang-cl.exe" -cc1 -triple i386-pc-windows-msvc18.0.0 -emit-obj -mrelax-all -mincremental-linker-compatible -disable-free -main-file-name meow.cpp -mrelocation-model static -mthread-model posix -mdisable-fp-elim -relaxed-aliasing -fmath-errno -masm-verbose -mconstructor-aliases -target-cpu pentium4 -D_MT -flto-visibility-public-std --dependent-lib=libcmt --dependent-lib=oldnames -stack-protector 2 -fcxx-exceptions -fexceptions -fexternc-nounwind -fms-volatile -fdiagnostics-format msvc -v -dwarf-column-info -debugger-tuning=gdb -resource-dir "C:\\Temp\\ClangLLVM\\lib\\clang\\5.0.0" -internal-isystem "C:\\Temp\\ClangLLVM\\lib\\clang\\5.0.0\\include" -internal-isystem "C:\\Temp\\binaries\\VC\\inc" -internal-isystem "C:\\Temp\\ExternalApis\\Windows\\10\\sdk\\inc" -internal-isystem "C:\\Temp\\ExternalApis\\UnifiedCRT\\inc" -WCL4 -fdeprecated-macro -fdebug-compilation-dir "C:\\Temp" -ferror-limit 19 -fmessage-length 120 -fms-extensions -fms-compatibility -fms-compatibility-version=18 -std=c++11 -fno-threadsafe-statics -fdelayed-template-parsing -fobjc-runtime=gcc -fdiagnostics-show-option -fcolor-diagnostics -o "C:\\Users\\stl\\AppData\\Local\\Temp\\2\\meow-514a85.obj" -x c++ meow.cpp
clang -cc1 version 5.0.0 based upon LLVM 5.0.0-rc2 default target x86_64-pc-windows-msvc
#include "..." search starts here:
#include <...> search starts here:
C:\Temp\ClangLLVM\lib\clang\5.0.0\include
C:\Temp\binaries\VC\inc
C:\Temp\ExternalApis\Windows\10\sdk\inc
C:\Temp\ExternalApis\UnifiedCRT\inc
End of search list.
In file included from meow.cpp:1:
In file included from C:\Temp\binaries\VC\inc\iostream:6:
In file included from C:\Temp\binaries\VC\inc\istream:6:
In file included from C:\Temp\binaries\VC\inc\ostream:6:
In file included from C:\Temp\binaries\VC\inc\ios:6:
In file included from C:\Temp\binaries\VC\inc\xlocnum:7:
In file included from C:\Temp\binaries\VC\inc\cmath:617:
In file included from C:\Temp\binaries\VC\inc\xtgmath.h:9:
C:\Temp\binaries\VC\inc\xtr1common(204,22) : error: use of undeclared identifier 'char16_t'
        struct _Is_integral<char16_t>
                            ^
[...more errors...]

C:\Temp>clang-cl -v -m32 /EHsc /nologo /W4 /MT -fms-compatibility-version=19.11.25506 meow.cpp && meow
clang version 5.0.0 (tags/RELEASE_500/rc2)
Target: i386-pc-windows-msvc
Thread model: posix
InstalledDir: C:\Temp\ClangLLVM\bin
"C:\\Temp\\ClangLLVM\\bin\\clang-cl.exe" -cc1 -triple i386-pc-windows-msvc19.11.25506 -emit-obj -mrelax-all -mincremental-linker-compatible -disable-free -main-file-name meow.cpp -mrelocation-model static -mthread-model posix -mdisable-fp-elim -relaxed-aliasing -fmath-errno -masm-verbose -mconstructor-aliases -target-cpu pentium4 -D_MT -flto-visibility-public-std --dependent-lib=libcmt --dependent-lib=oldnames -stack-protector 2 -fcxx-exceptions -fexceptions -fexternc-nounwind -fms-volatile -fdiagnostics-format msvc -v -dwarf-column-info -debugger-tuning=gdb -resource-dir "C:\\Temp\\ClangLLVM\\lib\\clang\\5.0.0" -internal-isystem "C:\\Temp\\ClangLLVM\\lib\\clang\\5.0.0\\include" -internal-isystem "C:\\Temp\\binaries\\VC\\inc" -internal-isystem "C:\\Temp\\ExternalApis\\Windows\\10\\sdk\\inc" -internal-isystem "C:\\Temp\\ExternalApis\\UnifiedCRT\\inc" -WCL4 -fdeprecated-macro -fdebug-compilation-dir "C:\\Temp" -ferror-limit 19 -fmessage-length 120 -fms-extensions -fms-compatibility -fms-compatibility-version=19.11.25506 -std=c++14 -fdelayed-template-parsing -fobjc-runtime=gcc -fdiagnostics-show-option -fcolor-diagnostics -o "C:\\Users\\stl\\AppData\\Local\\Temp\\2\\meow-a07665.obj" -x c++ meow.cpp
clang -cc1 version 5.0.0 based upon LLVM 5.0.0-rc2 default target x86_64-pc-windows-msvc
#include "..." search starts here:
#include <...> search starts here:
C:\Temp\ClangLLVM\lib\clang\5.0.0\include
C:\Temp\binaries\VC\inc
C:\Temp\ExternalApis\Windows\10\sdk\inc
C:\Temp\ExternalApis\UnifiedCRT\inc
End of search list.
"link.exe" -out:meow.exe -nologo "C:\\Users\\stl\\AppData\\Local\\Temp\\2\\meow-a07665.obj"
clang-cl.exe: error: unable to execute command: program not executable
clang-cl.exe: error: linker command failed with exit code 1 (use -v to see invocation)

C:\Temp>clang-cl -v -m32 /EHsc /nologo /W4 /MT -fms-compatibility-version=19.11.25506 -fuse-ld=lld-link meow.cpp && meow
clang version 5.0.0 (tags/RELEASE_500/rc2)
Target: i386-pc-windows-msvc
Thread model: posix
InstalledDir: C:\Temp\ClangLLVM\bin
"C:\\Temp\\ClangLLVM\\bin\\clang-cl.exe" -cc1 -triple i386-pc-windows-msvc19.11.25506 -emit-obj -mrelax-all -mincremental-linker-compatible -disable-free -main-file-name meow.cpp -mrelocation-model static -mthread-model posix -mdisable-fp-elim -relaxed-aliasing -fmath-errno -masm-verbose -mconstructor-aliases -target-cpu pentium4 -D_MT -flto-visibility-public-std --dependent-lib=libcmt --dependent-lib=oldnames -stack-protector 2 -fcxx-exceptions -fexceptions -fexternc-nounwind -fms-volatile -fdiagnostics-format msvc -v -dwarf-column-info -debugger-tuning=gdb -resource-dir "C:\\Temp\\ClangLLVM\\lib\\clang\\5.0.0" -internal-isystem "C:\\Temp\\ClangLLVM\\lib\\clang\\5.0.0\\include" -internal-isystem "C:\\Temp\\binaries\\VC\\inc" -internal-isystem "C:\\Temp\\ExternalApis\\Windows\\10\\sdk\\inc" -internal-isystem "C:\\Temp\\ExternalApis\\UnifiedCRT\\inc" -WCL4 -fdeprecated-macro -fdebug-compilation-dir "C:\\Temp" -ferror-limit 19 -fmessage-length 120 -fms-extensions -fms-compatibility -fms-compatibility-version=19.11.25506 -std=c++14 -fdelayed-template-parsing -fobjc-runtime=gcc -fdiagnostics-show-option -fcolor-diagnostics -o "C:\\Users\\stl\\AppData\\Local\\Temp\\2\\meow-17ad61.obj" -x c++ meow.cpp
clang -cc1 version 5.0.0 based upon LLVM 5.0.0-rc2 default target x86_64-pc-windows-msvc
#include "..." search starts here:
#include <...> search starts here:
C:\Temp\ClangLLVM\lib\clang\5.0.0\include
C:\Temp\binaries\VC\inc
C:\Temp\ExternalApis\Windows\10\sdk\inc
C:\Temp\ExternalApis\UnifiedCRT\inc
End of search list.
"C:\\Temp\\ClangLLVM\\bin\\lld-link" -out:meow.exe -nologo "C:\\Users\\stl\\AppData\\Local\\Temp\\2\\meow-17ad61.obj"
I am Clang: 5.0.0 (tags/RELEASE_500/rc2)
And I am 32-bit.

*** x64: ***

C:\Temp>echo %PATH%
C:\Temp\binaries\VC\bin\amd64;C:\Temp\ClangLLVM\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit\;C:\Users\stl\AppData\Local\Microsoft\WindowsApps;

C:\Temp>echo %INCLUDE%
C:\Temp\binaries\VC\inc;C:\Temp\ExternalApis\Windows\10\sdk\inc;C:\Temp\ExternalApis\UnifiedCRT\inc

C:\Temp>echo %LIB%
C:\Temp\binaries\VC\lib\amd64;C:\Temp\ExternalApis\Windows\10\sdk\lib\x64;C:\Temp\ExternalApis\UnifiedCRT\lib\amd64

C:\Temp>type meow.cpp
#include <iostream>
int main() {
#ifdef __clang__
    std::cout << "I am Clang: " << __clang_version__ << "\n";
#elif defined(_MSC_VER)
    std::cout << "I am C1XX: " << _MSC_FULL_VER << "\n";
#else
    std::cout << "I am something else.\n";
#endif
    std::cout << "And I am " << sizeof(void *) * 8 << "-bit.\n";
}

C:\Temp>where cl
C:\Temp\binaries\VC\bin\amd64\cl.exe

C:\Temp>where link
C:\Temp\binaries\VC\bin\amd64\link.exe

C:\Temp>cl /EHsc /nologo /W4 /MT meow.cpp && meow
meow.cpp
I am C1XX: 191125506
And I am 64-bit.

C:\Temp>where clang-cl
C:\Temp\ClangLLVM\bin\clang-cl.exe

C:\Temp>clang-cl -v -m64 /EHsc /nologo /W4 /MT meow.cpp && meow
clang version 5.0.0 (tags/RELEASE_500/rc2)
Target: x86_64-pc-windows-msvc
Thread model: posix
InstalledDir: C:\Temp\ClangLLVM\bin
"C:\\Temp\\ClangLLVM\\bin\\clang-cl.exe" -cc1 -triple x86_64-pc-windows-msvc19.11.25506 -emit-obj -mrelax-all -mincremental-linker-compatible -disable-free -main-file-name meow.cpp -mrelocation-model pic -pic-level 2 -mthread-model posix -relaxed-aliasing -fmath-errno -masm-verbose -mconstructor-aliases -munwind-tables -target-cpu x86-64 -D_MT -flto-visibility-public-std --dependent-lib=libcmt --dependent-lib=oldnames -stack-protector 2 -fcxx-exceptions -fexceptions -fexternc-nounwind -fms-volatile -fdiagnostics-format msvc -momit-leaf-frame-pointer -v -dwarf-column-info -debugger-tuning=gdb -resource-dir "C:\\Temp\\ClangLLVM\\lib\\clang\\5.0.0" -internal-isystem "C:\\Temp\\ClangLLVM\\lib\\clang\\5.0.0\\include" -internal-isystem "C:\\Temp\\binaries\\VC\\inc" -internal-isystem "C:\\Temp\\ExternalApis\\Windows\\10\\sdk\\inc" -internal-isystem "C:\\Temp\\ExternalApis\\UnifiedCRT\\inc" -WCL4 -fdeprecated-macro -fdebug-compilation-dir "C:\\Temp" -ferror-limit 19 -fmessage-length 120 -fms-extensions -fms-compatibility -fms-compatibility-version=19.11.25506 -std=c++14 -fdelayed-template-parsing -fobjc-runtime=gcc -fdiagnostics-show-option -fcolor-diagnostics -o "C:\\Users\\stl\\AppData\\Local\\Temp\\2\\meow-fcb42a.obj" -x c++ meow.cpp
clang -cc1 version 5.0.0 based upon LLVM 5.0.0-rc2 default target x86_64-pc-windows-msvc
#include "..." search starts here:
#include <...> search starts here:
C:\Temp\ClangLLVM\lib\clang\5.0.0\include
C:\Temp\binaries\VC\inc
C:\Temp\ExternalApis\Windows\10\sdk\inc
C:\Temp\ExternalApis\UnifiedCRT\inc
End of search list.
"C:\\Temp\\binaries\\VC\\bin\\amd64\\link.exe" -out:meow.exe -nologo "C:\\Users\\stl\\AppData\\Local\\Temp\\2\\meow-fcb42a.obj"
I am Clang: 5.0.0 (tags/RELEASE_500/rc2)
And I am 64-bit.

C:\Temp>clang-cl -v -m64 /EHsc /nologo /W4 /MT -fms-compatibility-version=19.11.25506 meow.cpp && meow
clang version 5.0.0 (tags/RELEASE_500/rc2)
Target: x86_64-pc-windows-msvc
Thread model: posix
InstalledDir: C:\Temp\ClangLLVM\bin
"C:\\Temp\\ClangLLVM\\bin\\clang-cl.exe" -cc1 -triple x86_64-pc-windows-msvc19.11.25506 -emit-obj -mrelax-all -mincremental-linker-compatible -disable-free -main-file-name meow.cpp -mrelocation-model pic -pic-level 2 -mthread-model posix -relaxed-aliasing -fmath-errno -masm-verbose -mconstructor-aliases -munwind-tables -target-cpu x86-64 -D_MT -flto-visibility-public-std --dependent-lib=libcmt --dependent-lib=oldnames -stack-protector 2 -fcxx-exceptions -fexceptions -fexternc-nounwind -fms-volatile -fdiagnostics-format msvc -momit-leaf-frame-pointer -v -dwarf-column-info -debugger-tuning=gdb -resource-dir "C:\\Temp\\ClangLLVM\\lib\\clang\\5.0.0" -internal-isystem "C:\\Temp\\ClangLLVM\\lib\\clang\\5.0.0\\include" -internal-isystem "C:\\Temp\\binaries\\VC\\inc" -internal-isystem "C:\\Temp\\ExternalApis\\Windows\\10\\sdk\\inc" -internal-isystem "C:\\Temp\\ExternalApis\\UnifiedCRT\\inc" -WCL4 -fdeprecated-macro -fdebug-compilation-dir "C:\\Temp" -ferror-limit 19 -fmessage-length 120 -fms-extensions -fms-compatibility -fms-compatibility-version=19.11.25506 -std=c++14 -fdelayed-template-parsing -fobjc-runtime=gcc -fdiagnostics-show-option -fcolor-diagnostics -o "C:\\Users\\stl\\AppData\\Local\\Temp\\2\\meow-61735d.obj" -x c++ meow.cpp
clang -cc1 version 5.0.0 based upon LLVM 5.0.0-rc2 default target x86_64-pc-windows-msvc
#include "..." search starts here:
#include <...> search starts here:
C:\Temp\ClangLLVM\lib\clang\5.0.0\include
C:\Temp\binaries\VC\inc
C:\Temp\ExternalApis\Windows\10\sdk\inc
C:\Temp\ExternalApis\UnifiedCRT\inc
End of search list.
"C:\\Temp\\binaries\\VC\\bin\\amd64\\link.exe" -out:meow.exe -nologo "C:\\Users\\stl\\AppData\\Local\\Temp\\2\\meow-61735d.obj"
I am Clang: 5.0.0 (tags/RELEASE_500/rc2)
And I am 64-bit.

C:\Temp>clang-cl -v -m64 /EHsc /nologo /W4 /MT -fms-compatibility-version=19.11.25506 -fuse-ld=lld-link meow.cpp && meow
clang version 5.0.0 (tags/RELEASE_500/rc2)
Target: x86_64-pc-windows-msvc
Thread model: posix
InstalledDir: C:\Temp\ClangLLVM\bin
"C:\\Temp\\ClangLLVM\\bin\\clang-cl.exe" -cc1 -triple x86_64-pc-windows-msvc19.11.25506 -emit-obj -mrelax-all -mincremental-linker-compatible -disable-free -main-file-name meow.cpp -mrelocation-model pic -pic-level 2 -mthread-model posix -relaxed-aliasing -fmath-errno -masm-verbose -mconstructor-aliases -munwind-tables -target-cpu x86-64 -D_MT -flto-visibility-public-std --dependent-lib=libcmt --dependent-lib=oldnames -stack-protector 2 -fcxx-exceptions -fexceptions -fexternc-nounwind -fms-volatile -fdiagnostics-format msvc -momit-leaf-frame-pointer -v -dwarf-column-info -debugger-tuning=gdb -resource-dir "C:\\Temp\\ClangLLVM\\lib\\clang\\5.0.0" -internal-isystem "C:\\Temp\\ClangLLVM\\lib\\clang\\5.0.0\\include" -internal-isystem "C:\\Temp\\binaries\\VC\\inc" -internal-isystem "C:\\Temp\\ExternalApis\\Windows\\10\\sdk\\inc" -internal-isystem "C:\\Temp\\ExternalApis\\UnifiedCRT\\inc" -WCL4 -fdeprecated-macro -fdebug-compilation-dir "C:\\Temp" -ferror-limit 19 -fmessage-length 120 -fms-extensions -fms-compatibility -fms-compatibility-version=19.11.25506 -std=c++14 -fdelayed-template-parsing -fobjc-runtime=gcc -fdiagnostics-show-option -fcolor-diagnostics -o "C:\\Users\\stl\\AppData\\Local\\Temp\\2\\meow-607867.obj" -x c++ meow.cpp
clang -cc1 version 5.0.0 based upon LLVM 5.0.0-rc2 default target x86_64-pc-windows-msvc
#include "..." search starts here:
#include <...> search starts here:
C:\Temp\ClangLLVM\lib\clang\5.0.0\include
C:\Temp\binaries\VC\inc
C:\Temp\ExternalApis\Windows\10\sdk\inc
C:\Temp\ExternalApis\UnifiedCRT\inc
End of search list.
"C:\\Temp\\ClangLLVM\\bin\\lld-link" -out:meow.exe -nologo "C:\\Users\\stl\\AppData\\Local\\Temp\\2\\meow-607867.obj"
I am Clang: 5.0.0 (tags/RELEASE_500/rc2)
And I am 64-bit.

Thanks,
STL

17

I'm in favor of this. If we find a cl.exe on PATH, we can take its version
and use that to set our compatibility mode flags. We don't need to validate
that it looks like its in a VC installation tree, and I'd rather not have
an option for it. Whatever we use the knowledge of the VC tree root for
would probably not work though. We won't be able to find and prefer the
x64/x86 cross-linker, for example.

18

[STL]
> Here's how to repro my directory structure in a clean VM.

[don hinton]
> Currently, for pre-2017, the root directory must be in the form <some
>/VC.

Note that these development builds (both my actual ones, and the repro
constructed from the nuget package) are 2017-class. They just don't conform
to 2017's normal installer.

> Could you try renaming x86ret and amd64ret to VC and try it again? e.g.:
> C:\Temp\binaries\x86ret\bin\i386 ==> C:\Temp\binaries\VC\bin\i386
> C:\Temp\binaries\amd64ret\bin\amd64 ==> C:\Temp\binaries\VC\bin\a
md64

Sure. x86 still misbehaves, but x64 works properly! Does this indicate
that Clang just needs to do slightly less strict validation of the root
directory (possibly controlled by an option)?

I'm in favor of this. If we find a cl.exe on PATH, we can take its version
and use that to set our compatibility mode flags. We don't need to validate
that it looks like its in a VC installation tree, and I'd rather not have
an option for it. Whatever we use the knowledge of the VC tree root for
would probably not work though. We won't be able to find and prefer the
x64/x86 cross-linker, for example.

Isn't "cl.exe on path" the common case for people running vcvarsall?
Wouldn't we want to validate in that case? (If not, why do it ever?)

19

The vcvars* / VsDevCmd.bat scripts all set VCINSTALLDIR which
short-circuits all that validation logic.

20

[Reid Kleckner]

The vcvars* / VsDevCmd.bat scripts all set VCINSTALLDIR which short-circuits all that validation logic.

I managed to build and debug into Clang, so I have a better idea of what it's doing now. While I could easily set an environment variable like VCINSTALLDIR, there are problems beyond validation.

findVCToolChainViaEnvironment() indeed finds cl.exe and link.exe on the path but performs too much validation. If I skip the validation (by jumping into the `if (llvm::sys::path::filename(ParentPath) == "VC")` branch which sets Path = ParentPath; and IsVS2017OrNewer = false; before returning - I know that's an inaccurate value for IsVS2017OrNewer but I wanted to see how far I could get), I think I've found an additional part of the problem.

MSVCToolChain::computeMSVCVersion() calls MSVCToolChain::getSubDirectoryPath() which has directory structure assumptions that don't apply to this development build. If I get here with VCToolChainPath set to "C:\\Temp\\binaries\\x86ret", it executes this code:

  case SubDirectoryType::Bin:
    if (IsVS2017OrNewer) {
      bool HostIsX64 =
          llvm::Triple(llvm::sys::getProcessTriple()).isArch64Bit();
      llvm::sys::path::append(Path, "bin", (HostIsX64 ? "HostX64" : "HostX86"),
                              llvmArchToWindowsSDKArch(TargetArch));

    } else {
      llvm::sys::path::append(Path, "bin", llvmArchToLegacyVCArch(TargetArch));
    }

The IsVS2017OrNewer codepath is not applicable to the dev build. The !IsVS2017OrNewer codepath is ALMOST applicable, but here the TargetArch is x86. This leads us to:

static const char *llvmArchToLegacyVCArch(llvm::Triple::ArchType Arch) {
  using ArchType = llvm::Triple::ArchType;
  switch (Arch) {
  case ArchType::x86:
    // x86 is default in legacy VC toolchains.
    // e.g. x86 libs are directly in /lib as opposed to /lib/x86.
    return "";
  case ArchType::x86_64:
    return "amd64";
  case ArchType::arm:
    return "arm";
  default:
    return "";
  }
}

So this emits the empty string. However, in the dev build, the compiler is located at C:\Temp\binaries\x86ret\bin\i386\cl.exe , i.e. it lives in an i386 subdirectory.

(Yes, I am endlessly screaming at all of:
* The varying names for x86/i386 and x64/amd64
* The varying directory structures between dev, pre-2017, and 2017+
* The continued persistence of 32-bit ANYTHING in this day and age)

I believe that this is why when Don Hinton asked me "Could you try renaming x86ret and amd64ret to VC and try it again?", the x86 dev build still failed but the amd64 dev build began working, because the compiler's path C:\Temp\binaries\amd64ret\bin\amd64\cl.exe lives in an "amd64" subdirectory like llvmArchToLegacyVCArch() expects.

So beyond the too-strict validation, the problem here seems to be that Clang is finding cl.exe on the path, but then attempting to regenerate that information, and the regeneration isn't lossless. (findVCToolChainViaEnvironment() accidentally handles the dev build's i386 subdirectory due to "// Strip any architecture subdir like "amd64"." logic.)

I am not sure how to fix any of this (I am amazed that I was able to get this far), but hopefully this info will help.

Thanks,
STL