Hi all,
This is my first post and my question is:
What is the state of affairs regarding flang-new vs ucrt?
Currently when i say flang-new -c source.f90, the compiler inserts /DEFAULTLIB: directives for the (to me obsolete) legacy runtimes (libcmt, msvcrt …) into the object file.
I can see this with
dumpbin -rawdata -section:.drectve source.o
I want to use the newer ucrt instead (available and standard/default since VS 2015).
This is with llvm version 18.1.8, i built flang-new (i.e. llvm) on a Windows 11 22H2 computer with VS 2022 17.12.1 (with Microsoft llvm of the exact same version 18.1.8).
All exeS and dllS of the newly built and installed llvm are linked against ucrt and not msvcrt & co.
Is flang-new special in this regard?
Is ist possible to do flang-new compiles without these legacy /DEFAULTLIB: directives (and how)?
Hi, thanks for bringing this to my attention.
I added these directives and I think I may have misunderstood the situation on Windows, as I thought we were already linking to the ucrt, especially as that seems to be available from Windows 10 onwards and that’s the earliest version we make any claim to support.
At a minimum we should add a flag to use the new libraries instead, but it may be better to just switch to linking to the ucrt libs, especially if that is what clang is doing already.
I just had a look on my own system, using a new build of flang from LLVM main. If I use -fms-runtime-lib=dll (to link against the dynamic runtimes, so I can see what is being pulled in) and then dumpbin /dependents on the exe, I see the following:
Dump of file .\a.exe
File Type: EXECUTABLE IMAGE
Image has the following dependencies:
KERNEL32.dll
VCRUNTIME140.dll
api-ms-win-crt-runtime-l1-1-0.dll
api-ms-win-crt-math-l1-1-0.dll
api-ms-win-crt-stdio-l1-1-0.dll
api-ms-win-crt-locale-l1-1-0.dll
api-ms-win-crt-heap-l1-1-0.dll
api-ms-win-crt-string-l1-1-0.dll
api-ms-win-crt-environment-l1-1-0.dll
api-ms-win-crt-convert-l1-1-0.dll
MSVCP140.dll
api-ms-win-crt-filesystem-l1-1-0.dll
I think this is correctly linking to the ucrt? The api-ms-win-crt-* files are ucrt files, as far as I understand it.
I think the libcmt etc libraries only contain startup code now, and are still necessary for that, rather than pointing to the old versions of the runtimes, according to this page: C runtime (CRT) and C++ standard library (STL) lib files | Microsoft Learn
It’s possible my understanding here is still wrong though, please let me know if you think that’s the case.
I think there may be a misunderstanding here.
The library names libcmt and msvcrt are what current Clang and also MSVC encode in the /defaultlib: directives, when linking static and dynamically against the MS C runtime.
Those names are the same as they used to be with older MSVC versions, even if using the modern UCRT - probably kept intact to preserve compatibility with third party projects and tools that refer to them.
msvcrt.lib is a small library that mostly defers to linking against ucrt.lib which is the actual UCRT import library, but also links against vcruntime.lib and contains a bunch of other object files that are needed for initializing the C runtime. Same with libcmt.lib which defers to libucrt.lib and libvcruntime.lib. If you skip these and link directly against ucrt.lib or libucrt.lib, you’ll miss out on the other functions that are provided by vcruntime and the other glue code object files.
Thanks for the added context! This matches my understanding having just investigated it.
I should add that flang does use the same flags as clang does, and in fact that’s where I got the flags and lib names etc from in the first place when adding this support to flang.
Thank you and looking forward. I find suprisingly little (i.e. no) information on how this information (/DEFAULTLIB:) is inserted into the objects, is it known how the Microsoft tools decide between old (msvcrt …) and new (ucrt …)?
In theory this could be achieved with #ifdef and #pragma comment(lib, … but i do not find (enough) ocurrences of them in the Microsoft headers.
Do you have any links for this?
Thx again
Microsoft tools never insert /defaultlib:ucrt, only /defaultlib:msvcrt or /defaultlib:libcmt. Please reread what was said above - it still does use the modern UCRT, but the msvcrt.lib and libcmt.lib libraries are wrappers that contain the UCRT.
Thx for information.
So how as a (compiler) end user can i choose between old and new?
If /defaultlib:msvcrt or /defaultlib:libcmt pull in the new runtimes, which /defaultlibS would pull in the old ones?
A short explanation were i start from:
I want to build (on Windows) the mixed language project
using current numpy.f2py.
I am using a version 18.1.8 non release build of flang-new (flang-new of the release build bails out at the Fortran sources, i will create a separate discussion about this).
Now with the llvm linker i get a conflict
libucrt versus ucrt which i have to resolve with
set LDFLAGS=-Wl,-nodefaultlib:libucrt -Wl,-defaultlib:ucrtd
With the Microsoft linker i get addional warnings about conflicting msvcrt and msvcrtd.
So i wondered what is going on and stumbled upon the /defaultlib directives.
I understand now that the"old" names (can) mean the new runtimes.
What i do not completely understand:
- how to pull in the old runtimes?
- are the new runtimes hard wired into the (Microsoft) compiler/linker?
- why does the flang-new build sometimes insert references to oldnames.lib?
- where in my example deduces the Microsoft the need for msvcrt(d), where the llvm linker does not?
The Microsoft link line is
“link” /MACHINE:x64 /OUT:_symbolic_regress2.cp312-win_amd64.pyd _symbolic_regress2.cp312-win_amd64.pyd.p/symbolic_regress2.f90.obj _symbolic_regress2.cp312-win_amd64.pyd.p/_symbolic_regress2module.c.obj _symbolic_regress2.cp312-win_amd64.pyd.p/symbolic_regress2-f2pywrappers.f.obj symbolic_regress2.cp312-win_amd64.pyd.p/90a77c7a48d9ce6c4735d2094c92e701521af764……_f2py_src_fortranobject.c.obj “/release” “/nologo” “/OPT:REF” “/DLL” “/IMPLIB:_symbolic_regress2.cp312-win_amd64.lib” “C:\DEV\FEH01589\Apps\Python312\libs\python312.lib” “ucrtd.lib” “kernel32.lib” “user32.lib” “gdi32.lib” “winspool.lib” “shell32.lib” “ole32.lib” “oleaut32.lib” “uuid.lib” “comdlg32.lib” “advapi32.lib” “FortranRuntime.lib” “FortranDecimal.lib”
The flag for choosing between the release/debug/dll/static builds with flang is -fms-runtime-lib=..., you shouldn’t need to pass -nodefaultlib: or -defaultlib as linker options. E.g. to use the “dynamic debug” version (ucrtd) as your example above, you would pass -fms-runtime-lib=dll_dbg. Note that you need this on the compile line not the link line, so in something like FFLAGS= instead of LDFLAGS= (depending on build system). That should resolve your conflicting libs issue as described above.
The need for the libraries isn’t “deduced”, it’s embedded in the .obj file that is generated by the compiler, which is why you need the -fms-runtime-lib=dll_dbg.
Your -fms-runtime-lib=... will always need to match whatever flag you’re using to build the C libraries. The relevant option to match each cl.exe option is as follows:
/MT → -fms-runtime-lib=static
/MTd → -fms-runtime-lib=static_dbg
/MD → -fms-runtime-lib=dll
/MDd → -fms-runtime-lib=dll_dbg
If these don’t match for every object file, the link will fail at the end, and this is intended behaviour. The same is true if e.g. you’re only using cl.exe and some files are compiled with /MT and some with /MD.
As an example, lets say you had a a.c and b.f90 and you wanted to link these together using the “dynamic+debug” library configuration, and using cl.exe for the C file and flang for the f90 file. You could then do:
cl.exe /MDd /c a.c
flang.exe -fms-runtime-lib=dll_dbg -c -o b.obj b.f90
link.exe a.obj b.obj -libpath:<path to flang runtimes>
and things ought to work. Although I think you also need to manually link clang-rt.builtins.lib for now, because we aren’t adding a dependency on that to the object file from flang (b.obj in this example) when we probably should be, due to an ongoing change in the naming of the clang-rt libraries.
Thanks for this detailed and useful explanation.
So there is no normal way to link against the pre ucrt dlls?
And if i get warnings regarding conflicting mscvrt(d) dlls this means they are some transitive dependency?
And one additional question: Which Fortran compiler flag will prefer the shared Fortran libs over the static ones?
I believe which crt DLLs you’ll get linked against depends on which version of the MSVC toolchain you have installed. E.g. if you have any release VS2017 or after you’ll link against the ucrt dlls. If you have a release before that you’ll get linked against the specific dlls for that release as they used to change with each VS release.
This is irrelevant as far as flang is concerned though, as we use features in the runtimes that were not introduced until the VS2019 C++ library (which depends on the ucrt), so there will not be any way to link the flang runtimes to the old CRT libraries.
If you get warnings regarding the conflicting DLLs then yeah there must be some transitive dependency that wasn’t linked against the same thing.
I think generally most dynamic libraries will link with msvcrt (/MD,-fms-runtime-lib=dll) and static libraries with libcmt (/MT,-fms-runtime-lib=static) although this isn’t enforced.
I think prebuilt dependencies rarely use the debug versions (msvcrtd (/MDd, -fms-runtime-lib=dll_dbg) and libcmtd (/MTd,-fms-runtime-lib=static_dbg)).
This is just my understanding of the broad defaults though; I might be wrong about what people generally do here.
As regards the Fortran runtimes, it depends how your LLVM was built. If it was built with -DBUILD_SHARED_LIBS=Off (the default) you’ll get static versions of the runtimes. If it was built with -DBUILD_SHARED_LIBS=On you will get dynamic versions. In either case, we build the runtimes 4 times, once for each ucrt version, and we link the correct one depending on the -fms-runtime-lib=.... flag without needing another flag.
I realise this can be a bit confusing as e.g. Fortran_runtime.static.dll would be a dynamic linked Fortran runtime linked against the static ucrt, and conversely Fortran_runtime.dynamic.lib would be a statically linked Fortran runtime linked against the dynamic ucrt. But whether the Fortran runtime itself is provided as DLLs or static libs is orthogonal to whether the ucrt linked against is static or dynamic; we provide the runtimes for all 4 ucrt versions in any case.
I should add that some build systems (e.g. CMake) will do all this for you; e.g. cmake will add the correct -fms-runtime-lib=... and /M{D,T}.. options to all the compile lines based on the MSVC_RUNTIME_LIBRARY property: MSVC_RUNTIME_LIBRARY — CMake 3.31.1 Documentation. So a simple cmake like the following will work:
cmake_minimum_required(VERSION 3.28) # cmake 3.28+ required for flang support
project(link_example LANGUAGES C Fortran)
add_executable(example a.c b.f90)
cmake -GNinja -DCMAKE_MSVC_RUNTIME_LIBRARY=MultiThreaded -DCMAKE_C_COMPILER=cl -DCMAKE_Fortran_COMPILER=flang adds /MT to the cl.exe lines and -fms-runtime-lib=static to the flang.exe lines.
If you change to -DCMAKE_MSVC_RUNTIME_LIBRARY=MultiThreadedDebugDLL it adds -fms-runtime-lib=dll_dbg to the flang.exe lines and /MDd to the cl.exe lines.
It’ll also add the paths to the Fortran runtimes, and the other required LLVM libraries.
Again thanks a lot for your information 
This (-fms-runtime-lib / without linker flags) does not work out of the box for me, but that might well be due to the special python/f2py/ninja situation.
What compiles and links without errors and warnings is
F90=flang-new
CC=clang
CXX=clang++
LD=flang-new
FFLAGS=-fuse-ld=lld -fms-runtime-lib=dll
CFLAGS=-fuse-ld=lld -fms-runtime-lib=dll
CXXFLAGS=-fuse-ld=lld -fms-runtime-lib=dll
LDFLAGS=-Wl,ucrtd.lib
The final link is done by f2py/ninja with clang.
I have to list the runtime in LDFLAGS, otherwise i get bunches of unresolved externals.
I have to use the debug version ucrtd, because i built llvm with DEBUG and now the Fortran libraries contain debug symbols which cannot be resolved against ucrt.
If i use the _dbg variants, f2py and/or ninja silently map python311.lib to python311_d.lib, which does not exist, making the link fail (-> very spooky).
I can’t really help with what the python stuff is doing, I’m not familiar with that at all. I wouldn’t rely on mixing the debug and non debug versions of the ucrt though, that looks spooky to me. That looks like a bug in how we’re building the runtimes, as whether they have the debug symbols shouldn’t depend on how LLVM as a whole was built.
I know @h-vetinari has some experience mixing Python and flang, I wonder if they might be able to help.
The problem is, the flang Release build fails to compile the f90 sources in question (i opened a discussion about this), so i need the Debug build (with debug symbols in the Fortran libraries).
As soon as this is sorted out i can use ucrt throughout.
Regarding the implicit python debug renaming i have to talk to the f2py and/or ninja guys.
Today i had a fresh look at the _dbg situation and the python debug library is in fact there, it is in a separate directory (libs subdirectory) and therefore not found without a separate -L directive.
So this renaming (python → python_d) is in fact a good thing which leads to a consistent (debug) build/link.
The remaining noncritical miracles are
- why do i have to list ucrtd explicit in LDFLAGS?
- from where does the Microsoft linker pull in the legacy dlls?
The latter one i will investigate via -verbose:link -map:.
And the remaining problem is the non-working flang-new Release build (18.1.8, i did not try 19.1.4 yet).
In a consistent debug build i can in fact omit the LDFLAGS.