mlir::ModuleOp creation fails with MLIR on windows in exe + dll projects

I want to create a mlir::ModuleOp, and print it on windows, using MSVC.
I am using commit d7b669b3a30345cfcdb2fde2af6f48aa4b94845d of LLVM, compiled with MSVC, enabling MLIR subproject.

Project structure is the following:

CMakeLists.txt
exe.cpp
lib/CMakeLists.txt
lib/lib.cpp
lib/lib.h

I create the mlir::MLIRContext in the exe, then I call a function of my library that tries to create mlir::ModuleOp like so:

mlir::ModuleOp module = mlir::ModuleOp::create(builder.getUnknownLoc());

where builder has been initialized in lib constructor like so:

mlir::Builder builder(&context);

with the context created in the exe.

If then I try to print the module in a release build I have a crash.

I crash at module creation instead if using a debug build, because of a type mismatch of the dictionary of attributes of the context:

Assertion failed: isa<U>(), file C:\llvm-project\mlir\include\mlir/IR/Attributes.h, line 118

Indeed, with the debugger we can see that the attribute dictionary is of type exe.exe!mlir::TypeID::Storage, but it should be of type lib.dll!mlir::TypeID::Storage.

If I create the context and the module in the same exe/dll, module is printed with no issues both in release and debug builds.

Is this a bug of Windows build of MLIR or am I missing something obvious?

Below the exe+dll project for reference.

Content of CMakeLists.txt:

cmake_minimum_required(VERSION 3.21)
set(CMAKE_CXX_STANDARD 17)
project(mlirTest)

# LLVM
set_tests_properties(${noArgsTests} PROPERTIES TIMEOUT 10)
find_package(LLVM REQUIRED CONFIG)
list(APPEND CMAKE_MODULE_PATH "${LLVM_CMAKE_DIR}")
include(AddLLVM)
include_directories(${LLVM_INCLUDE_DIRS})
add_definitions(${LLVM_DEFINITIONS})

# MLIR
find_package(MLIR REQUIRED CONFIG)
list(APPEND CMAKE_MODULE_PATH "${MLIR_CMAKE_DIR}")
include(AddMLIR)
include_directories(${MLIR_INCLUDE_DIRS})
add_definitions(${MLIR_DEFINITIONS})

add_compile_options("/std:c++17")

add_executable(exe exe.cpp)
target_include_directories(exe PUBLIC
    ./lib
)

add_subdirectory(lib)

target_link_libraries(
        exe PRIVATE
        lib
        MLIRIR
)

if(WIN32)
        add_custom_command(TARGET exe POST_BUILD COMMAND ${CMAKE_COMMAND} -E copy
        $<TARGET_RUNTIME_DLLS:exe> $<TARGET_FILE_DIR:exe> COMMAND_EXPAND_LISTS)
endif()

Content of exe.cpp:

#include "lib.h"

int main() {
	mlir::MLIRContext context;

	lib::lowerer low(context);
	low.run();
}

Content of lib/CMakeLists.txt:

if(WIN32)
    set(CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS ON)
endif()

add_library(lib SHARED
    lib.cpp 
)

target_link_libraries(
        lib PUBLIC
        MLIRIR
)

Content of lib/lib.h:

#include "mlir/IR/MLIRContext.h"
#include "mlir/IR/Builders.h"

namespace lib {
	class lowerer {
	private:
		mlir::Builder builder;
	public:
		lowerer(mlir::MLIRContext& context);
		void lib::lowerer::run();
	};
}

Content of lib/lib.cpp:

#include "lib.h"
#include <llvm/Support/Error.h>
#include <llvm/Support/FileSystem.h>
#include <mlir/IR/BuiltinOps.h>

lib::lowerer::lowerer(mlir::MLIRContext& context) : builder(&context){}

void lib::lowerer::run() {
	std::error_code error;
	llvm::raw_fd_ostream os("-", error, llvm::sys::fs::OF_None);
	mlir::ModuleOp module = mlir::ModuleOp::create(builder.getUnknownLoc());
	os << *module;
}

This appears to be the dreaded issue of non-unique type IDs being assigned to an IR object kind due to multiple shared objects. You may want to see:

Thanks for bringing those posts to my attention. I now read both. I don’t use libLLVM, because it is not supported on Windows.

I think it is the issue that Uday referred to but with added twists for windows dll’s. If you end up with a typeid in a header (which there are still instances of), that will absolutely not work across DLL boundaries. PE/COFF executable have no mechanism for supporting weak linkage across boundaries, which is what is happening: the exe and DLL are each getting a separate value for the typeid of an attribute, and that is triggering the assertion. (I’ve heard/read that mingw does some runtime patching to try to help with this but that is different).

There is a thread with unfinished work to put all typeid 's in a fixed translation unit, which should fix this. Will post separately.

Here is the thread which outlines the work still needed: More work needed for TypeID duplication (help wanted)

So far I understood that I have two possibilities:

  1. try to port project to mingw and see if TypeID works there
  2. try to patch MLIR

Will try the above in the next days and report back.
Thanks for the help!

I can add a third: you can link MLIR statically in your project.

MLIR is already linked statically to my project because I build LLVM with both BUILD_SHARED_LIBS=OFF (this only works with mingw) and LLVM_BUILD_LLVM_DYLIB=OFF (this does not work on Windows).

I don’t know what to tell you: the core project doesn’t support building DLLs in the way you are trying to do as a consumer of it. The references above advertise necessary fixes to support cross DSO/DLL scenarios. That may be all that is needed and it may not – we won’t know until fixed and tested upstream. Patches for the above issues are welcome.

As an aside, also, I’ve not had great luck with just taking a project and assuming I can link it on Windows differently than it was set up to support and that all of the things that go into making good DLL setups are not going to have issues. For a cross platform project, such use cases typically represent a high bar, and it is one this project has not passed, as evidence by the restrictions on dynamic linking in the project itself.

We do support and have tests for creating a DLL that exports the C API, but that is all that the current state of the code supports now.

Now that we have our bug tracking system back, I’ve filed a real issue for this: https://github.com/llvm/llvm-project/issues/52621