Using LLD to create a .lib from a .def

I’m copying some LLD code into my codebase like this:

// workaround for LLD not exposing ability to convert .def to .lib

#include

namespace lld {
namespace coff {

class SymbolBody;
class StringChunk;
struct Symbol;

struct Export {
StringRef Name; // N in /export:N or /export:E=N
StringRef ExtName; // E in /export:E=N
SymbolBody *Sym = nullptr;
uint16_t Ordinal = 0;
bool Noname = false;
bool Data = false;
bool Private = false;

// If an export is a form of /export:foo=dllname.bar, that means
// that foo should be exported as an alias to bar in the DLL.
// ForwardTo is set to “dllname.bar” part. Usually empty.
StringRef ForwardTo;
StringChunk *ForwardChunk = nullptr;

// True if this /export option was in .drectves section.
bool Directives = false;
StringRef SymbolName;
StringRef ExportName; // Name in DLL

bool operator==(const Export &E) {
return (Name == E.Name && ExtName == E.ExtName &&
Ordinal == E.Ordinal && Noname == E.Noname &&
Data == E.Data && Private == E.Private);
}
};

enum class DebugType {
None = 0x0,
CV = 0x1, /// CodeView
PData = 0x2, /// Procedure Data
Fixup = 0x4, /// Relocation Table
};

struct Configuration {
enum ManifestKind { SideBySide, Embed, No };
llvm::COFF::MachineTypes Machine = llvm::COFF::IMAGE_FILE_MACHINE_UNKNOWN;
bool Verbose = false;
llvm::COFF::WindowsSubsystem Subsystem = llvm::COFF::IMAGE_SUBSYSTEM_UNKNOWN;
SymbolBody *Entry = nullptr;
bool NoEntry = false;
std::string OutputFile;
bool DoGC = true;
bool DoICF = true;
bool Relocatable = true;
bool Force = false;
bool Debug = false;
bool WriteSymtab = true;
unsigned DebugTypes = static_cast(DebugType::None);
StringRef PDBPath;

// Symbols in this set are considered as live by the garbage collector.
std::set<SymbolBody *> GCRoot;

std::set NoDefaultLibs;
bool NoDefaultLibAll = false;

// True if we are creating a DLL.
bool DLL = false;
StringRef Implib;
std::vector Exports;
std::setstd::string DelayLoads;
std::map<std::string, int> DLLOrder;
SymbolBody *DelayLoadHelper = nullptr;

// Used for SafeSEH.
Symbol *SEHTable = nullptr;
Symbol *SEHCount = nullptr;

// Used for /opt:lldlto=N
unsigned LTOOptLevel = 2;

// Used for /opt:lldltojobs=N
unsigned LTOJobs = 1;

// Used for /merge:from=to (e.g. /merge:.rdata=.text)
std::map<StringRef, StringRef> Merge;

// Used for /section=.name,{DEKPRSW} to set section attributes.
std::map<StringRef, uint32_t> Section;

// Options for manifest files.
ManifestKind Manifest = SideBySide;
int ManifestID = 1;
StringRef ManifestDependency;
bool ManifestUAC = true;
std::vectorstd::string ManifestInput;
StringRef ManifestLevel = “‘asInvoker’”;
StringRef ManifestUIAccess = “‘false’”;
StringRef ManifestFile;

// Used for /failifmismatch.
std::map<StringRef, StringRef> MustMatch;

// Used for /alternatename.
std::map<StringRef, StringRef> AlternateNames;

uint64_t ImageBase = -1;
uint64_t StackReserve = 1024 * 1024;
uint64_t StackCommit = 4096;
uint64_t HeapReserve = 1024 * 1024;
uint64_t HeapCommit = 4096;
uint32_t MajorImageVersion = 0;
uint32_t MinorImageVersion = 0;
uint32_t MajorOSVersion = 6;
uint32_t MinorOSVersion = 0;
bool DynamicBase = true;
bool AllowBind = true;
bool NxCompat = true;
bool AllowIsolation = true;
bool TerminalServerAware = true;
bool LargeAddressAware = false;
bool HighEntropyVA = false;

// This is for debugging.
bool DebugPdb = false;
bool DumpPdb = false;
};

extern Configuration *Config;

void writeImportLibrary();
void parseModuleDefs(MemoryBufferRef MB);

} // namespace coff
} // namespace lld

//=========================================================================

This is so that I can write the following user code:

// writes the output to dll_path with .dll replaced with .lib
void ZigLLDDefToLib(Buf *def_contents, Buf *dll_path) {
lld::coff::Config = new lld::coff::Configuration;
auto mem_buf = MemoryBuffer::getMemBuffer(buf_ptr(def_contents));
MemoryBufferRef mbref(*mem_buf);
lld::coff::parseModuleDefs(mbref);
lld::coff::Config->OutputFile = buf_ptr(dll_path);
lld::coff::writeImportLibrary();
}

Then I give it def_contents that looks like:

LIBRARY kernel32
EXPORTS
ExitProcess
GetConsoleMode
GetStdHandle
GetFileInformationByHandleEx
WriteFile
GetLastError

with dll_path set to ./zig-cache/all.dll. This generates ./zig-cache/all.lib.

The generated LLVM IR looks like:

; Function Attrs: noreturn nounwind
declare void @ExitProcess(i32) #6

; Function Attrs: nounwind
declare i1 @GetConsoleMode(i8* nonnull, i32* nonnull) #3
; Function Attrs: nounwind
declare i8* @GetStdHandle(i32) #3

; Function Attrs: nounwind
declare i1 @GetFileInformationByHandleEx(i8* nonnull, i32, i8* nonnull, i32) #3

; Function Attrs: nounwind
declare i1 @WriteFile(i8* nonnull, i8* nonnull readonly, i32, i32*, %OVERLAPPED*) #3
; Function Attrs: nounwind
declare i32 @GetLastError() #3

…with code you would expect to call these functions.

Then I link with

lld -NOLOGO -MACHINE:X64 -OUT:hello.exe -NODEFAULTLIB -ENTRY:_start ./zig-cache/hello.obj ./zig-cache/builtin.obj ./zig-cache/compiler_rt.obj ./zig-cache/all.lib

and I get the following errors:

./zig-cache/hello.obj: undefined symbol: ExitProcess
./zig-cache/hello.obj: undefined symbol: GetConsoleMode
./zig-cache/hello.obj: undefined symbol: GetStdHandle
./zig-cache/hello.obj: undefined symbol: GetFileInformationByHandleEx
./zig-cache/hello.obj: undefined symbol: GetLastError
./zig-cache/hello.obj: undefined symbol: WriteFile
error: link failed

  1. Is there something else I need to be doing to make this work correctly?
    (Note: I already tried renaming “all” to “kernel32”. Ideally I could generate 1 .lib file for all the DLL calls needed.)
  2. Can LLD expose this ability directly so I don’t have to copy paste a bunch of LLD internal stuff?

Regards,
Andrew Kelley
ziglang.org

I'm copying some LLD code into my codebase like this:

// workaround for LLD not exposing ability to convert .def to .lib

#include <set>

namespace lld {
namespace coff {

class SymbolBody;
class StringChunk;
struct Symbol;

struct Export {
  StringRef Name; // N in /export:N or /export:E=N
  StringRef ExtName; // E in /export:E=N
  SymbolBody *Sym = nullptr;
  uint16_t Ordinal = 0;
  bool Noname = false;
  bool Data = false;
  bool Private = false;

  // If an export is a form of /export:foo=dllname.bar, that means
  // that foo should be exported as an alias to bar in the DLL.
  // ForwardTo is set to "dllname.bar" part. Usually empty.
  StringRef ForwardTo;
  StringChunk *ForwardChunk = nullptr;

  // True if this /export option was in .drectves section.
  bool Directives = false;
  StringRef SymbolName;
  StringRef ExportName; // Name in DLL

  bool operator==(const Export &E) {
    return (Name == E.Name && ExtName == E.ExtName &&
            Ordinal == E.Ordinal && Noname == E.Noname &&
            Data == E.Data && Private == E.Private);
  }
};

enum class DebugType {
  None = 0x0,
  CV = 0x1, /// CodeView
  PData = 0x2, /// Procedure Data
  Fixup = 0x4, /// Relocation Table
};

struct Configuration {
  enum ManifestKind { SideBySide, Embed, No };
  llvm::COFF::MachineTypes Machine = llvm::COFF::IMAGE_FILE_
MACHINE_UNKNOWN;
  bool Verbose = false;
  llvm::COFF::WindowsSubsystem Subsystem = llvm::COFF::IMAGE_SUBSYSTEM_
UNKNOWN;
  SymbolBody *Entry = nullptr;
  bool NoEntry = false;
  std::string OutputFile;
  bool DoGC = true;
  bool DoICF = true;
  bool Relocatable = true;
  bool Force = false;
  bool Debug = false;
  bool WriteSymtab = true;
  unsigned DebugTypes = static_cast<unsigned>(DebugType::None);
  StringRef PDBPath;

  // Symbols in this set are considered as live by the garbage collector.
  std::set<SymbolBody *> GCRoot;

  std::set<StringRef> NoDefaultLibs;
  bool NoDefaultLibAll = false;

  // True if we are creating a DLL.
  bool DLL = false;
  StringRef Implib;
  std::vector<Export> Exports;
  std::set<std::string> DelayLoads;
  std::map<std::string, int> DLLOrder;
  SymbolBody *DelayLoadHelper = nullptr;

  // Used for SafeSEH.
  Symbol *SEHTable = nullptr;
  Symbol *SEHCount = nullptr;

  // Used for /opt:lldlto=N
  unsigned LTOOptLevel = 2;

  // Used for /opt:lldltojobs=N
  unsigned LTOJobs = 1;

  // Used for /merge:from=to (e.g. /merge:.rdata=.text)
  std::map<StringRef, StringRef> Merge;

  // Used for /section=.name,{DEKPRSW} to set section attributes.
  std::map<StringRef, uint32_t> Section;

  // Options for manifest files.
  ManifestKind Manifest = SideBySide;
  int ManifestID = 1;
  StringRef ManifestDependency;
  bool ManifestUAC = true;
  std::vector<std::string> ManifestInput;
  StringRef ManifestLevel = "'asInvoker'";
  StringRef ManifestUIAccess = "'false'";
  StringRef ManifestFile;

  // Used for /failifmismatch.
  std::map<StringRef, StringRef> MustMatch;

  // Used for /alternatename.
  std::map<StringRef, StringRef> AlternateNames;

  uint64_t ImageBase = -1;
  uint64_t StackReserve = 1024 * 1024;
  uint64_t StackCommit = 4096;
  uint64_t HeapReserve = 1024 * 1024;
  uint64_t HeapCommit = 4096;
  uint32_t MajorImageVersion = 0;
  uint32_t MinorImageVersion = 0;
  uint32_t MajorOSVersion = 6;
  uint32_t MinorOSVersion = 0;
  bool DynamicBase = true;
  bool AllowBind = true;
  bool NxCompat = true;
  bool AllowIsolation = true;
  bool TerminalServerAware = true;
  bool LargeAddressAware = false;
  bool HighEntropyVA = false;

  // This is for debugging.
  bool DebugPdb = false;
  bool DumpPdb = false;
};

extern Configuration *Config;

void writeImportLibrary();
void parseModuleDefs(MemoryBufferRef MB);

} // namespace coff
} // namespace lld

//==========================================================

This is so that I can write the following user code:

// writes the output to dll_path with .dll replaced with .lib
void ZigLLDDefToLib(Buf *def_contents, Buf *dll_path) {
    lld::coff::Config = new lld::coff::Configuration;
    auto mem_buf = MemoryBuffer::getMemBuffer(buf_ptr(def_contents));
    MemoryBufferRef mbref(*mem_buf);
    lld::coff::parseModuleDefs(mbref);
    lld::coff::Config->OutputFile = buf_ptr(dll_path);
    lld::coff::writeImportLibrary();
}

Then I give it def_contents that looks like:
LIBRARY kernel32
EXPORTS
ExitProcess
GetConsoleMode
GetStdHandle
GetFileInformationByHandleEx
WriteFile
GetLastError

with dll_path set to ./zig-cache/all.dll. This generates
./zig-cache/all.lib.

The generated LLVM IR looks like:

; Function Attrs: noreturn nounwind
declare void @ExitProcess(i32) #6
; Function Attrs: nounwind
declare i1 @GetConsoleMode(i8* nonnull, i32* nonnull) #3
; Function Attrs: nounwind
declare i8* @GetStdHandle(i32) #3
; Function Attrs: nounwind
declare i1 @GetFileInformationByHandleEx(i8* nonnull, i32, i8* nonnull,
i32) #3
; Function Attrs: nounwind
declare i1 @WriteFile(i8* nonnull, i8* nonnull readonly, i32, i32*,
%OVERLAPPED*) #3
; Function Attrs: nounwind
declare i32 @GetLastError() #3

...with code you would expect to call these functions.

Then I link with
lld -NOLOGO -MACHINE:X64 -OUT:hello.exe -NODEFAULTLIB -ENTRY:_start
./zig-cache/hello.obj ./zig-cache/builtin.obj ./zig-cache/compiler_rt.obj
./zig-cache/all.lib

and I get the following errors:
./zig-cache/hello.obj: undefined symbol: ExitProcess
./zig-cache/hello.obj: undefined symbol: GetConsoleMode
./zig-cache/hello.obj: undefined symbol: GetStdHandle
./zig-cache/hello.obj: undefined symbol: GetFileInformationByHandleEx
./zig-cache/hello.obj: undefined symbol: GetLastError
./zig-cache/hello.obj: undefined symbol: WriteFile
error: link failed

1. Is there something else I need to be doing to make this work correctly?

The first thing I would check is to make sure that you created your .lib
file for x86-64. Windows uses different name mangling scheme for x86 and
x86-64, so if you mix the two, it could result in an "undefined symbol"
error.

(Note: I already tried renaming "all" to "kernel32". Ideally I could
generate 1 .lib file for all the DLL calls needed.)
2. Can LLD expose this ability directly so I don't have to copy paste a
bunch of LLD internal stuff?

Martell recently factored out the code to parse and generate module
definition files. You might be able to use that. See
llvm/Object/COFFModuleDefinition.h.

The location at where LLD uses COFFModuleDefinition.h is in parseModuleDefs
and createImportLibrary in COFF/Driver.cpp.

Regards,

I'm copying some LLD code into my codebase like this:

// workaround for LLD not exposing ability to convert .def to .lib

#include <set>

namespace lld {
namespace coff {

class SymbolBody;
class StringChunk;
struct Symbol;

struct Export {
  StringRef Name; // N in /export:N or /export:E=N
  StringRef ExtName; // E in /export:E=N
  SymbolBody *Sym = nullptr;
  uint16_t Ordinal = 0;
  bool Noname = false;
  bool Data = false;
  bool Private = false;

  // If an export is a form of /export:foo=dllname.bar, that means
  // that foo should be exported as an alias to bar in the DLL.
  // ForwardTo is set to "dllname.bar" part. Usually empty.
  StringRef ForwardTo;
  StringChunk *ForwardChunk = nullptr;

  // True if this /export option was in .drectves section.
  bool Directives = false;
  StringRef SymbolName;
  StringRef ExportName; // Name in DLL

  bool operator==(const Export &E) {
    return (Name == E.Name && ExtName == E.ExtName &&
            Ordinal == E.Ordinal && Noname == E.Noname &&
            Data == E.Data && Private == E.Private);
  }
};

enum class DebugType {
  None = 0x0,
  CV = 0x1, /// CodeView
  PData = 0x2, /// Procedure Data
  Fixup = 0x4, /// Relocation Table
};

struct Configuration {
  enum ManifestKind { SideBySide, Embed, No };
  llvm::COFF::MachineTypes Machine = llvm::COFF::IMAGE_FILE_MACHINE
_UNKNOWN;
  bool Verbose = false;
  llvm::COFF::WindowsSubsystem Subsystem = llvm::COFF::IMAGE_SUBSYSTEM_UN
KNOWN;
  SymbolBody *Entry = nullptr;
  bool NoEntry = false;
  std::string OutputFile;
  bool DoGC = true;
  bool DoICF = true;
  bool Relocatable = true;
  bool Force = false;
  bool Debug = false;
  bool WriteSymtab = true;
  unsigned DebugTypes = static_cast<unsigned>(DebugType::None);
  StringRef PDBPath;

  // Symbols in this set are considered as live by the garbage collector.
  std::set<SymbolBody *> GCRoot;

  std::set<StringRef> NoDefaultLibs;
  bool NoDefaultLibAll = false;

  // True if we are creating a DLL.
  bool DLL = false;
  StringRef Implib;
  std::vector<Export> Exports;
  std::set<std::string> DelayLoads;
  std::map<std::string, int> DLLOrder;
  SymbolBody *DelayLoadHelper = nullptr;

  // Used for SafeSEH.
  Symbol *SEHTable = nullptr;
  Symbol *SEHCount = nullptr;

  // Used for /opt:lldlto=N
  unsigned LTOOptLevel = 2;

  // Used for /opt:lldltojobs=N
  unsigned LTOJobs = 1;

  // Used for /merge:from=to (e.g. /merge:.rdata=.text)
  std::map<StringRef, StringRef> Merge;

  // Used for /section=.name,{DEKPRSW} to set section attributes.
  std::map<StringRef, uint32_t> Section;

  // Options for manifest files.
  ManifestKind Manifest = SideBySide;
  int ManifestID = 1;
  StringRef ManifestDependency;
  bool ManifestUAC = true;
  std::vector<std::string> ManifestInput;
  StringRef ManifestLevel = "'asInvoker'";
  StringRef ManifestUIAccess = "'false'";
  StringRef ManifestFile;

  // Used for /failifmismatch.
  std::map<StringRef, StringRef> MustMatch;

  // Used for /alternatename.
  std::map<StringRef, StringRef> AlternateNames;

  uint64_t ImageBase = -1;
  uint64_t StackReserve = 1024 * 1024;
  uint64_t StackCommit = 4096;
  uint64_t HeapReserve = 1024 * 1024;
  uint64_t HeapCommit = 4096;
  uint32_t MajorImageVersion = 0;
  uint32_t MinorImageVersion = 0;
  uint32_t MajorOSVersion = 6;
  uint32_t MinorOSVersion = 0;
  bool DynamicBase = true;
  bool AllowBind = true;
  bool NxCompat = true;
  bool AllowIsolation = true;
  bool TerminalServerAware = true;
  bool LargeAddressAware = false;
  bool HighEntropyVA = false;

  // This is for debugging.
  bool DebugPdb = false;
  bool DumpPdb = false;
};

extern Configuration *Config;

void writeImportLibrary();
void parseModuleDefs(MemoryBufferRef MB);

} // namespace coff
} // namespace lld

//==========================================================

This is so that I can write the following user code:

// writes the output to dll_path with .dll replaced with .lib
void ZigLLDDefToLib(Buf *def_contents, Buf *dll_path) {
    lld::coff::Config = new lld::coff::Configuration;
    auto mem_buf = MemoryBuffer::getMemBuffer(buf_ptr(def_contents));
    MemoryBufferRef mbref(*mem_buf);
    lld::coff::parseModuleDefs(mbref);
    lld::coff::Config->OutputFile = buf_ptr(dll_path);
    lld::coff::writeImportLibrary();
}

Then I give it def_contents that looks like:
LIBRARY kernel32
EXPORTS
ExitProcess
GetConsoleMode
GetStdHandle
GetFileInformationByHandleEx
WriteFile
GetLastError

with dll_path set to ./zig-cache/all.dll. This generates
./zig-cache/all.lib.

The generated LLVM IR looks like:

; Function Attrs: noreturn nounwind
declare void @ExitProcess(i32) #6
; Function Attrs: nounwind
declare i1 @GetConsoleMode(i8* nonnull, i32* nonnull) #3
; Function Attrs: nounwind
declare i8* @GetStdHandle(i32) #3
; Function Attrs: nounwind
declare i1 @GetFileInformationByHandleEx(i8* nonnull, i32, i8* nonnull,
i32) #3
; Function Attrs: nounwind
declare i1 @WriteFile(i8* nonnull, i8* nonnull readonly, i32, i32*,
%OVERLAPPED*) #3
; Function Attrs: nounwind
declare i32 @GetLastError() #3

...with code you would expect to call these functions.

Then I link with
lld -NOLOGO -MACHINE:X64 -OUT:hello.exe -NODEFAULTLIB -ENTRY:_start
./zig-cache/hello.obj ./zig-cache/builtin.obj ./zig-cache/compiler_rt.obj
./zig-cache/all.lib

and I get the following errors:
./zig-cache/hello.obj: undefined symbol: ExitProcess
./zig-cache/hello.obj: undefined symbol: GetConsoleMode
./zig-cache/hello.obj: undefined symbol: GetStdHandle
./zig-cache/hello.obj: undefined symbol: GetFileInformationByHandleEx
./zig-cache/hello.obj: undefined symbol: GetLastError
./zig-cache/hello.obj: undefined symbol: WriteFile
error: link failed

1. Is there something else I need to be doing to make this work correctly?

The first thing I would check is to make sure that you created your .lib
file for x86-64. Windows uses different name mangling scheme for x86 and
x86-64, so if you mix the two, it could result in an "undefined symbol"
error.

I tested this by setting the machine in the config directly to AMD64.

(Note: I already tried renaming "all" to "kernel32". Ideally I could
generate 1 .lib file for all the DLL calls needed.)
2. Can LLD expose this ability directly so I don't have to copy paste a
bunch of LLD internal stuff?

Martell recently factored out the code to parse and generate module
definition files. You might be able to use that. See llvm/Object/
COFFModuleDefinition.h.

The location at where LLD uses COFFModuleDefinition.h is in
parseModuleDefs and createImportLibrary in COFF/Driver.cpp.

OK, thanks. I'll get a build going with latest trunk and see if I can get
it to work.

When I build llvm-project from source (revision 53a3cb539f7e92e70d4eef8d1b219cbef7c8ba1c from https://github.com/llvm-project/llvm-project-20170507)

The build and install seemed to work correctly. But then when I run llvm-config --libfiles, I get this:

llvm-config: error: component libraries and shared library

llvm-config: error: missing: /home/andy/local/lib/libLLVMTestingSupport.a

The library got built: ./llvm/build/lib/libLLVMTestingSupport.a
But make install didn’t copy it to the prefix path.

I saw this change come in yesterday:

commit 572ad839e2f66eaa82ffc71b1061eb3d06a4d126
Author: Saleem Abdulrasool <compnerd@compnerd.org>

COFF: add support for lib mode usage

When link is invoked with /def: and no input files, it behaves as if
lib.exe was invoked. Emulate this behaviour, generating the import
library from the def file that was passed. Because there is no input to
actually generate the dll, we simply process the def file early and exit
once we have created the import library.

Thank you for this, Saleem.

I still have this problem though, with latest master of llvm-project, which is bd8af0ee30bc1a308bf286089b172f2ae87f3de5

llvm-config: error: component libraries and shared library

llvm-config: error: missing: /home/andy/local/lib/libLLVMTestingSupport.a

I’m pretty sure it’s not just me, anyone should be able to reproduce this issue with a clean build.

Regards,
Andrew

It looks like it is an unrelated build issue (probably related to LLVM_BUILD_LLVM_DYLIB), no?

It’s a pretty standard build, all I did is change the prefix path:

cmake … -DCMAKE_INSTALL_PREFIX=/home/andy/local -DCMAKE_PREFIX_PATH=/home/andy/local -DCMAKE_BUILD_TYPE=Debug