Clang (with Visual Studio) wrongly complains about missing variables

Hi there - I'm building quite a complex Windows program here using VS2019. Obviously there's an EXE and there's maybe 30 or so DLLs. Some DLL's might have code which looks like this in a header file:-

     class whatever {
         static int revision_num;
     };

or if there's no class involved it'd maybe look like this:-

     extern int revision_num;

The actual int would then be instantiated in one of the DLL's source files. IIUC it's done like this to ensure that there's only ever one copy of 'revision_num'. It's internal to the DLL and typically, the DLL will offer an exported function called 'get_revision_num()' so that code in other modules (e.g. in the exe) can access it. This all builds fine if I use VS2019's MSVC compiler.

But if I switch VS2019 to use Clang (when building the EXE) Clang's linker will complain that it can't find the variable 'revision_num'. But of course, 'revision_num' is an internal variable that's private to the DLL - so the EXE should never need to know about it. Is this a known issue when using Clang with Visual Studio? Or is there maybe some linker option that'll tell Clang to ignore variables if the code never needs access to them? Hope that makes sense...

John

Another possibility just occurred to me... here's a real-world example from our code:-

#if defined (BUILDING_DLL)
   #define DLL_API __declspec(dllexport)
#else
   #define DLL_API __declspec(dllimport)
#endif

namespace Gtkmm2ext {

   class DLL_API Keyboard
   {
     public:
       Keyboard ();
       ~Keyboard ();

       static Keyboard& the_keyboard() { return *_the_keyboard };

     protected:
       static Keyboard* _the_keyboard;
   };

} /* namespace */

The above example is from a DLL but when I try to build the corresponding EXE, Clang's linker complains that it can't find '_the_keyboard' - so did the compiler (maybe) implement its call to 'the_keyboard()' as inline code, rather than importing it from the DLL? Maybe for very simple code like this, Clang will try to be clever and implement stuff inline if it can? And if so, is there some way to turn off that feature? Thanks,

John

I’m not sure it’s the right/necessary solution, but one way would be to move the function definition (for the_keyboard) out of line (define it in some .cpp/.cc/whatever file, not in the header) - if you don’t want it to be inlined.

'revision_num' is declared static. Try 'extern static int revision_num'.

Getting this sort of thing to compile under different compilers can be
tricky. Especially when exporting templates.

Jeff

Many thanks Jeff & David,

Overnight I remembered that Visual Studio itself offers a specific option to disable inline function expansion. For MSVC itself, this gets done by setting a compiler option called "/Ob" ("/Ob0" indicates disabled). If "/Ob0" isn't specified at compile time, inlining of functions is basically left to the compiler's discretion.

I checked this morning and sure enough the "/Ob" option doesn't get sent if I build with VS2019 and Clang - so I'm guessing "/Ob0" wouldn't be recognised by Clang's compiler?

So does Clang have it's own command-line option to disable inline function expansion? Or is that something which hasn't been implemented for Clang? Thanks,

John

...
I checked this morning and sure enough the "/Ob" option doesn't get sent
if I build with VS2019 and Clang - so I'm guessing "/Ob0" wouldn't be
recognised by Clang's compiler?

So does Clang have it's own command-line option to disable inline
function expansion? Or is that something which hasn't been implemented
for Clang? Thanks,

The U&L option is -fno-inline. But I think Clang has an option to
consume MSVC style options, so you may be able to use /Ob.

I think you can check which MSVC style options Clang accepts with
'clang-cl /?' or 'clang-cl -?'.

I don't use Clang on Windows, so I'm not really the person to answer
your Windows questions. Sorry about that.

Jeff

Thanks again Jeff and you're right... clang-cl /? reveals that it'll handle the Microsoft variant of /Ob0

And according to Visual Studio it is sending that option...

So do you happen to know if clang-cl produces some kinda log file (i.e. that'd let me see which commands it actually received?)

John

I just found something else which might be significant... I noticed that when linking this exe, I had a linker option enabled called /FORCE:MULTIPLE

If I remove that option, the Clang linker then gives me a list of about a dozen duplicated symbols. And with that option I get a similarly sized list showing unresolved symbols.

Alternatively, changing it to /FORCE makes the exe compile!! It crashes as soon as I try to run it but at least it gets me a bit closer.

John

All the signs here are that Clang is still trying to inline stuff - even when it's been told not to... so is there some way I could check if it responds correctly to "/Ob0"? Maybe ask a question on cfe-dev ?

John

LLVM dev would probably be a good place to bring up the topic.

Several people work on the Windows port. There's one person in
particular who does most of the heavy lifting (but I don't recall the
person's name). LLVM dev is where to find the people.

Jeff

Probably Reid and Hans are folks to start with for Windows support

Looking back in the thread, I found the example code, and I see that MSVC refuses to inline this helper, but clang will inline it. I believe clang is permitted to inline it, MSVC will export the static data member (_the_keyboard), so everything should work as long as you provide a definition of _the_keyboard in the DLL and set -DBUILDING_DLL.

Example:

$ cat t.cpp
#if defined (BUILDING_DLL)
#define DLL_API __declspec(dllexport)
#else
#define DLL_API __declspec(dllimport)
#endif
class DLL_API Keyboard
{
public:
Keyboard ();
~Keyboard ();
static Keyboard& the_keyboard() { return _the_keyboard; }
protected:
static Keyboard
_the_keyboard;
};
#if defined(BUILDING_DLL)
Keyboard *Keyboard::_the_keyboard = nullptr;
#else
Keyboard &useit() { return Keyboard::the_keyboard(); }
#endif

$ cl -c t.cpp -O2 -DBUILDING_DLL && dumpbin -directives t.obj

/EXPORT:?_the_keyboard@Keyboard@@1PEAV1@EA,DATA

clang-cl does support /Ob0 if you want to prevent it from inlining, but that’s a big hammer, you could use __declspec(noinline) as a more targeted workaround.

So, I think this just comes down to different inliner heuristics in MSVC and clang. If you provide and export a definition of this static global, things should link as expected.

Thanks for coming in on this, Reid. As Jeff suggested, I transferred the question to llvm-dev where there's a small discussion going on now. I must admit though, my personal view is that declaring something as __declspec(dllimport) should automatically exclude it from being inlined (I'm pretty sure that's what Microsoft itself does...)

Ultimately, the main advantage of a DLL is that it offers dynamic linkage. In other words, DLL features can be updated simply by updating the DLL (i.e. without needing to update all the apps which use it). In the dim old days this could lead to something called "DLL Hell" but WinSxS has largely consigned that to history now. So whilst inlining code from a DLL might seem like a good idea, it throws away the main advantage of DLL's without offering anything better.

Over on llvm-dev I'm trying to persuade them that declaring something as __declspec(dllimport) should automatically exclude it from being inlined. And to be honest, I'd be quite surprised if that's not what Microsoft intended.

John

Looking back in the thread, I found the example code, and I see that
MSVC refuses to inline this helper, but clang will inline it. I
believe clang is permitted to inline it, MSVC will export the static
data member (_the_keyboard), so everything should work as long as you
provide a definition of _the_keyboard in the DLL and set -DBUILDING_DLL.

Thanks for coming in on this, Reid. As Jeff suggested, I transferred the
question to llvm-dev where there’s a small discussion going on now. I
must admit though, my personal view is that declaring something as
__declspec(dllimport) should automatically exclude it from being inlined
(I’m pretty sure that’s what Microsoft itself does…)

Ultimately, the main advantage of a DLL is that it offers dynamic
linkage. In other words, DLL features can be updated simply by updating
the DLL (i.e. without needing to update all the apps which use it). In
the dim old days this could lead to something called “DLL Hell” but
WinSxS has largely consigned that to history now. So whilst inlining
code from a DLL might seem like a good idea, it throws away the main
advantage of DLL’s without offering anything better.

Presumably it offers the benefit of inlining optimizations - so there’s probably some execution speed improvement to tradeoff. (& a way the user can address the issue by moving functions out of line)

Over on llvm-dev I’m trying to persuade them that declaring something as
__declspec(dllimport) should automatically exclude it from being
inlined. And to be honest, I’d be quite surprised if that’s not what
Microsoft intended.

I think if it’s clearly demonstrated that that’s Microsoft’s implementation - that no matter how hard you ask it to optimize and how simple the function is, that it won’t inline a dllexported function that’s inline in a header (both implicitly inline in a class definition, and probably check the case of a standalone dllexported inline non-member function in a header) that I’d say (though I have little sway/weight in this design decision) clang-cl should implement the same behavior, because it is observable/can be relied upon as you have (though also - dllexported variables should be defined somewhere, generally) & an opt-in flag to what is the current behavior (dllexport-inlining).

  • Dave

I think this is the old binary compatibility problem, like exceptions
crossing module boundaries. I think part of the problem is, Microsoft
does not publish a specification. For years Borland had to figure it
out by reverse engineering what MS was doing.

I guess the situation has not gotten any better.

Jeff

Many thanks Dave,

Over on llvm-dev a contributer called Hans found this on Microsoft's MSDN site:-

   Defining Inline C++ Functions with dllexport and dllimport | Microsoft Docs

So the current theory is that Microsoft's compiler can indeed inline DLL functions - but probably only if they're specifically declared using 'inline'

And I've posted a question on the MSDN Developer forum to ask if someone can clarify this for us. Fingers crossed!

John