Specific clang-cl preprocessor emulation problem

I have reported bug 27380 located at https://llvm.org/bugs/show_bug.cgi?id=27380 for a clang-cl preprocessor problem as it emulates the VC++ preprocessor. The problem can be illustrated wit the code:

#define TEST_MACRO(x,y) TEST_EXPAND(2, 0, TEST_SOMETHING(x,y,1))
#define TEST_SOMETHING(x,y,z) 2

#define TEST_EXPAND(...) TEST_EXPAND_I(TEST_OVR(TEST_NAME, __VA_ARGS__), (__VA_ARGS__))
#define TEST_OVR(name,...) TEST_SOMENAME
#define TEST_EXPAND_I(m, args) TEST_EXPAND_II(m, args)
#define TEST_EXPAND_II(m, args) TEST_CAT(m ## args,)
#define TEST_SOMENAME(x,y,z) ;

#define TEST_CAT(a, b) TEST_CAT_I(a, b)
#define TEST_CAT_I(a, b) TEST_CAT_II(~, a ## b)
#define TEST_CAT_II(p, res) res

int main()
  {
  TEST_MACRO(1,2)
     }

which compiles correctly with VC++14 but with the latest clang-cl built from the trunk gives:

test_clang.cpp(17,2): error: pasting formed 'TEST_SOMENAME(', an invalid preprocessing token [-Winvalid-token-paste]
         TEST_MACRO(1,2)
         ^
test_clang.cpp(2,25): note: expanded from macro 'TEST_MACRO'
#define TEST_MACRO(x,y) TEST_EXPAND(2, 0, TEST_SOMETHING(x,y,1))
                         ^
test_clang.cpp(5,26): note: expanded from macro 'TEST_EXPAND'
#define TEST_EXPAND(...) TEST_EXPAND_I(TEST_OVR(TEST_NAME, __VA_ARGS__), (__VA_ARGS__))
                          ^
test_clang.cpp(7,32): note: expanded from macro 'TEST_EXPAND_I'
#define TEST_EXPAND_I(m, args) TEST_EXPAND_II(m, args)
                                ^
test_clang.cpp(8,44): note: expanded from macro 'TEST_EXPAND_II'
#define TEST_EXPAND_II(m, args) TEST_CAT(m ## args,)

The bug is instrumental in keeping Boost PP from working correctly with clang-cl, and therefore keeping many other Boost libraries, which use Boost PP, from working with clang-cl. There appear to be other problems but until this gets fixed I won't know for sure, since fixing this problem may make any others which have shown up go away.

Thanks for listening !

Did you try disabling -Winvalid-token-paste (-Wno-invalid-token-paste)?
Possibly it should be disabled by default in MSVC emulation mode, but
macro_paste_identifier_error.c explicitly turns it off as well as
specifying -fms-extensions.

-- James

Thanks ! Your suggestion of using -Wno-invalid-token-paste worked.

However:

1) If clang-cl is trying to emulate the non-standard VC++ preprocessor, and clearly the VC++ preprocessor does not think that the token pasting is invalid doesn't it behoove clang-cl to also follow this behavior ? If it is not following the behavior of the VC++ preprocessor, whether that preprocessor is correct or not according to the C++ standard, then it is simply failing its mission.

2) I have never believed it correct for clang-cl to emulate the non-standard VC++ preprocessor, except in cases in which it must do so in order to process Windows or VC++ header files, but since it has decided to do so in all situations there should be no need for the end-user to pass a compiler option to have clang-cl get this right.

3) I do not know what your reference to macro_paste_identifier_error.c means. I assume that is a source file used to build clang-cl.

3) I do not know what your reference to macro_paste_identifier_error.c
means.

clang/test/Preprocessor/macro_paste_identifier_error.c

I assume that is a source file used to build clang-cl.

No, it's a test.

I see it but I don't understand why you should be expecting a programmer using clang-cl ( or clang/clang++ targeting VC++ ) to have to pass a compiler option to emulate VC++ when you have already pushed clang-cl as emulating VC++ by default.

Please, don't assume that anyone is expecting such a thing. I was just
providing the test as an example of how things work today.

My reason for asking whether you'd tried it was to ascertain whether
disabling that warning would resolve your issue. It sounds like it would,
so thank you for confirming that.

MSVC++ support is still fairly new, and Boost PP pushes the limits. I
suspect that people are entirely open to changing this, given a polite
request and a straightforward rationale. I think (though I'm not sure)
that MSVC++ mode turns off various other diagnostics already.

-- James

I added to the bug report my comment that '-Wno-invalid-token-paste' should be the default when compiling clang-cl in order to emulate VC++. I am not good at politeness, but I am good at straightforwardness. I do want clang-cl to succeed so that Boost users can use it with Boost libraries if they want to go that route.