About mismatching calling conventions

Hi llvm-dev,

Every now and then, when building a direct ``CallInst`` either from IRBuilder
or through ``Create*``, I have a latent bug because the calling
convention (CC) expected by the function is not set on the call, and I need
to manually call ``setCallingConv``.

Now, from LangRef#calling-conventions,

    The calling convention of any pair of dynamic caller/callee must
    match, or the behavior of the program is undefined.

So the behavior is correctly defined, as emphasised by the check in
``InstCombineCalls`` that turns mismatching convention into an ``undef``.

This is however very error-prone. Maybe we should either:

- update the API to enforce the CC when building a ``CallInst``

- update LangRef to state that mismatching CC in static call
  is an error (note that LangRef is unclear about static call, the quote
  only references indirect calls)

- add an optional setting in the ``CreateCall`` that enforces CC
  (and let it default to yes?)

Any thoughts on that?

Serge

This is an old FAQ that I remember running into 8 years ago: https://llvm.org/docs/FAQ.html#why-does-instcombine-simplifycfg-turn-a-call-to-a-function-with-a-mismatched-calling-convention-into-unreachable-why-not-make-the-verifier-reject-it

However, I think you are right, we could make CreateCall enforce calling convention match for direct calls, and require callers to manually set the calling convention with setCallingConv if they want the convention to not match.

Somewhat related to this patchset, for adding support for specifying address space on functions.

https://reviews.llvm.org/D37052