[libc++] Guarantee that TUs built with different versions of libc++ can be linked together

Hi,

libc++ currently allows TUs that were built using different versions of libc++ to be linked together. Roughly speaking, this is achieved by making sure that anything not ABI-stable is local to each TU. This also allows distributing a static archive without caring too much about the version of libc++ it was compiled with, at least in principle. Supporting this use case creates code bloat for all users of libc++, since private functions are duplicated in each TU.

I posit that few clients actually care about this, yet everybody is paying for it in terms of executable size. Hence, I would like to change the default guarantee offered by libc++ from "this use case is supported" to "this use case is not supported". The use case would still be supported by defining a macro for those that need it.

Does anybody have objections to this change?

Louis

I 100% support not providing this guarantee by default. It always seemed a bit too heroic to me for libc++ to try to provide this support.

I think most people with standard library version skew typically only have skew between separate DSOs.

I support this change too.

At least in the short term, I think vendors should still have the capability to provide this guarantee without user opt-in by changing the default macro settings (switching it to a user opt-out).

I support this change too.

At least in the short term, I think vendors should still have the capability to provide this guarantee without user opt-in by changing the default macro settings (switching it to a user opt-out).

I agree with both of these points.

John.

I support this change too.

At least in the short term, I think vendors should still have the capability to provide this guarantee without user opt-in by changing the default macro settings (switching it to a user opt-out).

I agree with both of these points.

I implemented these in the latest revision of https://reviews.llvm.org/D50652.

With https://reviews.llvm.org/D50652, by default, internal_linkage is NOT used and it is NOT safe to link TUs compiled with different versions of libc++. When _LIBCPP_ABI_HIDDEN_USE_INTERNAL_LINKAGE is defined, internal_linkage is used and it IS safe again to link TUs compiled with different versions of libc++.

To make that the default behavior, libc++ can be configured with cmake <…> -DLIBCXX_ABI_HIDDEN_USE_INTERNAL_LINKAGE=ON and _LIBCPP_ABI_HIDDEN_USE_INTERNAL_LINKAGE will be defined by default.

Bikeshedding on the name of that option is welcome on the review page — but I’d like to keep the form _LIBCPP_ABI_XXX to be consistent with other ABI-related macros.

Louis

I support this change too.

At least in the short term, I think vendors should still have the capability to provide this guarantee without user opt-in by changing the default macro settings (switching it to a user opt-out).

I agree with both of these points.

I implemented these in the latest revision of https://reviews.llvm.org/D50652.

With https://reviews.llvm.org/D50652, by default, internal_linkage is NOT used and it is NOT safe to link TUs compiled with different versions of libc++. When _LIBCPP_ABI_HIDDEN_USE_INTERNAL_LINKAGE is defined, internal_linkage is used and it IS safe again to link TUs compiled with different versions of libc++.

I may be confused about the meaning of the patch, but it looks to me like that macro is choosing between two ways of making such TU-libc+±version-mixing safe: always_inline and internal_linkage. Which seems like a good thing, but distinct.

I support this change too.

At least in the short term, I think vendors should still have the capability to provide this guarantee without user opt-in by changing the default macro settings (switching it to a user opt-out).

I agree with both of these points.

I implemented these in the latest revision of https://reviews.llvm.org/D50652.

With https://reviews.llvm.org/D50652, by default, internal_linkage is NOT used and it is NOT safe to link TUs compiled with different versions of libc++. When _LIBCPP_ABI_HIDDEN_USE_INTERNAL_LINKAGE is defined, internal_linkage is used and it IS safe again to link TUs compiled with different versions of libc++.

I may be confused about the meaning of the patch, but it looks to me like that macro is choosing between two ways of making such TU-libc+±version-mixing safe: always_inline and internal_linkage. Which seems like a good thing, but distinct.

That’s the current implementation, just because we can’t get rid of always_inline without getting link errors. The contract implied by not defining _LIBCPP_ABI_HIDDEN_USE_INTERNAL_LINKAGE is still that TUs can’t be linked if they’ve been compiled with different versions of libc++. In the not-too-distant future, we can drop always_inline and replace it by something that allows odr-deduplication.

Louis

I support this change too.

At least in the short term, I think vendors should still have the capability to provide this guarantee without user opt-in by changing the default macro settings (switching it to a user opt-out).

I agree with both of these points.

I implemented these in the latest revision of https://reviews.llvm.org/D50652.

With https://reviews.llvm.org/D50652, by default, internal_linkage is NOT used and it is NOT safe to link TUs compiled with different versions of libc++. When _LIBCPP_ABI_HIDDEN_USE_INTERNAL_LINKAGE is defined, internal_linkage is used and it IS safe again to link TUs compiled with different versions of libc++.

I may be confused about the meaning of the patch, but it looks to me like that macro is choosing between two ways of making such TU-libc+±version-mixing safe: always_inline and internal_linkage. Which seems like a good thing, but distinct.

That’s the current implementation, just because we can’t get rid of always_inline without getting link errors. The contract implied by not defining _LIBCPP_ABI_HIDDEN_USE_INTERNAL_LINKAGE is still that TUs can’t be linked if they’ve been compiled with different versions of libc++. In the not-too-distant future, we can drop always_inline and replace it by something that allows odr-deduplication.

We chatted offline, but I forgot to update this cfe-dev post: this makes sense to me as long as we use a macro name that indicates the guarantee rather than the mechanism. (I believe Louis went with …_HIDE_FROM_ABI_PER_TU, which WFM.)