_LIBCPP_INLINE_VISIBILITY and std::string::length

Hello!

I’m just curious about what are the reasons to forcing inlining of that sort of functions like std::string::length? Precluding them to be included into libc++ binary breaks linkage of the following snippet of code

int main(int ARGC, char *ARGV[])() {

std::vectorstd::string pray = { “I”, “will”, “not”, “aim”, “for”, “the”, “head” };
std::vector<size_t> lengths;

std::transform(pray.begin(), pray.end(), std::back_inserter(lengths), std::mem_fn(&std::string::size));

return 0;

}

with the error:

Undefined symbols for architecture x86_64:
“std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator >::length() const”, referenced from:
_main in assignment-F0ysIg.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Best regards,
Alexey Kudinkin

I believe this is due to a poor interaction in the compiler between extern templates and __attribute__ ((__always_inline__)). If std::string is not declared extern template, then the compiler will outline a size() member and you won't get this link error.

In my opinion, clang should not assume that extern templates have definitions for inlined members, especially those marked always_inline, and the fact that it does is a clang bug resulting in the link error you see.

The rationale for the use of always_inline in libc++ is to control the ABI of libc++. In the past I have watched compilers use different heuristics from release to release on making the inline/outline decision. This can cause code to be silently added to and removed from a dylib. With the use of always_inline, I am telling the compiler to never add that code to the libc++.dylib binary.

Each of the macros defined and used by libc++ can be overridden.

_LIBCPP_INLINE_VISIBILITY controls how an inlined function is attributed and defaults to:

#ifndef _LIBCPP_INLINE_VISIBILITY
#define _LIBCPP_INLINE_VISIBILITY __attribute__ ((__visibility__("hidden"), __always_inline__))
#endif

You can turn this off with:

-D_LIBCPP_INLINE_VISIBILITY=""

And extern templates are done with:

#ifndef _LIBCPP_EXTERN_TEMPLATE
#define _LIBCPP_EXTERN_TEMPLATE(...) extern template __VA_ARGS__;
#endif

This latter one is more difficult to "turn off". The incantation is:

-D'_LIBCPP_EXTERN_TEMPLATE(...)='

Using either (or both) of these workarounds will silence your link error. But a bug report against clang might be a better long term solution.

Howard

Correction, only the second work around works. clang assumes extern templates have inline member definitions whether they are marked always_inline or not.

Howard

Hello!

I'm just curious about what are the reasons to forcing inlining of that sort of functions like std::string::length? Precluding them to be included into libc++ binary breaks linkage of the following snippet of code

int main(int ARGC, char *ARGV[])() {

std::vector<std::string> pray = { "I", "will", "not", "aim", "for", "the", "head" };
std::vector<size_t> lengths;

std::transform(pray.begin(), pray.end(), std::back_inserter(lengths), std::mem_fn(&std::string::size));

For what it's worth, IIRC this code is not required to work, because you are not generally allowed to take the address of functions from the standard library.

I believe this is due to a poor interaction in the compiler between extern templates and __attribute__ ((__always_inline__)). If std::string is not declared extern template, then the compiler will outline a size() member and you won't get this link error.

always_inline doesn't stop the dylib from exporting that function symbol. It's the hidden visibility that does that.

In my opinion, clang should not assume that extern templates have definitions for inlined members, especially those marked always_inline, and the fact that it does is a clang bug resulting in the link error you see.

Every member of a class template is implicitly inline. We are not going to completely break explicit instantiation declarations.

We do already treat always_inline as a special case, and I'm not sure why it's not applying here. That does seem like a bug.

The rationale for the use of always_inline in libc++ is to control the ABI of libc++. In the past I have watched compilers use different heuristics from release to release on making the inline/outline decision. This can cause code to be silently added to and removed from a dylib. With the use of always_inline, I am telling the compiler to never add that code to the libc++.dylib binary.

This is an incredibly brute-force way of accomplishing this goal, and it causes major problems downstream, e.g. with debugging programs that use libc++.

Have you considered only explicitly instantiating the functions you actually want to show up in the library instead of explicitly instantiating the entire class?

John.

Hello!

I'm just curious about what are the reasons to forcing inlining of that sort of functions like std::string::length? Precluding them to be included into libc++ binary breaks linkage of the following snippet of code

int main(int ARGC, char *ARGV[])() {

std::vector<std::string> pray = { "I", "will", "not", "aim", "for", "the", "head" };
std::vector<size_t> lengths;

std::transform(pray.begin(), pray.end(), std::back_inserter(lengths), std::mem_fn(&std::string::size));

For what it's worth, IIRC this code is not required to work, because you are not generally allowed to take the address of functions from the standard library.

Actually I think this was more true in C++03 than in C++11. The specification says the vendor can add default arguments and/or overloads to a non-virtual member function. So you can't assume the member function's precise signature. Now with variadic templates it is a lot easier to take the address without assuming the precise signature of the member function.

The rationale for the use of always_inline in libc++ is to control the ABI of libc++. In the past I have watched compilers use different heuristics from release to release on making the inline/outline decision. This can cause code to be silently added to and removed from a dylib. With the use of always_inline, I am telling the compiler to never add that code to the libc++.dylib binary.

This is an incredibly brute-force way of accomplishing this goal, and it causes major problems downstream, e.g. with debugging programs that use libc++.

Have you considered only explicitly instantiating the functions you actually want to show up in the library instead of explicitly instantiating the entire class?

No, I hadn't considered that. I'll give that some thought. It sounds tedious but it may be worth it.

Thanks,
Howard

Sorry, for the not so prompt response.