Option to disable inline namespacing completely?

Hi,

I was wondering, would it be possible to add an option to disable inline namespacing completely (as a CMake
configuration flag) in libc++ for the sake of being able to use shorthand mangling and without having to resort to
handling it on IA64 mangler level. This has many use cases for example distributions of anything that includes libc++ as
one and only libc++ and does not allow non-vendor software to be installed. On an embedded system, assuming debug info
is generated, and given how common some of the debug data takes a very significant amount of space given the complex
definition of something like `std::__2::basic_string<...>` versus the short form (`std::string` having a shorthand
mangling is a godsend since it's 95% shorter, not an accurate figure but basically definitely above 90%).

Realistically it would be cool to just have shorthand manglings for inline namespaced versions of it but that's
something that has to be implemented in the mangler and is fairly distinct from mono-ABI cases where versioning is
undesirable due to impossibility of collisions.

I think I suggested something like that to Eric (disabling inline namespaces) though I'm not sure if there is any plans
to implement that in which case I may open a Clang differential that adds the option for the mangler to strip versioned
namespaces as this is simply an extremely useful option to have when shipping an entire OS with all software only being
available as provided by the vendor.

Thank you.
- Kristina

No, there isn’t.

I meant to add one, but I got blocked by having to fix the mangled names used under MSVC in iostream.cpp

/Eric

I’m confused here.
Why are you comparing std::string to `std::__2::basic_string<…" ?

std::__2::string is quite short.

– Marshall

The itanium specification provides special compressed mangled names for std::string when it’s not in an inline namespace [1]

Specifically, Ss vs St3__112basic_string

[1] https://itanium-cxx-abi.github.io/cxx-abi/abi.html#mangling-compression

The itanium specification provides special compressed mangled names for std::string when it’s not in an inline namespace [1]

Specifically, Ss vs St3__112basic_string

Correction… Ss vs NSt3__112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEE

Yes exactly, seems like an obvious win on a mono-ABI system, if preserving debug data
this saves a lot, not to mention symbol table strings. Also, I'm purely speculating here
but this would also speed up compilation and linking time by an insignificant margin, but
nevertheless, it's an improvement.

If the ABI ever needs to change, on those kinds of systems, it's easy enough with an OS
update.

Thanks.

Yes exactly, seems like an obvious win on a mono-ABI system, if preserving debug data
this saves a lot, not to mention symbol table strings. Also, I'm purely speculating here
but this would also speed up compilation and linking time by an insignificant margin, but
nevertheless, it's an improvement.

If the ABI ever needs to change, on those kinds of systems, it's easy enough with an OS
update.

Or, we could update the Itanium ABI so that the current mangling still works and is compatible with a new mangling which is shorter for libc++ (yet remains purposefully incompatible with libstdc++’s mangling) :slightly_smiling_face:

Yes changing this on IA64 mangler level would make more sense and leave room for compact mangling
for versioned ABI though I think I may need to pitch an RFC to the C++ committee or whoever is in
charge of the C++ IA64 ABI.

Thanks.

(It seems putting the mailing list in the CC field made it not go through so resending)

Yes changing this on IA64 mangler level would make more sense and leave room for compact mangling
for versioned ABI though I think I may need to pitch an RFC to the C++ committee or whoever is in
charge of the C++ IA64 ABI.

Thanks.

Yes exactly, seems like an obvious win on a mono-ABI system, if preserving debug data
this saves a lot, not to mention symbol table strings. Also, I'm purely speculating here
but this would also speed up compilation and linking time by an insignificant margin, but
nevertheless, it's an improvement.

If the ABI ever needs to change, on those kinds of systems, it's easy enough with an OS
update.

Or, we could update the Itanium ABI so that the current mangling still works and is compatible with a new mangling which is shorter for libc++ (yet remains purposefully incompatible with libstdc++’s mangling) :slightly_smiling_face:

Is it necessary/desirable to distinguish between implementations of the standard library at the Itanium ABI level? Wouldn't it be possible to define a compressed scheme for `std::__N` in full generality, and handle the ability to differentiate between implementations differently?

Louis

Well that was in my original suggestion, but I don't know who is in charge of IA64 mangling, I
don't think Clang developers can push a change like this forward without some RFC to whatever
committee maintains the IA64 ABI or am I wrong? I suppose libc++ ABI could be considered separate
enough in which case we just need to come up with a compatible scheme for "short" manglings that
is also guaranteed not to clash with the standard IA64 ABI.

Since you and Eric are mostly in charge of libcxxabi, could you propose a draft for a compatible
short mangling that would allow doing this as a breaking change (for unstable ABI versions, even
2+ since it's not stable yet unless Fuchsia has settled on it)? That way it's possible to add that
into the Clang mangler. I'm still not sure if this requires a blessing from whoever is in charge
of IA64 ABI, I think that would be a good idea to mention the draft to them at least.

Thanks.

Well that was in my original suggestion, but I don’t know who is in charge of IA64 mangling, I
don’t think Clang developers can push a change like this forward without some RFC to whatever
committee maintains the IA64 ABI or am I wrong? I suppose libc++ ABI could be considered separate
enough in which case we just need to come up with a compatible scheme for “short” manglings that
is also guaranteed not to clash with the standard IA64 ABI.

Since you and Eric are mostly in charge of libcxxabi, could you propose a draft for a compatible
short mangling that would allow doing this as a breaking change (for unstable ABI versions, even
2+ since it’s not stable yet unless Fuchsia has settled on it)? That way it’s possible to add that
into the Clang mangler. I’m still not sure if this requires a blessing from whoever is in charge
of IA64 ABI, I think that would be a good idea to mention the draft to them at least.

According to: https://itanium-cxx-abi.github.io/cxx-abi/

The primary discussion forum for the ABI is the GitHub repository. Please open a new issue for any new topic you want to discuss. Previously, the ABI was discussed on a mailing list, cxx-abi-dev, whose archives are still accessible.

CC’ing John since he’ll know more about this, seeing as how he’s the most active contributor to the repository :slight_smile:

This issue from Richard seems like a good starting point: https://github.com/itanium-cxx-abi/cxx-abi/issues/42

Thanks.

Yes exactly, seems like an obvious win on a mono-ABI system, if preserving debug data
this saves a lot, not to mention symbol table strings. Also, I’m purely speculating here
but this would also speed up compilation and linking time by an insignificant margin, but
nevertheless, it’s an improvement.

If the ABI ever needs to change, on those kinds of systems, it’s easy enough with an OS
update.

Or, we could update the Itanium ABI so that the current mangling still works and is compatible with a new mangling which is shorter for libc++ (yet remains purposefully incompatible with libstdc++’s mangling) :slightly_smiling_face:

Is it necessary/desirable to distinguish between implementations of the standard library at the Itanium ABI level? Wouldn’t it be possible to define a compressed scheme for std::__N in full generality, and handle the ability to differentiate between implementations differently?

What I had in mind was distinguishing std::__N as you say, and by construction that would distinguish between implementations because libc++ and libstdc++ use different namespaces.

I wrote a reply on the issue, however, I am not really known within the C++ community and
since you're a C++ standards committee member, your input would likely be of very high
value here as well. Could I ask you to chime in on the GitHub thread you linked
(the https://github.com/itanium-cxx-abi/cxx-abi/issues/42 one).

Thank you in advance.
- Kristina

Well that was in my original suggestion, but I don’t know who is in charge of IA64 mangling, I
don’t think Clang developers can push a change like this forward without some RFC to whatever
committee maintains the IA64 ABI or am I wrong? I suppose libc++ ABI could be considered separate
enough in which case we just need to come up with a compatible scheme for “short” manglings that
is also guaranteed not to clash with the standard IA64 ABI.

Since you and Eric are mostly in charge of libcxxabi, could you propose a draft for a compatible
short mangling that would allow doing this as a breaking change (for unstable ABI versions, even
2+ since it’s not stable yet unless Fuchsia has settled on it)? That way it’s possible to add that
into the Clang mangler. I’m still not sure if this requires a blessing from whoever is in charge
of IA64 ABI, I think that would be a good idea to mention the draft to them at least.

According to: https://itanium-cxx-abi.github.io/cxx-abi/

The primary discussion forum for the ABI is the GitHub repository. Please open a new issue for any new topic you want to discuss. Previously, the ABI was discussed on a mailing list, cxx-abi-dev, whose archives are still accessible.

CC’ing John since he’ll know more about this, seeing as how he’s the most active contributor to the repository :slight_smile:

This issue from Richard seems like a good starting point: https://github.com/itanium-cxx-abi/cxx-abi/issues/42

Thanks.

Yes exactly, seems like an obvious win on a mono-ABI system, if preserving debug data
this saves a lot, not to mention symbol table strings. Also, I’m purely speculating here
but this would also speed up compilation and linking time by an insignificant margin, but
nevertheless, it’s an improvement.

If the ABI ever needs to change, on those kinds of systems, it’s easy enough with an OS
update.

Or, we could update the Itanium ABI so that the current mangling still works and is compatible with a new mangling which is shorter for libc++ (yet remains purposefully incompatible with libstdc++’s mangling) :slightly_smiling_face:

Is it necessary/desirable to distinguish between implementations of the standard library at the Itanium ABI level? Wouldn’t it be possible to define a compressed scheme for std::__N in full generality, and handle the ability to differentiate between implementations differently?

What I had in mind was distinguishing std::__N as you say, and by construction that would distinguish between implementations because libc++ and libstdc++ use different namespaces.

But that would also prevent libstdc++ from ever using that mangling scheme (not that I care about libstdc++, but you know).

Louis

I wrote a reply on the issue, however, I am not really known within the C++ community and
since you’re a C++ standards committee member, your input would likely be of very high
value here as well. Could I ask you to chime in on the GitHub thread you linked
(the https://github.com/itanium-cxx-abi/cxx-abi/issues/42 one).

I opened a strawman proposal here: https://github.com/itanium-cxx-abi/cxx-abi/pull/69

This is naive, but that way folks can chime in and at least we have something concrete to work on.

Louis

Well that was in my original suggestion, but I don’t know who is in charge of IA64 mangling, I
don’t think Clang developers can push a change like this forward without some RFC to whatever
committee maintains the IA64 ABI or am I wrong? I suppose libc++ ABI could be considered separate
enough in which case we just need to come up with a compatible scheme for “short” manglings that
is also guaranteed not to clash with the standard IA64 ABI.

Since you and Eric are mostly in charge of libcxxabi, could you propose a draft for a compatible
short mangling that would allow doing this as a breaking change (for unstable ABI versions, even
2+ since it’s not stable yet unless Fuchsia has settled on it)? That way it’s possible to add that
into the Clang mangler. I’m still not sure if this requires a blessing from whoever is in charge
of IA64 ABI, I think that would be a good idea to mention the draft to them at least.

According to: https://itanium-cxx-abi.github.io/cxx-abi/

The primary discussion forum for the ABI is the GitHub repository. Please open a new issue for any new topic you want to discuss. Previously, the ABI was discussed on a mailing list, cxx-abi-dev, whose archives are still accessible.

CC’ing John since he’ll know more about this, seeing as how he’s the most active contributor to the repository :slight_smile:

This issue from Richard seems like a good starting point: https://github.com/itanium-cxx-abi/cxx-abi/issues/42

Thanks.

Yes exactly, seems like an obvious win on a mono-ABI system, if preserving debug data
this saves a lot, not to mention symbol table strings. Also, I’m purely speculating here
but this would also speed up compilation and linking time by an insignificant margin, but
nevertheless, it’s an improvement.

If the ABI ever needs to change, on those kinds of systems, it’s easy enough with an OS
update.

Or, we could update the Itanium ABI so that the current mangling still works and is compatible with a new mangling which is shorter for libc++ (yet remains purposefully incompatible with libstdc++’s mangling) :slightly_smiling_face:

Is it necessary/desirable to distinguish between implementations of the standard library at the Itanium ABI level? Wouldn’t it be possible to define a compressed scheme for std::__N in full generality, and handle the ability to differentiate between implementations differently?

What I had in mind was distinguishing std::__N as you say, and by construction that would distinguish between implementations because libc++ and libstdc++ use different namespaces.

But that would also prevent libstdc++ from ever using that mangling scheme (not that I care about libstdc++, but you know).

Not really, they can share the namespace if we don’t collide. They get the odd numbers we get the even ones? As long as we don’t cause ABI mismatches it’s fine.

Well that was in my original suggestion, but I don’t know who is in charge of IA64 mangling, I
don’t think Clang developers can push a change like this forward without some RFC to whatever
committee maintains the IA64 ABI or am I wrong? I suppose libc++ ABI could be considered separate
enough in which case we just need to come up with a compatible scheme for “short” manglings that
is also guaranteed not to clash with the standard IA64 ABI.

Since you and Eric are mostly in charge of libcxxabi, could you propose a draft for a compatible
short mangling that would allow doing this as a breaking change (for unstable ABI versions, even
2+ since it’s not stable yet unless Fuchsia has settled on it)? That way it’s possible to add that
into the Clang mangler. I’m still not sure if this requires a blessing from whoever is in charge
of IA64 ABI, I think that would be a good idea to mention the draft to them at least.

According to: https://itanium-cxx-abi.github.io/cxx-abi/

The primary discussion forum for the ABI is the GitHub repository. Please open a new issue for any new topic you want to discuss. Previously, the ABI was discussed on a mailing list, cxx-abi-dev, whose archives are still accessible.

CC’ing John since he’ll know more about this, seeing as how he’s the most active contributor to the repository :slight_smile:

This issue from Richard seems like a good starting point: https://github.com/itanium-cxx-abi/cxx-abi/issues/42

Thanks.

Yes exactly, seems like an obvious win on a mono-ABI system, if preserving debug data
this saves a lot, not to mention symbol table strings. Also, I’m purely speculating here
but this would also speed up compilation and linking time by an insignificant margin, but
nevertheless, it’s an improvement.

If the ABI ever needs to change, on those kinds of systems, it’s easy enough with an OS
update.

Or, we could update the Itanium ABI so that the current mangling still works and is compatible with a new mangling which is shorter for libc++ (yet remains purposefully incompatible with libstdc++’s mangling) :slightly_smiling_face:

Is it necessary/desirable to distinguish between implementations of the standard library at the Itanium ABI level? Wouldn’t it be possible to define a compressed scheme for std::__N in full generality, and handle the ability to differentiate between implementations differently?

What I had in mind was distinguishing std::__N as you say, and by construction that would distinguish between implementations because libc++ and libstdc++ use different namespaces.

But that would also prevent libstdc++ from ever using that mangling scheme (not that I care about libstdc++, but you know).

Not really, they can share the namespace if we don’t collide. They get the odd numbers we get the even ones? As long as we don’t cause ABI mismatches it’s fine.

Actually, I think the Itanium ABI provides a way to add vendor-specific tags to mangling:

A <[mangled-name](http://itanium-cxx-abi.github.io/cxx-abi/abi.html#mangle.mangled-name)> containing a period represents a vendor-specific version or portion of the entity named by the <[encoding](http://itanium-cxx-abi.github.io/cxx-abi/abi.html#mangle.encoding)> prior to the first period. There is no restriction on the characters that may be used in the suffix following the period.

So if libstdc++ wanted to use inline namespaces for versioning, they could do it with a bit of tooling support IIUC.

Re-reading this, I’m now wondering why we’re not piggy-backing on this mechanism to encode the version of libc++? It does seem like this mechanism was made explicitly for that use case.

Louis

Well that was in my original suggestion, but I don’t know who is in charge of IA64 mangling, I
don’t think Clang developers can push a change like this forward without some RFC to whatever
committee maintains the IA64 ABI or am I wrong? I suppose libc++ ABI could be considered separate
enough in which case we just need to come up with a compatible scheme for “short” manglings that
is also guaranteed not to clash with the standard IA64 ABI.

Since you and Eric are mostly in charge of libcxxabi, could you propose a draft for a compatible
short mangling that would allow doing this as a breaking change (for unstable ABI versions, even
2+ since it’s not stable yet unless Fuchsia has settled on it)? That way it’s possible to add that
into the Clang mangler. I’m still not sure if this requires a blessing from whoever is in charge
of IA64 ABI, I think that would be a good idea to mention the draft to them at least.

According to: https://itanium-cxx-abi.github.io/cxx-abi/

The primary discussion forum for the ABI is the GitHub repository. Please open a new issue for any new topic you want to discuss. Previously, the ABI was discussed on a mailing list, cxx-abi-dev, whose archives are still accessible.

CC’ing John since he’ll know more about this, seeing as how he’s the most active contributor to the repository :slight_smile:

This issue from Richard seems like a good starting point: https://github.com/itanium-cxx-abi/cxx-abi/issues/42

Thanks.

Yes exactly, seems like an obvious win on a mono-ABI system, if preserving debug data
this saves a lot, not to mention symbol table strings. Also, I’m purely speculating here
but this would also speed up compilation and linking time by an insignificant margin, but
nevertheless, it’s an improvement.

If the ABI ever needs to change, on those kinds of systems, it’s easy enough with an OS
update.

Or, we could update the Itanium ABI so that the current mangling still works and is compatible with a new mangling which is shorter for libc++ (yet remains purposefully incompatible with libstdc++’s mangling) :slightly_smiling_face:

Is it necessary/desirable to distinguish between implementations of the standard library at the Itanium ABI level? Wouldn’t it be possible to define a compressed scheme for std::__N in full generality, and handle the ability to differentiate between implementations differently?

What I had in mind was distinguishing std::__N as you say, and by construction that would distinguish between implementations because libc++ and libstdc++ use different namespaces.

But that would also prevent libstdc++ from ever using that mangling scheme (not that I care about libstdc++, but you know).

Not really, they can share the namespace if we don’t collide. They get the odd numbers we get the even ones? As long as we don’t cause ABI mismatches it’s fine.

Actually, I think the Itanium ABI provides a way to add vendor-specific tags to mangling:

A <[mangled-name](http://itanium-cxx-abi.github.io/cxx-abi/abi.html#mangle.mangled-name)> containing a period represents a vendor-specific version or portion of the entity named by the <[encoding](http://itanium-cxx-abi.github.io/cxx-abi/abi.html#mangle.encoding)> prior to the first period. There is no restriction on the characters that may be used in the suffix following the period.

So if libstdc++ wanted to use inline namespaces for versioning, they could do it with a bit of tooling support IIUC.

Re-reading this, I’m now wondering why we’re not piggy-backing on this mechanism to encode the version of libc++? It does seem like this mechanism was made explicitly for that use case.

It was added to bless the existing practice of vendors who want to do things like emit various private symbols associated with an ABI entity without invading on the set of names that the ABI could theoretically want to use someday. It is not intended for symbols whose mangling is controlled by the ABI, and in fact its motivation precludes the ABI from intruding on that space.

I agree that it would be nice to use the abbreviations. We could add some sort of mode prefix to the symbol that says that std in all the standard abbreviations actually refers to std::__1, and then anything that doesn’t obey that — because it’s differently-versioned or part of the small subset that’s not in std::__1 — wouldn’t get to use it. We’d need some unambiguous way to decide whether to use the prefix; a reasonable heuristic would be “what’s the exact namespace of the first declaration we try to mangle that’s ultimately within std:: ?”.

Of course, actually enabling that would be totally ABI-breaking.

John.

I think the ABI breaking part goes without saying, it’s the reason we have unstable (currently sitting at version 2) ABI, it’s already massively incompatible with ABI version 1 due to different layouts of things like std::basic_string (due to SBO changes).

As far as enabling it goes, currently to use version 2 of the ABI (which is still unstable), one has to explicitly specify that when building libc++, said libc++ is not going to be ABI compatible with anything earlier than it. Currently one can simulate that by enabling LIBCXX_ABI_UNSTABLE without bumping up the ABI version which basically toggles ABI v2 without changing the versioning prefix from __1 causing anything linked to it to break during early initialization.

Regarding the heuristic, yes that was my suggestion since this can be handled in the Clang mangler, meaning we could simply check if it’s __1 (the reserved “stable” inline namespace, which is currently in use, as such, short mangling would be disabled during IR CodeGen). Otherwise if the namespace is a reserved (as discussed previously regarding numeric versioning being reserved for libc++'s own use while) and is in the form of __N with N above 1, short mangling could be applied, for example with LIBCXX_ABI_VERSION=2 (Currently in use by Fuchsia, although they do not mind breaking changes so far, as explicitly stated during discussions of marking 2 as “stable”).

Thank you.

FWIW, I think it may be worth moving this discussion over to https://github.com/itanium-cxx-abi/cxx-abi/pull/69 since
it's kind of out of scope of libc++(abi) alone.