RFC: A mechanism for importing a declaration only when it exists

Hi,

The C++ Standard Library has multiple headers of the form <cfoobar>, which are
basically importing declarations from the corresponding <foobar.h> header into
namespace std. For example, <cstdio> basically imports all the declarations from
<stdio.h> into namespace std:

    namespace std {
        using ::FILE;
        using ::fpos_t;
        using ::size_t;
        ...
    }

When porting libc++ to new platforms, especially more embedded-ish platforms,
it is very common that some declarations are not provided by the underlying
C Standard Library. This leads to having to detect what platform we're on,
and then to conditionally exclude some declarations:

    namespace std {
    #if !defined(SOME_WEIRD_PLATFORM)
        using ::FILE;
        using ::fpos_t;
    #endif
        using ::size_t;
    }

Unfortunately, different platforms often offer slightly different subsets, so
these #ifs quickly become difficult to maintain. Trying to find common themes
for excluding declarations (e.g. #if !defined(_LIBCPP_HAS_NO_FILE)) is vain,
because the subset provided by a platform is often arbitrary. For example, I've
seen platforms where malloc() and free() were not provided, however operator new
was -- so trying to carve out something like _LIBCPP_HAS_NO_ALLOCATION wouldn't
really make sense.

Given the difficulty of manually excluding such using declarations, I came to
the conclusion that what we wanted was often something like (pseudocode):

    namespace std {
    #if __has_declaration(::FILE)
        using ::FILE;
    #endif

    #if __has_declaration(::fpos_t)
        using ::fpos_t;
    #endif

    #if __has_declaration(::size_t)
        using ::size_t;
    #endif

        ...
    }

Basically, we want to import each declaration into namespace std only if the
global namespace has such a declaration.

Now, I understand this raises several questions. Of course, this couldn't be
done at the preprocessor level because I don't think we can know whether a
declaration exists at that time. However, I was curious to ask on this list
whether someone could think of a reasonable way to solve this problem.

Having some mechanism for doing this would be a huge life improvement for libc++.

Cheers,
Louis

I agree that something like this would be incredibly useful. I wonder if it's sufficient to provide a built-in type trait and write something like:

using FILE = std::conditional<__type_exists_v<::FILE>, ::FILE, void>;

(With some built-in equivalent of std::conditional that does short-circuit evaluation) You'd still have the std::FILE type, but it would be void on platforms where it didn't exist. Would this be sufficient?

David

std::conditional should already not evaluate the “other” “branch” depending on the condition. I’m not exactly sure about what you mean by short-circuiting otherwise.

I like this builtin idea, however, there is a slight catch: what the argument for __builtin_has_type or __builtin_type_exists is, syntactically? A name? An elaborated name? It will end up having to be a sort of “super unevaluated context” and that might make parsing it a bit of a hell…

All type_traits so far expect their template argument T to be a type.

In addition, what happens if your condition evaluates to:
using FILE = void;

This still makes “std::FILE” a thing, which will cause weird issues later on, unless the rest of the library is defended against each of these cases. Which seems even more tedious work.

David Chisnall via cfe-dev <cfe-dev@lists.llvm.org> ezt írta (időpont: 2020. jún. 26., P, 11:34):

Hi,

The C++ Standard Library has multiple headers of the form , which are
basically importing declarations from the corresponding <foobar.h> header into
namespace std. For example, basically imports all the declarations from
<stdio.h> into namespace std:

namespace std {
using ::FILE;
using ::fpos_t;
using ::size_t;

}

When porting libc++ to new platforms, especially more embedded-ish platforms,
it is very common that some declarations are not provided by the underlying
C Standard Library. This leads to having to detect what platform we’re on,
and then to conditionally exclude some declarations:

namespace std {
#if !defined(SOME_WEIRD_PLATFORM)
using ::FILE;
using ::fpos_t;
#endif
using ::size_t;
}

Unfortunately, different platforms often offer slightly different subsets, so
these #ifs quickly become difficult to maintain. Trying to find common themes
for excluding declarations (e.g. #if !defined(_LIBCPP_HAS_NO_FILE)) is vain,
because the subset provided by a platform is often arbitrary. For example, I’ve
seen platforms where malloc() and free() were not provided, however operator new
was – so trying to carve out something like _LIBCPP_HAS_NO_ALLOCATION wouldn’t
really make sense.

Given the difficulty of manually excluding such using declarations, I came to
the conclusion that what we wanted was often something like (pseudocode):

namespace std {
#if __has_declaration(::FILE)
using ::FILE;
#endif

#if __has_declaration(::fpos_t)
using ::fpos_t;
#endif

#if __has_declaration(::size_t)
using ::size_t;
#endif


}

Basically, we want to import each declaration into namespace std only if the
global namespace has such a declaration.

Now, I understand this raises several questions. Of course, this couldn’t be
done at the preprocessor level because I don’t think we can know whether a
declaration exists at that time. However, I was curious to ask on this list
whether someone could think of a reasonable way to solve this problem.

Having some mechanism for doing this would be a huge life improvement for libc++.

Do you always want to guard a namespace & scope using-declaration? We could probably support an attribute on such declarations that would result in them having no effect if no names are found – or perhaps being invalid to reference if no names are found. (Otherwise, Clang already has some support for MSVC’s __if_exists, but it’s got some deep semantic problems, and I don’t think we should recommend its use.)

I agree with Whisperity that having “FILE = void” on some platforms would be a cure worse than the disease.
It seems to me that what you want is less like an alias declaration and more like a using-declaration:

namespace std {
__using_if_exists ::FILE;
__using_if_exists ::fopen;
}

This syntax would also match Louis’s original intent better, since the thing it’s replacing is literally a using-declaration, not an alias declaration.
(I don’t know off the top of my head whether there’s a visible difference between using ::T and using T = ::T. I bet Richard knows. ;))

In any case, I concur with Louis that this seems like it’d have to be done at the C++ level, not the preprocessor level.

The semantics of my hypothetical “__using_if_exists QUALNAME;” declaration are that it does a name lookup on QUALNAME, and then if the lookup fails it does nothing, and if the lookup succeeds it acts like a regular using declaration. This feels like it should be super easy to implement as a compiler extension, right?
Bikeshed: __using_if_exists, __using_or_ignoring, __using_if_present, __using_unless_absent, __using_maybe_absent.

But, doesn’t libc++ also need to work with GCC and maybe other compilers too? So won’t you need to convince those compilers to implement whatever you pick here, or else keep the existing preprocessor hacks around forever anyway?

–Arthur

That seems like a great approach.

One question tho, can we make this publically available?

In my case, char16_t and char32_t are not builtin types, they're defined in uchar.h, and the workarounds I'm currently using are pretty hackish, so it'd be great if there was a `__type_exists(char16_t)` operator or maybe `__type_matches(char16_t, uint_least16_t)`, preferably that was part of the preprocessor.

Now, I understand this raises several questions. Of course, this couldn’t be
done at the preprocessor level because I don’t think we can know whether a
declaration exists at that time. However, I was curious to ask on this list
whether someone could think of a reasonable way to solve this problem.

Having some mechanism for doing this would be a huge life improvement for libc++.

I agree that something like this would be incredibly useful. I wonder
if it’s sufficient to provide a built-in type trait and write something
like:

using FILE = std::conditional<__type_exists_v<::FILE>, ::FILE, void>;

(With some built-in equivalent of std::conditional that does
short-circuit evaluation) You’d still have the std::FILE type, but it
would be void on platforms where it didn’t exist.

I agree with Whisperity that having “FILE = void” on some platforms would be a cure worse than the disease.
It seems to me that what you want is less like an alias declaration and more like a using-declaration:

Indeed, I don’t think FILE = void is a viable path forward. Also, it wouldn’t work for things that aren’t types, which is actually the vast majority of such using declarations (memcpy & friends, etc).

namespace std {
__using_if_exists ::FILE;
__using_if_exists ::fopen;
}

This syntax would also match Louis’s original intent better, since the thing it’s replacing is literally a using-declaration, not an alias declaration.
(I don’t know off the top of my head whether there’s a visible difference between using ::T and using T = ::T. I bet Richard knows. ;))

In any case, I concur with Louis that this seems like it’d have to be done at the C++ level, not the preprocessor level.

The semantics of my hypothetical “__using_if_exists QUALNAME;” declaration are that it does a name lookup on QUALNAME, and then if the lookup fails it does nothing, and if the lookup succeeds it acts like a regular using declaration. This feels like it should be super easy to implement as a compiler extension, right?

Yes, that’s exactly what I was asking. I wondered whether folks could spot a fundamental issue with doing this, but it looks like perhaps there is none.

Bikeshed: __using_if_exists, __using_or_ignoring, __using_if_present, __using_unless_absent, __using_maybe_absent.

But, doesn’t libc++ also need to work with GCC and maybe other compilers too? So won’t you need to convince those compilers to implement whatever you pick here, or else keep the existing preprocessor hacks around forever anyway?

AFAICT, libc++ works with recent-ish GCCs and Clang, that’s it. There’s been some work for it to work on other compilers, but none of it is maintained, and all of it is probably broken – as far as I’m aware.

If we went down that route, I’d kindly ask GCC to implement the same builtin, and if they say no, I could always resort to:

#if defined(GNUC)

define _LIBCPP_USING_IF_EXISTS(…) using VA_ARGS

#else

define _LIBCPP_USING_IF_EXISTS(…) __using_if_exists VA_ARGS

#endif

Or whatever form that ends up taking. I might have to keep existing workarounds for a little bit, but it would be reasonable to require that this extension be supported in order to add support for a new platform. Otherwise the headers just become a huge spaghetti.

Louis

Hi,

The C++ Standard Library has multiple headers of the form , which are
basically importing declarations from the corresponding <foobar.h> header into
namespace std. For example, basically imports all the declarations from
<stdio.h> into namespace std:

namespace std {
using ::FILE;
using ::fpos_t;
using ::size_t;

}

When porting libc++ to new platforms, especially more embedded-ish platforms,
it is very common that some declarations are not provided by the underlying
C Standard Library. This leads to having to detect what platform we’re on,
and then to conditionally exclude some declarations:

namespace std {
#if !defined(SOME_WEIRD_PLATFORM)
using ::FILE;
using ::fpos_t;
#endif
using ::size_t;
}

Unfortunately, different platforms often offer slightly different subsets, so
these #ifs quickly become difficult to maintain. Trying to find common themes
for excluding declarations (e.g. #if !defined(_LIBCPP_HAS_NO_FILE)) is vain,
because the subset provided by a platform is often arbitrary. For example, I’ve
seen platforms where malloc() and free() were not provided, however operator new
was – so trying to carve out something like _LIBCPP_HAS_NO_ALLOCATION wouldn’t
really make sense.

Given the difficulty of manually excluding such using declarations, I came to
the conclusion that what we wanted was often something like (pseudocode):

namespace std {
#if __has_declaration(::FILE)
using ::FILE;
#endif

#if __has_declaration(::fpos_t)
using ::fpos_t;
#endif

#if __has_declaration(::size_t)
using ::size_t;
#endif


}

Basically, we want to import each declaration into namespace std only if the
global namespace has such a declaration.

Now, I understand this raises several questions. Of course, this couldn’t be
done at the preprocessor level because I don’t think we can know whether a
declaration exists at that time. However, I was curious to ask on this list
whether someone could think of a reasonable way to solve this problem.

Having some mechanism for doing this would be a huge life improvement for libc++.

Do you always want to guard a namespace & scope using-declaration? We could probably support an attribute on such declarations that would result in them having no effect if no names are found – or perhaps being invalid to reference if no names are found. (Otherwise, Clang already has some support for MSVC’s __if_exists, but it’s got some deep semantic problems, and I don’t think we should recommend its use.)

Yes, as far as I can tell, I only want to apply this to using declarations inside namespace std. It’s a pretty constrained use case.

I think the better semantics would be to have no effect if no names are found, since that’s what’s closest in effect to having an #ifdef in front that removes it. The resulting error is also quite straightforward, i.e. the compiler just says “oops, no such member in namespace std”. We could potentially augment the diagnostic to say “there’s no such member in namespace std BECAUSE we tried importing it and it wasn’t in the global namespace”.

Louis

Now, I understand this raises several questions. Of course, this couldn’t be
done at the preprocessor level because I don’t think we can know whether a
declaration exists at that time. However, I was curious to ask on this list
whether someone could think of a reasonable way to solve this problem.

Having some mechanism for doing this would be a huge life improvement for libc++.

I agree that something like this would be incredibly useful. I wonder
if it’s sufficient to provide a built-in type trait and write something
like:

using FILE = std::conditional<__type_exists_v<::FILE>, ::FILE, void>;

(With some built-in equivalent of std::conditional that does
short-circuit evaluation) You’d still have the std::FILE type, but it
would be void on platforms where it didn’t exist.

I agree with Whisperity that having “FILE = void” on some platforms would be a cure worse than the disease.
It seems to me that what you want is less like an alias declaration and more like a using-declaration:

namespace std {
__using_if_exists ::FILE;
__using_if_exists ::fopen;
}

This syntax would also match Louis’s original intent better, since the thing it’s replacing is literally a using-declaration, not an alias declaration.
(I don’t know off the top of my head whether there’s a visible difference between using ::T and using T = ::T. I bet Richard knows. ;))

I do, and there is, but I’m not sure if it’ll be relevant here. “using T = ::T;” introduces a typedef-name, whereas “using ::T;” preserves the kind of the name. You can’t refer to a name introduced by typedef-name with an elaborated-type-specifier (eg, “struct std::FILE” would be invalid with the “using T = ::T;” approach even if ::FILE is a struct), and you don’t get the “non-types hide types” behavior with a typedef-name (which might have some impact on a hypothetical std::stat – but that’s not the only problem in that scenario).

Hi,

The C++ Standard Library has multiple headers of the form , which are
basically importing declarations from the corresponding <foobar.h> header into
namespace std. For example, basically imports all the declarations from
<stdio.h> into namespace std:

namespace std {
using ::FILE;
using ::fpos_t;
using ::size_t;

}

When porting libc++ to new platforms, especially more embedded-ish platforms,
it is very common that some declarations are not provided by the underlying
C Standard Library. This leads to having to detect what platform we’re on,
and then to conditionally exclude some declarations:

namespace std {
#if !defined(SOME_WEIRD_PLATFORM)
using ::FILE;
using ::fpos_t;
#endif
using ::size_t;
}

Unfortunately, different platforms often offer slightly different subsets, so
these #ifs quickly become difficult to maintain. Trying to find common themes
for excluding declarations (e.g. #if !defined(_LIBCPP_HAS_NO_FILE)) is vain,
because the subset provided by a platform is often arbitrary. For example, I’ve
seen platforms where malloc() and free() were not provided, however operator new
was – so trying to carve out something like _LIBCPP_HAS_NO_ALLOCATION wouldn’t
really make sense.

Given the difficulty of manually excluding such using declarations, I came to
the conclusion that what we wanted was often something like (pseudocode):

namespace std {
#if __has_declaration(::FILE)
using ::FILE;
#endif

#if __has_declaration(::fpos_t)
using ::fpos_t;
#endif

#if __has_declaration(::size_t)
using ::size_t;
#endif


}

Basically, we want to import each declaration into namespace std only if the
global namespace has such a declaration.

Now, I understand this raises several questions. Of course, this couldn’t be
done at the preprocessor level because I don’t think we can know whether a
declaration exists at that time. However, I was curious to ask on this list
whether someone could think of a reasonable way to solve this problem.

Having some mechanism for doing this would be a huge life improvement for libc++.

Do you always want to guard a namespace & scope using-declaration? We could probably support an attribute on such declarations that would result in them having no effect if no names are found – or perhaps being invalid to reference if no names are found. (Otherwise, Clang already has some support for MSVC’s __if_exists, but it’s got some deep semantic problems, and I don’t think we should recommend its use.)

Yes, as far as I can tell, I only want to apply this to using declarations inside namespace std. It’s a pretty constrained use case.

I think the better semantics would be to have no effect if no names are found, since that’s what’s closest in effect to having an #ifdef in front that removes it.

That semantic model may be a better fit for your use case, but it’s not really a great fit for C++ more broadly. Consider a case such as:

template int f() {
[[clang::its_ok_if_this_doesnt_exist]] using T::x;
return x;
}

During the phase 1 name lookup for ‘x’ in the return-statement, we find the local (but dependent) using-declaration and bind to that. During template instantiation, it would be challenging to pretend that that ‘x’ doesn’t exist and fall back on an enclosing name ‘x’. The semantic model that fits better for this case would be to say that name lookup stops at the using-declaration (and doesn’t consider enclosing scopes), but then the result is an error because the name ‘x’ didn’t resolve to anything. (Roughly as if a failed using-declaration with the attribute in question behaves as if it were =delete’d or 'requires false’d in some sense.)

If we’re going to add an extension for this, I’d prefer that it’s something that can be applied soundly in general, rather than introducing another situation like the one for __if_exists where we support it only partially and only in certain contexts.

It’s not clear to me whether this approach would actually be bad for your use case, or merely different from what you get with #ifdefs. An error on use of std::fopen that says “can’t refer to std::fopen because there is no declaration named ‘fopen’ in the global namespace” or similar seems like it might actually result in a better user experience than one that says “no name ‘fopen’ in namespace ‘std’” or similar.

I think the better semantics would be to have no effect if no names are found, since that’s what’s closest in effect to having an #ifdef in front that removes it.

That semantic model may be a better fit for your use case, but it’s not really a great fit for C++ more broadly. Consider a case such as:

template int f() {
[[clang::its_ok_if_this_doesnt_exist]] using T::x;
return x;
}

During the phase 1 name lookup for ‘x’ in the return-statement, we find the local (but dependent) using-declaration and bind to that. During template instantiation, it would be challenging to pretend that that ‘x’ doesn’t exist and fall back on an enclosing name ‘x’. The semantic model that fits better for this case would be to say that name lookup stops at the using-declaration (and doesn’t consider enclosing scopes), but then the result is an error because the name ‘x’ didn’t resolve to anything.

IIUC, you’re saying that two-phase lookup inside a C++ template requires that the compiler always be certain what names are introduced where. Therefore, a declaration that (dependently) “might or might not” introduce a name is incompatible with the first phase of two-phase lookup. Did I get that roughly correct?

If so, I still think it would be simpler to patch __using_if_exists N::x; so that it would simply fail to compile at all if the name N::x was dependent.

AFAICT, using T::x; (with dependent T) is only ever well-formed inside a class body, which is not Louis’s use-case.
https://godbolt.org/z/JvdCmd

It would be quite reasonable for Clang to reject __using_if_exists in this context (but still permit it at file or namespace scope).

AFAICT, using T::x; can never be dependent at function scope, either,
https://godbolt.org/z/gSvR_n

so you could permit it even at function scope… although I see no reason to. Louis wants it only at namespace scope, where AFAICT this whole two-phase-lookup thing poses no problem at all.

If we’re going to add an extension for this, I’d prefer that it’s something that can be applied soundly in general, rather than introducing another situation like the one for __if_exists where we support it only partially and only in certain contexts.

using-declarations at class scope already behave radically differently from using-declarations in any other kind of scope. So I don’t see anything inconsistent or unsound about saying “This __using_if_exists thing is able to stand in only for a non-member using-declaration.” (I don’t know what the Standardese for that is, but there must be some, given how radically “member using-declarations” differ semantically from “non-member using-declarations.”)

–Arthur