Why does libcxx include its own files this way?

Inside of libcxx, you can see #include being used when the library is clearly intending for it to find the file that #include “name” would find. I say it’s clear because, if any other file in the world were chosen, then the library would likely fail to compile.

It looks to me like the use of #include is an attempt to enforce a reservation on this name in the filesystem of the build machine. If true, then it’s kind of an overreach. I’d personally much rather if these sideways includes used #include “name” instead.

Is there a better explanation for why this is done this way?

Olivier

Inside of libcxx, you can see #include being used when the library is clearly intending for it to find the file that #include “name” would find. I say it’s clear because, if any other file in the world were chosen, then the library would likely fail to compile.

It looks to me like the use of #include is an attempt to enforce a reservation on this name in the filesystem of the build machine. If true, then it’s kind of an overreach. I’d personally much rather if these sideways includes used #include “name” instead.

Is there a better explanation for why this is done this way?

Olivier

+libcxx-dev, which got dropped
+Bruno

But you agree that it does, though, right?

Hmm. To be concrete, this is what I was worried about:

#!/bin/bash

run() { echo "$*" >&2; "$@" || exit 1; }
run mkdir -p quote
run cat </dev/null >quote/type_traits
run /usr/bin/printf "%s\n" \
    "#include <vector>" \
    "std::vector<int> g;" \
    > t.cpp
run clang -iquote quote -fsyntax-only t.cpp

which should give this output:

$ bash t.sh
mkdir -p quote
cat
/usr/bin/printf %s\n #include <vector> std::vector<int> g;
clang -iquote quote -fsyntax-only t.cpp

(i.e., no errors)

I thought if we switched to quoted "type_traits" everywhere I'd see a spew of errors. However, I'd forgotten that quoted includes check relative to the includer *first*, before hitting the search paths. So I was wrong here and this testcase doesn't expose a problem with your suggestion.

One other concern I have is that the funny #include_next dance between libc++ headers, clang headers, and libc (especially on Darwin-derived SDKs) continues work for modules. I kind of doubt your idea would cause a problem there since libc++ should be the outer most layer, but it's pretty subtle and I'd want to be sure. Bruno might be able to check that out for you if you make a patch.

Bruno, are there other concerns for modules as well?

Ok. If nobody stops me then I’ll make a patch. I’ve looked at it before and it’s a big patch. Would it be acceptable if I started with the Freestanding headers as a trial balloon?

Olivier

I’ve tested replacing #include with #include “name” for all names in the table https://en.cppreference.com/w/cpp/freestanding and all names in the transitive dependency closure in libcxx (e.g. __config), and except for a small gotcha in the /experimental subdirectory (where some need to become “…/name”) this passes all tests on Mac. Obviously this is a 145-file patch, a couple lines per.

Are you interested in this patch?

Olivier

It looks to me like the use of #include is an attempt to enforce a reservation on this name in the filesystem of the build machine.

It’s not enforcing a reservation on the header name. But it’s critically important that libc++'s includes are found in the include path, before almost all headers.
Otherwise our “#include_next” directives don’t work.

Additionally, Google wraps the STL headers with their own intermediate headers that add or adjust declarations as needed. This depends on our internal includes
being picked up first for #include <foo>. Your change would break that.

I’m not sure exactly what you’re trying to do here. Are you trying to make libc++ work when another STL is first on the include path?

Are you trying to make libc++ work when another STL is first on the include path?

Yes, this is a big part of the issue. I can’t practically replace all the host STLs, nor make all the host STLs grow accelerator support overnight. So I introduce my STL in namespace::std:: and under a <subdir/name> include path, if you recall, but then the libcxx self-includes go to the wrong place.

Google wraps the STL headers with their own intermediate headers that add or adjust declarations as needed

In a nutshell, that’s what we do too, but it’s convoluted and brittle in some cases.

Given your use-case long predates ours… it sounds like either a change like this one would have to live in our fork, or we choose to keep and maintain the redirection wrappers.

Olivier

Are you trying to make libc++ work when another STL is first on the include path?

Yes, this is a big part of the issue. I can’t practically replace all the host STLs, nor make all the host STLs grow accelerator support overnight. So I introduce my STL in namespace::std:: and under a <subdir/name> include path, if you recall, but then the libcxx self-includes go to the wrong place.

A couple of questions:

  1. why simt::std and not std::simt? ie. leading with the fact it’s a standard library.
  2. How important is it that you expose <simt/foo> to the user? Could <foo> act like a header that did #include_next <foo> #include <simt/foo>?

Google wraps the STL headers with their own intermediate headers that add or adjust declarations as needed

In a nutshell, that’s what we do too, but it’s convoluted and brittle in some cases.

Can you provide an example cases where it’s problematically brittle?

Are you trying to make libc++ work when another STL is first on the include path?

Yes, this is a big part of the issue. I can’t practically replace all the host STLs, nor make all the host STLs grow accelerator support overnight. So I introduce my STL in namespace::std:: and under a <subdir/name> include path, if you recall, but then the libcxx self-includes go to the wrong place.

A couple of questions:

  1. why `simt::std` and not `std::simt`? ie. leading with the fact it's a standard library.

I don’t have strong feelings about this. Do you think the alternative is simpler?

2) How important is it that you expose `<simt/foo>` to the user? Could `<foo>` act like a header that did `#include_next <foo> #include <simt/foo>`?

That could also work. Noting that Windows requires the kludge #include <../include/foo> instead of #include_next <foo>. (Potentially that could become just #include <foo> if the libcxx headers can be isolated in a subdirectory and use #include "name" when they mean to find each other, and not the host.)

Google wraps the STL headers with their own intermediate headers that add or adjust declarations as needed

In a nutshell, that’s what we do too, but it’s convoluted and brittle in some cases.
Can you provide an example cases where it's problematically brittle?

In hindsight, I was just repeating the previous point that it’s convoluted. It’s hard to keep it all straight in all the compiler scenarios we need to deal with, so there’s a lot of whack-a-mole play. Obviously testing is the only real answer to that, but one shouldn’t discount the discount one gets from simplicity.
Given your use-case long predates ours… it sounds like either a change like this one would have to live in our fork, or we choose to keep and maintain the redirection wrappers.

Olivier