A question about neglect of 'include-fixed' using libclang with GCC toolchain only

Hi,
I’m working on a Linux environment with a self-configured GCC. This GCC is built from scratch, without any patch from distro vendor. Then I’m trying to use Clangd alone on this platform, only to find that Clangd would emit error about missing include header ‘limits.h’.
The error occurs at /usr/include/limits.h, where this header is using #include_next <limits.h> to include next limits.h in system headers.
I tried to provide Clangd with correct -resource-dir but nothing works. I noticed that this “missing” header ‘limits.h’ is located in this include-fixed dir. I then did some research and found this trying to explain why GCC would like to put some headers in this quirk dir.
I’ve also looked into GCC build script of CentOS. It looks like I should copy these limits.h files from include-fixed to include dir directly. But even without this step, GCC that built from scratch still works on my system. I found that this is because GCC would search this include-fixed dir. Then it comes to me that maybe Clang did not handle this case:
I did a rough look on Clang Driver, and It appears to me that clang/lib/Driver/ToolChains/Linux.cpp:AddClangSystemIncludeArgs is where Clang injects search dirs. (See here on trunk).

   // Add 'include' in the resource directory, which is similar to
   // GCC_INCLUDE_DIR (private headers) in GCC. Note: the include directory
   // contains some files conflicting with system /usr/include. musl systems
   // prefer the /usr/include copies which are more relevant.
   SmallString<128> ResourceDirInclude(D.ResourceDir);
   llvm::sys::path::append(ResourceDirInclude, "include");
   if (!DriverArgs.hasArg(options::OPT_nobuiltininc) &&
       (!getTriple().isMusl() || DriverArgs.hasArg(options::OPT_nostdlibinc)))
     addSystemInclude(DriverArgs, CC1Args, ResourceDirInclude);

The logic here did not handle the case of include-fixed. I patched this code like this and rebuilt Clangd, then it works like a charm.

  SmallString<128> ResourceDirInclude(D.ResourceDir);
  llvm::sys::path::append(ResourceDirInclude, "include");
  SmallString<128> ResourceDirIncludeFixed(D.ResourceDir);
  llvm::sys::path::append(ResourceDirIncludeFixed, "include-fixed");
  if (!DriverArgs.hasArg(options::OPT_nobuiltininc) &&
      (!getTriple().isMusl() || DriverArgs.hasArg(options::OPT_nostdlibinc))) {
        addSystemInclude(DriverArgs, CC1Args, ResourceDirInclude);
        addSystemInclude(DriverArgs, CC1Args, ResourceDirIncludeFixed);
      }

So my question is, can anyone explain the reason behind this? I don’t know if this is by intention. If not, is this the correct way to fix this?

My environment detail is as follows:
OS: CentOS 7 with GCC 7.5 built from source.

$ gcc --print-search-dirs
install: /usr/local/bin/../lib/gcc/x86_64-pc-linux-gnu/7.5.0/
$ gcc -v
gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/local/bin/../libexec/gcc/x86_64-pc-linux-gnu/7.5.0/lto-wrapper
Target: x86_64-pc-linux-gnu
Configured with: ../gcc-7.5.0/configure --prefix=/home/qspace/install_tools/gcc7/install --with-dwarf-2 --disable-libstdcxx-dual-abi
Thread model: posix
gcc version 7.5.0 (GCC)

Clangd: Built from source, without any other components.
Running test:

// test.cpp
#include <limits.h>
$ clangd --resource-dir=/usr/local/lib/gcc/x86_64-pc-linux-gnu/7.5.0/ --check=test.cpp

Without my patch, it emits:

E[01:14:35.533] [pp_file_not_found] Line 1: in included file: 'limits.h' file not found
I[01:14:35.536] All checks completed, 1 errors

After my patch:

I[01:15:13.054] All checks completed, 0 errors

I don’t have any insights about the built-in include paths that clang chooses by default, but please note that clangd has a --query-driver command-line option (discussed in Troubleshooting, see also clangd --help for details) which can be used to get it to query an external compiler (such as gcc) for its built-in include paths and use those.

Well, I don’t know if I’m misunderstanding something, but command like this

clangd --resource-dir=/usr/local/lib/gcc/x86_64-pc-linux-gnu/7.5.0/ --check=test.cpp --query-driver=/usr/local/bin/g++

doesn’t work for me.

PS: Path structure:

$ ll /usr/local/lib/gcc/x86_64-pc-linux-gnu/7.5.0/ 
total 3208
drwxr-xr-x 3 root root    4096 Dec 23  2020 32
-rwxr-xr-x 1 root root    2408 Oct 26  2020 crtbegin.o
-rwxr-xr-x 1 root root    2776 Oct 26  2020 crtbeginS.o
-rwxr-xr-x 1 root root    2928 Oct 26  2020 crtbeginT.o
-rwxr-xr-x 1 root root    1163 Oct 26  2020 crtend.o
-rwxr-xr-x 1 root root    1163 Oct 26  2020 crtendS.o
-rwxr-xr-x 1 root root    1488 Oct 26  2020 crtfastmath.o
-rwxr-xr-x 1 root root    1488 Oct 26  2020 crtprec32.o
-rwxr-xr-x 1 root root    1496 Oct 26  2020 crtprec64.o
-rwxr-xr-x 1 root root    1488 Oct 26  2020 crtprec80.o
drwxr-xr-x 2 root root    4096 Dec 23  2020 finclude
drwxr-xr-x 6 root root    4096 Dec 23  2020 include
drwxr-xr-x 4 root root    4096 Dec 23  2020 include-fixed
drwxr-xr-x 3 root root    4096 Dec 23  2020 install-tools
-rwxr-xr-x 1 root root   59764 Oct 26  2020 libcaf_single.a
-rwxr-xr-x 1 root root     965 Oct 23  2020 libcaf_single.la
-rwxr-xr-x 1 root root 3031566 Oct 26  2020 libgcc.a
-rwxr-xr-x 1 root root   53282 Oct 26  2020 libgcc_eh.a
-rwxr-xr-x 1 root root   62512 Oct 26  2020 libgcov.a
drwxr-xr-x 3 root root    4096 Dec 23  2020 plugin
$ ll /usr/local/lib/gcc/x86_64-pc-linux-gnu/7.5.0/include-fixed/
total 24
-rwxr-xr-x 1 root root  750 Oct 23  2020 README
-rwxr-xr-x 1 root root 6089 Oct 23  2020 limits.h
drwxr-xr-x 2 root root 4096 Dec 23  2020 linux
drwxr-xr-x 2 root root 4096 Dec 23  2020 openssl
-rwxr-xr-x 1 root root  330 Oct 23  2020 syslimits.h

Note cc1 args are,

I[15:20:21.427] internal (cc1) args are: -cc1 -triple x86_64-unknown-linux-gnu -fsyntax-only -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name test.cpp -mrelocation-model pic -pic-level 2 -pic-is-pie -mframe-pointer=all -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu x86-64 -tune-cpu generic -mllvm -treat-scalable-fixed-error-as-warning -debugger-tuning=gdb -fcoverage-compilation-dir=/home/qspace -resource-dir /usr/local/lib/gcc/x86_64-pc-linux-gnu/7.5.0/ -internal-isystem /usr/local/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0 -internal-isystem /usr/local/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/x86_64-pc-linux-gnu -internal-isystem /usr/local/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/backward -internal-isystem /usr/local/lib/gcc/x86_64-pc-linux-gnu/7.5.0/include -internal-isystem /usr/local/include -internal-isystem /usr/local/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../x86_64-pc-linux-gnu/include -internal-externc-isystem /include -internal-externc-isystem /usr/include -fdeprecated-macro -fdebug-compilation-dir=/home/qspace -ferror-limit 19 -fgnuc-version=4.2.1 -fcxx-exceptions -fexceptions -no-round-trip-args -faddrsig -D__GCC_HAVE_DWARF2_CFI_ASM=1 -x c++ /home/qspace/test.cpp

I don’t see any path related to include-fixed here.

Right, this is a common point of confusion: --query-driver=/usr/local/bin/g++ just tells clangd that it’s allowed to execute /usr/local/bin/g++ for the purpose of extracting built-in include paths.

In order for clangd to actually use built-in include paths extracted from /usr/local/bin/g++ for processing a given source file, /usr/local/bin/g++ needs to appear as the argv[0] of the command in the file’s entry in compile_commands.json (or, in clangd 16 (currently trunk), as the value of the Compiler key in the clangd config file).

Yes, this works. Thanks for your clarification.

Although my issue on Clangd got solved, (Thanks to @HighCommander4 again and I found this commit relates to this) I’d still like to know if it is possible to add this include-fixed path to clang driver for GCC toolchain search path by default?

I mean, I saw clang/lib/Driver/ToolChains/Gnu.cpp seem to add tons of paths related to GCC, then I think it makes sense to add something what GCC searches (e.g., /usr/local/bin/../lib/gcc/x86_64-pc-linux-gnu/7.5.0/include-fixed) by default: This is improvement to match the behavior of GCC furthermore, though Clang doesn’t provide something like include-fixed when installing. If this looks promising to you, I’d love to have a try on fixing this :innocent: