Some error in clangd

Hi,
I encounter an error in clangd

I[12:49:12.197] Updating file /root/workspace/eos4/libraries/chain/include/eosio/chain/types.hpp with command [/root/workspace/eos4/build/libraries/wasm-jit/Source/IR] /usr/bin/clang+±4.0 -D ENABLE_SIMD_PROTOTYPE=0 -D ENABLE_THREADING_PROTOTYPE=0 -D IR_API=DLL_EXPORT -D PRETEND_32BIT_ADDRESS_SPACE=0 -D WAVM_METRICS_OUTPUT=0 -I /root/workspace/eos4/libraries/wasm-jit/Include -I /root/workspace/eos4/libraries/wasm-jit/Include/IR -Wall -Wno-invalid-partial-specialization -O3 -D NDEBUG -c -std=gnu++14 /root/workspace/eos4/libraries/chain/include/eosio/chain/types.hpp -resource-dir=/root/.vim/bundle/YouCompleteMe/third_party/ycmd/third_party/clang/lib/clang/7.0.0
V[12:49:12.199] Preamble for file /root/workspace/eos4/libraries/chain/include/eosio/chain/types.hpp cannot be reused. Attempting to rebuild it.
V[12:49:12.238] Built preamble of size 213512 for file /root/workspace/eos4/libraries/chain/include/eosio/chain/types.hpp
I[12:49:12.242] Dropped diagnostic outside main file: : too many errors emitted, stopping now

some other file in same folder update correctly

I[12:49:03.898] Updating file /root/workspace/eos4/libraries/chain/include/eosio/chain/resource_limits.hpp with command [/root/workspace/eos4/build/libraries/chain] /usr/bin/clang+±4.0 -I /root/workspace/eos4/libraries/chain/include -I /root/workspace/eos4/build/libraries/chain/include -I /root/workspace/eos4/libraries/chain/…/wasm-jit/Include -I /root/workspace/eos4/libraries/chain/…/…/externals/binaryen/src -I /root/workspace/eos4/libraries/utilities/include -I /root/workspace/eos4/libraries/utilities/…/wasm-jit/Include -I /root/workspace/eos4/libraries/fc/include -I /root/opt/boost/include -I /usr/local/include -I /root/workspace/eos4/libraries/fc/vendor/websocketpp -I /root/workspace/eos4/libraries/chainbase/include -I /root/workspace/eos4/libraries/wasm-jit/Source/Runtime/…/…/…/chain/include -I /root/workspace/eos4/libraries/softfloat/source/include -I /root/workspace/eos4/libraries/softfloat/source/8086-SSE -I /root/workspace/eos4/libraries/softfloat/build/Linux-x86_64-GCC -I /root/workspace/eos4/libraries/builtins -I /root/workspace/eos4/libraries/builtins…/softfloat/source/include -Wall -Wno-invalid-partial-specialization -O3 -D NDEBUG -c -std=gnu++14 /root/workspace/eos4/libraries/chain/include/eosio/chain/resource_limits.hpp -resource-dir=/root/.vim/bundle/YouCompleteMe/third_party/ycmd/third_party/clang/lib/clang/7.0.0
V[12:49:03.901] Preamble for file /root/workspace/eos4/libraries/chain/include/eosio/chain/resource_limits.hpp cannot be reused. Attempting to rebuild it.
V[12:49:10.676] <<< {“id”:“3”,“jsonrpc”:“2.0”,“method”:“textDocument/definition”,“params”:{“position”:{“character”:51,“line”:53},“textDocument”:{“uri”:“file:///root/workspace/eos4/libraries/chain/include/eosio/chain/resource_limits.hpp”}}}

I don’t know why, and what’s the difference. why the flags are different.
How to fix the problem

root@39470746ee31:~/workspace/eos4# /usr/bin/clang+±4.0 -D ENABLE_SIMD_PROTOTYPE=0 -D ENABLE_THREADING_PROTOTYPE=0 -D IR_API=DLL_EXPORT -D PRETEND_32BIT_ADDRESS_SPACE=0 -D WAVM_METRICS_OUTPUT=0 -I /root/workspace/eos4/libraries/wasm-jit/Include -I /root/workspace/eos4/libraries/wasm-jit/Include/IR -Wall -Wno-invalid-partial-specialization -O3 -D NDEBUG -c -std=gnu++14 /root/workspace/eos4/libraries/chain/include/eosio/chain/types.hpp -resource-dir=/root/.vim/bundle/YouCompleteMe/third_party/ycmd/third_party/clang/lib/clang/7.0.0
/root/workspace/eos4/libraries/chain/include/eosio/chain/types.hpp:6:10: fatal error: ‘eosio/chain/name.hpp’ file not found
#include <eosio/chain/name.hpp>
^~~~~~~~~~~~~~~~~~~~~~
1 error generated.

attach the error and compilation database

Thanks.

Brs
Su Yu

clangd.tar.gz (30.3 KB)

Hi Su,

Since compile_commands.json only contains compile commands for .cpp files, we use a heuristics-based approach to infer compile commands for headers.
More precisely, based on the path to the header file we try to guess the best .cpp file for a header (e.g. an implementation file) and pick the compile command from it.

In your case, we fail to guess a good compile command. I don’t know of a good workaround, unfortunately.
+Sam McCall is the author of this approach, he might have ideas on how to fix this.

Is the project you’re working on open-source? If so, could you provide a link to the source code? If not, compilation database should also be enough to reproduce.

Hi,

Many thanks for your prompt answer.
It’s an open source project.
But it’s a little difficult to build. It has many dependency
Hope it can help.

https://github.com/EOSIO/eos

Thanks.

Brs,
Su Yu

Ilya Biryukov <ibiryukov@google.com> 于2019年3月22日周五 下午5:18写道:

Hi,

By the way,
If a header has no source file (*.cpp), how it been handled.

Brs,
Su Yu

yus <suyu.nu@gmail.com> 于2019年3月22日周五 下午7:37写道:

Hi,

By the way,
If a header has no source file (*.cpp), how it been handled.

In that case we would fallback to something like clang -xc++ test.hpp.
But that almost never happens, our heuristics will always try to pick some cpp file first.

E.g. in the extreme case when compile_commands.json has just a single file, we will take the compile command from that file for all headers we encounter.

+via clangd-dev for the record

HI,

Got it.
So my problem has no relationship with front-end vim-YoucompleteMe
I just need to wait for your solution or workaround.

Thanks for your help.

Brs,
Su Yu

Ilya Biryukov <ibiryukov@google.com> 于2019年3月22日周五 下午8:43写道:

Excuse me, is there any solution for my problem?
Thanks

yus <suyu.nu@gmail.com> 于2019年3月23日周六 上午9:40写道:

I don’t think there’s any simple solution to this. You could try patching your ‘compile_commands.json’ to add the header files in there, but CMake won’t do this automatically for you.
I believe the following project on GitHub tries to do something similar, but I haven’t tried it myself: https://github.com/Sarcasm/compdb

+Sam McCall, now that you’re back from leave, any ideas for improving the heuristics to find proper matching .cpp files in more cases?

Hi! Sorry the heuristics aren’t working here.
The most likely thing is the bare name “types” is matching some *.cpp file from a base library, and we’re weighing that too heavily and the directory not heavily enough. This is probably due to heuristics tuned for the LLVM project itself, which has few dependencies and parallel source/include trees.
Can you attach compile_commands.json please?

Apologies, missed the attachment in the original email. Will look in the morning!

Indeed, the heuristics are valuing base filenames over directory structure match in this case:

sed -i s%/root/workspace/eos4%$PWD%g compile_commands.json

mkdir -p libraries/chain/include/eosio/chain

touch libraries/chain/include/eosio/chain/{types,resource_limits,zzz}.hpp

clang-check -debug-only=interpolate libraries/chain/include/eosio/chain/types.hpp

interpolate: chose …/libraries/wasm-jit/Source/IR/Types.cpp as proxy for …/libraries/chain/include/eosio/chain/types.hpp preferring c++ score=2

(this is a bad result: prefer e.g. …/libraries/chain/abi_serializer.cpp)

clang-check -debug-only=interpolate libraries/chain/include/eosio/chain/resource_limits.hpp

interpolate: chose …/libraries/chain/resource_limits.cpp as proxy for …/libraries/chain/include/eosio/chain/resource_limits.hpp preferring c++ score=3

(this is a good result)

clang-check -debug-only=interpolate libraries/chain/include/eosio/chain/zzz.hpp
interpolate: chose …/libraries/chain/abi_serializer.cpp as proxy for …/libraries/chain/include/eosio/chain/zzz.hpp preferring c++ score=1

(this is a good result)

— details —

Points are awarded to entries in compile_commands:

  • 2 points for filename match without extension (1 for prefix match)
  • up to 3 points for directory structure match (1 for each of last 2 segments, 1 for the rest)

For files in this directory, the relevant directory structure parts are …/libraries/chain/include eosio chain.
So:

  • anything under “…/libraries/chain/include” would get 1 point (there are no cpp files there)
  • anything with “eosio” in the last 4 directory segments gets 1 point (no matches)
  • anything with “chain” in the last 4 directory segments gets 1 point (1 point for sources in “chain” subproject)
    With this directory structure, we do manage to pair headers/sources in the same project, but we only give a 1 point bonus for doing so, and so the 2 point bonus for matching basename can take precedence.

— how to fix in clang —

We only need to tweak the scores here by one point: a tie will be broken in favor of the longest prefix match, which would be when sources are in the same subproject with this directory structure.

One option is to make the name match worth less: 0.5/1 rather than 1/2 points, though I do worry this will break some cases in the other direction.

Another option is to recognize this directory structure match as particularly good, and boost it more - 1 seems relatively low for e.g. …/chain/abi_serializer.cpp matching …/chain/types.hpp where it’s the immediate parent directories that match.
This would be somewhat involved: the index data structure doesn’t currently capture where the word “chain” appeared within the cpp filename.

I’ll look into this…

— how to work around —

The only thing I can suggest is to change your compilation database, either:

  • adding an explicit compile command for types.hpp, and avoid overwriting it, or
  • only generate the CDB for a smaller subset of the project that avoids Types.cpp

Hi,

Many thanks for your analysis and suggestion.
For you suggestion, I know what to do but don’t know how to do.

Can you give some details about the option 1:

  • adding an explicit compile command for types.hpp, and avoid overwriting it,

How to do it.

Thanks.

Brs,
Su Yu

Sam McCall <sammccall@google.com> 于2019年4月3日周三 上午12:24写道:

Add an entry to compile_commands.json:
{
“file”: “…/libraries/chain/include/eosio/chain/types.hpp”,
“directory”: “…/build/libraries/chain”,
“command”: “/usr/bin/clang+±4.0 … -c …/libraries/chain/include/eosio/chain/types.hpp”
}

where the … path is adjusted for your system, and the command is taken from the entry from some other relevant file (e.g. abi_serializer.cpp)

You’ll also need to prevent your build system from overwriting this change.

HI,

It works.
Thank you very much.

Brs,
Su Yu

Sam McCall <sammccall@google.com> 于2019年4月3日周三 下午3:17写道:

HI,

Sorry, another similar problem.
In the CDB generated by cmake,
all source file in the contracts, use wrong command.
for example

{
“directory”: “/root/workspace/eos4/build/contracts/eosio.system”, “command”: “/usr/bin/clang+±4.0 -I/root/workspace/eos4/contracts/eosio.system/… -Wall -Wno-invalid-partial-specialization -O3 -DNDEBUG -std=gnu++14 -o CMakeFiles/eosio.system.tmp.dir/eosio.system.cpp.o -c /root/workspace/eos4/contracts/eosio.system/eosio.system.cpp”,
“file”: “/root/workspace/eos4/contracts/eosio.system/eosio.system.cpp”
},

the right command should be :
/root/opt/wasm/bin/clang -emit-llvm -O3 --std=c++14 --target=wasm32 -ffreestanding -nostdlib -nostdlibinc -fno-threadsafe-statics -fno-rtti -fno-exceptions -c /root/workspace/eos4/contracts/eosio.system/eosio.system.cpp -o eosio.system.cpp.bc -Weverything -Wno-c++98-compat -Wno-old-style-cast -Wno-vla -Wno-vla-extension -Wno-c++98-compat-pedantic -Wno-missing-prototypes -Wno-missing-variable-declarations -Wno-packed -Wno-padded -Wno-c99-extensions -Wno-documentation-unknown-command -I /root/workspace/eos4/contracts -I /root/workspace/eos4/externals/magic_get/include -isystem /root/workspace/eos4/contracts/libc++/upstream/include -isystem /root/workspace/eos4/contracts/musl/upstream/include -isystem /root/opt/boost/include -isystem /root/workspace/eos4/contracts/libc++/upstream/include -isystem /root/workspace/eos4/contracts/musl/upstream/include -isystem /root/opt/boost/include

I have modified some in the CDB, but there are too many file in contracts. Is there any simple way to fix it.
Thanks.

Brs,
Su Yu

yus <suyu.nu@gmail.com> 于2019年4月3日周三 下午3:56写道:

Sorry, you’ll need to raise that issue with CMake developers I think.

Got it,
Thanks

Sam McCall <sammccall@google.com> 于2019年4月4日周四 下午5:22写道: