BPF: question in regard to .rodata.cst32

Hi,

Hope this is the right place to ask questions, I ran into an issue in
https://github.com/cilium/cilium/issues/18616 where I have defined
macro in header file node_config.h like below:

grep 'VTEP' /var/run/cilium/state/globals/node_config.h

#define ENABLE_VTEP 1
#define VTEP_ENDPOINT (__u32){ 0xeb48a90a, 0xec48a90a, 0xed48a90a,
0xee48a90a, }
#define VTEP_MAC (__u64){ 0x562e984c3682, 0x552e984c3682,
0x542e984c3682, 0x532e984c3682, }
#define VTEP_NUMS 4

then the cilium bpf/bpf_lxc.c file include the node_config.h file and
code to use above macro like below:

#ifdef ENABLE_VTEP

        {
                int i;
                for (i = 0; i < VTEP_NUMS; i++) {
                        if (tunnel_endpoint == VTEP_ENDPOINT[i]) {
                                vtep_mac = VTEP_MAC[i];
                                break;
                       }
                }

                if (vtep_mac && tunnel_endpoint) {
                        if (eth_store_daddr(ctx, (__u8 *)&vtep_mac, 0) < 0)
                                return DROP_WRITE_ERROR;
                        return __encap_and_redirect_with_nodeid(ctx,
tunnel_endpoint,

WORLD_ID, monitor);
                }
        }
#endif

cilium will invoke clang to compile the bpf_lxc.o like below:

clang ...CUT... -DENABLE_VTEP=1
-I/home/go/src/github.com/f5devcentral/cilium/bpf/include
-I/home/go/src/github.com/f5devcentral/cilium/bpf -D__NR_CPUS__=2 -O2
-g -target bpf -std=gnu89 -nostdinc -emit-llvm -Wall -Wextra -Werror
-Wshadow -Wno-address-of-packed-member -Wno-unknown-warning-option
-Wno-gnu-variable-sized-type-not-at-end -Wdeclaration-after-statement
-Wimplicit-int-conversion -Wenum-conversion -c bpf_lxc.c -o bpf_lxc.ll

llc -march=bpf -mattr=dwarfris -mcpu=v2 -filetype=obj -o bpf_lxc.o bpf_lxc.ll

the bpf_lxc.o file ended up with a .rodata.cst32 section like below

readelf -x .rodata.cst32 /var/run/cilium/state/1378_next_fail/bpf_lxc.o

Hex dump of section '.rodata.cst32':
  0x00000000 82364c98 2e560000 82364c98 2e550000 .6L..V...6L..U..
  0x00000010 82364c98 2e540000 82364c98 2e530000 .6L..T...6L..S..

as you can see the .rodata.cst32 contains the VTEP_MAC data.

but if I reduce the VTEP_ENDPOINT/VTEP_MAC member to 2 like below, no
any .rodata generated from the clang compiler

grep 'VTEP' /var/run/cilium/state/globals/node_config.h
#define ENABLE_VTEP 1
#define VTEP_ENDPOINT (__u32){ 0xeb48a90a, 0xec48a90a, }
#define VTEP_MAC (__u64){ 0x562e984c3682, 0x552e984c3682, }
#define VTEP_NUMS 2

readelf -e /var/run/cilium/state/1378/bpf_lxc.o | grep 'rodata', nothing

The problem I have is cilium/ebpf fail to recognize .rodata.cst32 and
lead cilium to fail.

My question is why clang creates .rodata.cst32 when
VTEP_ENDPOINT/VTEP_MAC exceed 2 members, and why no rodata when the
member <= 2.

I assume cilium/ebpf project will likely to address the issue to
recognize .rodata.cst32, I am just wondering if there is anything from
clang BPF side that needs to be addressed.

Thank you for you patience to read this email :slight_smile:

Regards,

Vincent

Hi Yonghong,

Sorry to miss your answer to BPF mailing list before I write this
email, I guess it works as designed then.

Thanks

Vincent

>
> Hi,
>
> Hope this is the right place to ask questions, I ran into an issue in
> https://github.com/cilium/cilium/issues/18616 where I have defined
> macro in header file node_config.h like below:
>
> grep 'VTEP' /var/run/cilium/state/globals/node_config.h
>
> #define ENABLE_VTEP 1
> #define VTEP_ENDPOINT (__u32){ 0xeb48a90a, 0xec48a90a, 0xed48a90a,
> 0xee48a90a, }
> #define VTEP_MAC (__u64){ 0x562e984c3682, 0x552e984c3682,
> 0x542e984c3682, 0x532e984c3682, }
> #define VTEP_NUMS 4
>
> then the cilium bpf/bpf_lxc.c file include the node_config.h file and
> code to use above macro like below:
>
> #ifdef ENABLE_VTEP
>
> {
> int i;
> for (i = 0; i < VTEP_NUMS; i++) {
> if (tunnel_endpoint == VTEP_ENDPOINT[i]) {
> vtep_mac = VTEP_MAC[i];
> break;
> }
> }
>
> if (vtep_mac && tunnel_endpoint) {
> if (eth_store_daddr(ctx, (__u8 *)&vtep_mac, 0) < 0)
> return DROP_WRITE_ERROR;
> return __encap_and_redirect_with_nodeid(ctx,
> tunnel_endpoint,
>
> WORLD_ID, monitor);
> }
> }
> #endif
>
> cilium will invoke clang to compile the bpf_lxc.o like below:
>
> clang ...CUT... -DENABLE_VTEP=1
> -I/home/go/src/github.com/f5devcentral/cilium/bpf/include
> -I/home/go/src/github.com/f5devcentral/cilium/bpf -D__NR_CPUS__=2 -O2
> -g -target bpf -std=gnu89 -nostdinc -emit-llvm -Wall -Wextra -Werror
> -Wshadow -Wno-address-of-packed-member -Wno-unknown-warning-option
> -Wno-gnu-variable-sized-type-not-at-end -Wdeclaration-after-statement
> -Wimplicit-int-conversion -Wenum-conversion -c bpf_lxc.c -o bpf_lxc.ll
>
> llc -march=bpf -mattr=dwarfris -mcpu=v2 -filetype=obj -o bpf_lxc.o bpf_lxc.ll
>
> the bpf_lxc.o file ended up with a .rodata.cst32 section like below
>
> readelf -x .rodata.cst32 /var/run/cilium/state/1378_next_fail/bpf_lxc.o
>
> Hex dump of section '.rodata.cst32':
> 0x00000000 82364c98 2e560000 82364c98 2e550000 .6L..V...6L..U..
> 0x00000010 82364c98 2e540000 82364c98 2e530000 .6L..T...6L..S..
>
> as you can see the .rodata.cst32 contains the VTEP_MAC data.
>
> but if I reduce the VTEP_ENDPOINT/VTEP_MAC member to 2 like below, no
> any .rodata generated from the clang compiler
>
> grep 'VTEP' /var/run/cilium/state/globals/node_config.h
> #define ENABLE_VTEP 1
> #define VTEP_ENDPOINT (__u32){ 0xeb48a90a, 0xec48a90a, }
> #define VTEP_MAC (__u64){ 0x562e984c3682, 0x552e984c3682, }
> #define VTEP_NUMS 2
>
> readelf -e /var/run/cilium/state/1378/bpf_lxc.o | grep 'rodata', nothing
>
> The problem I have is cilium/ebpf fail to recognize .rodata.cst32 and
> lead cilium to fail.
>
> My question is why clang creates .rodata.cst32 when
> VTEP_ENDPOINT/VTEP_MAC exceed 2 members, and why no rodata when the
> member <= 2.
>
> I assume cilium/ebpf project will likely to address the issue to
> recognize .rodata.cst32, I am just wondering if there is anything from
> clang BPF side that needs to be addressed.
>
> Thank you for you patience to read this email :slight_smile:
>
> Regards,
>
> Vincent

Hi Yonghong,

Sorry to miss your answer to BPF mailing list before I write this
email, I guess it works as designed then.

Right. As I explained in bpf mailing list
  can't get BTF: type .rodata.cst32: not found
This is expected based on your code pattern.