global alignment

Hello all,
I found that the alignment for stack value has no limitation, but for global value, it has a limitation.

Here is an example:

#include <stdio.h>
#include <stdlib.h>

char x[4000] attribute((aligned(4096)));

int
main (int argc, char ** argv) {
char y[4000] attribute((aligned(4096)));
printf(“x is %p\n”, x);
printf(“y is %p\n”, y);

return 0;
}

After compiled with clang, the result is:
x is 0x804b000
y is 0xbf9d8000

They are both aligned to be 4096 as we expected.

Then we change the example as the following:

char x[4000] attribute((aligned(8192)));

int
main (int argc, char ** argv) {
char y[4000] attribute((aligned(8192)));
printf(“x is %p\n”, x);
printf(“y is %p\n”, y);

return 0;
}

The result is :
x is 0x804d000
y is 0xbffd2000

We can see that the stack value y is aligned to be 8192, but the global value x is not!

My target OS is 32-bit Linux. Anyone can explain this? or is this a bug of clang?

It is very unlikely to be a frontend bug, although I suppose it's possible that it's a backend problem.

It is much more likely that your environment just doesn't support aligning symbols at granularities larger than a page. The page size on x86-32 is 4K because that's what the hardware supports (or 4M if you're using PSE, which Linux does support, but it's not the default, and I don't think it'd solve your problem).

The stack example works because it has to dynamically realign anyway, i.e. it's just %ing the stack pointer, which has no inherent limitations. Note that this is quite expensive and is leaving a potentially huge gap on your stack.

I'll admit to being curious as to why you actually need larger-than-page-size alignment.

John.

Hello all,
I found that the alignment for stack value has no limitation, but for global value, it has a limitation.

Baozen, with what version of LLVM did you do the following test? Was it LLVM mainline?

Here is an example:

#include <stdio.h>
#include <stdlib.h>

char x[4000] __attribute__((aligned(4096)));

int
main (int argc, char ** argv) {
   char y[4000] __attribute__((aligned(4096)));
   printf("x is %p\n", x);
   printf("y is %p\n", y);
      return 0;
}

After compiled with clang, the result is:
x is 0x804b000
y is 0xbf9d8000

They are both aligned to be 4096 as we expected.

Then we change the example as the following:

char x[4000] __attribute__((aligned(8192)));

int
main (int argc, char ** argv) {
   char y[4000] __attribute__((aligned(8192)));
   printf("x is %p\n", x);
   printf("y is %p\n", y);
      return 0;
}

The result is :
x is 0x804d000
y is 0xbffd2000

We can see that the stack value y is aligned to be 8192, but the global value x is not!

My target OS is 32-bit Linux. Anyone can explain this? or is this a bug of clang?

It is very unlikely to be a frontend bug, although I suppose it's possible that it's a backend problem.

It is much more likely that your environment just doesn't support aligning symbols at granularities larger than a page. The page size on x86-32 is 4K because that's what the hardware supports (or 4M if you're using PSE, which Linux does support, but it's not the default, and I don't think it'd solve your problem).

I don't think that's the issue. I've had problems aligning global variables on a 4096 byte boundary with Clang/LLVM 3.0. Using hand-written assembly code to force the alignment fixed the issue.

Additionally, OS page alignment has nothing to do with symbol alignment. Symbol alignment is done by the loader/linker, and from what I've seen, ELF places no restriction on alignment size (other than it being a 32-bit or 64-bit value).

I suspect what we're seeing is a bug in the backend; we'd like to know (if possible) if someone is aware of the limitation.

The stack example works because it has to dynamically realign anyway, i.e. it's just %ing the stack pointer, which has no inherent limitations. Note that this is quite expensive and is leaving a potentially huge gap on your stack.

I'll admit to being curious as to why you actually need larger-than-page-size alignment.

Baggy Bounds Checking: http://static.usenix.org/event/sec09/tech/full_papers/sec09_memory.pdf

-- John T.

We can see that the stack value y is aligned to be 8192, but the global value x is not!

My target OS is 32-bit Linux. Anyone can explain this? or is this a bug of clang?

It is very unlikely to be a frontend bug, although I suppose it's possible that it's a backend problem.

It is much more likely that your environment just doesn't support aligning symbols at granularities larger than a page. The page size on x86-32 is 4K because that's what the hardware supports (or 4M if you're using PSE, which Linux does support, but it's not the default, and I don't think it'd solve your problem).

I don't think that's the issue. I've had problems aligning global variables on a 4096 byte boundary with Clang/LLVM 3.0. Using hand-written assembly code to force the alignment fixed the issue.

Does it actually fix the issue, or does it produce slightly different code such that you win the 50% lottery of being 8K-aligned? Does the assembly output from LLVM look actively wrong?

Additionally, OS page alignment has nothing to do with symbol alignment. Symbol alignment is done by the loader/linker, and from what I've seen, ELF places no restriction on alignment size (other than it being a 32-bit or 64-bit value).

I agree that ELF is certainly not going to hard-code any OS limitations, but that doesn't mean much. The linker is just going to place things appropriately-aligned relative to the start of the section, and there are certainly loaders that will just map sections to any old page.

The stack example works because it has to dynamically realign anyway, i.e. it's just %ing the stack pointer, which has no inherent limitations. Note that this is quite expensive and is leaving a potentially huge gap on your stack.

I'll admit to being curious as to why you actually need larger-than-page-size alignment.

Baggy Bounds Checking: http://static.usenix.org/event/sec09/tech/full_papers/sec09_memory.pdf

Ok.

John.

So here’s what I found: 1) We already know from Baozeng’s experiments that LLVM 3.0 and Clang 3.0 do not align globals properly when the alignment is large. 2) LLVM 3.1 will properly align globals with alignment up to 0x10000 (which is larger than the 8192 byte alignment given in Baozeng’s original example). However, Clang 3.1 will assert out because it uses an unsigned short for representing alignments of lvalues. Changing the alignment class member to unsigned long magically fixes everything, and the code will align variables up to 0x10000 properly. 3) LLVM/Clang mainline have the same behavior as LLVM 3.1. Baozeng, for your Baggy Bounds Checking work, the upgrade to LLVM mainline that SAFECode is currently undergoing will just fix the problem that you’re seeing. You and I can discuss separately whether to make the LLVM mainline switch for BBC during GSoC. As for the bug in Clang, I’ll file a bug report. – John T.