clang .code16 with -Os producing larger code that it needs to

When experimenting with compiling GRUB2 with clang using integrated as,
I found out that it generates a 16-bit code bigger than gas counterpart
and result gets too big for size constraints of bootsector. This was
traced mainly to 2 problems.
32-bit access to 16-bit addresses.
source:
  movl LOCAL(kernel_sector), %ebx
  movl %ebx, 8(%si)
clang:
    7cbc: 67 66 8b 1d 5c 7c 00 addr32 mov 0x7c5c,%ebx
    7cc3: 00
    7cc4: 66 89 5c 08 mov %ebx,0x8(%si)

gas:
    7cbc: 66 8b 1e 5c 7c mov 0x7c5c,%ebx
    7cc1: 66 89 5c 08 mov %ebx,0x8(%si)
32-bit jump.
source:
  jnb LOCAL(floppy_probe)
clang:
+ 7cb5: 66 0f 83 07 01 00 00 jae 7dc3 <L_floppy_probe>
gas:
- 7cb5: 0f 83 0a 01 jae 7dc3 <L_floppy_probe>
The last one is particularly problematic as it never makes sense to
issue 32-bit jump if %ip is only 16 bits and it eats 3 extra bytes per
jump. Is it possible to force clang to generate 16-bit jumps?
On bright side if I remove error strings the code is functional.

When experimenting with compiling GRUB2 with clang using integrated as,
I found out that it generates a 16-bit code bigger than gas counterpart
and result gets too big for size constraints of bootsector. This was
traced mainly to 2 problems.
32-bit access to 16-bit addresses.
source:
  movl LOCAL(kernel_sector), %ebx
  movl %ebx, 8(%si)
clang:
    7cbc: 67 66 8b 1d 5c 7c 00 addr32 mov 0x7c5c,%ebx
    7cc3: 00
    7cc4: 66 89 5c 08 mov %ebx,0x8(%si)

gas:
    7cbc: 66 8b 1e 5c 7c mov 0x7c5c,%ebx
    7cc1: 66 89 5c 08 mov %ebx,0x8(%si)
32-bit jump.
source:
  jnb LOCAL(floppy_probe)
clang:
+ 7cb5: 66 0f 83 07 01 00 00 jae 7dc3 <L_floppy_probe>
gas:
- 7cb5: 0f 83 0a 01 jae 7dc3 <L_floppy_probe>

Minimal example would be:
  .code16
  jmp 1f
  .space 256
1: nop
clang:
   0: 66 e9 00 01 00 00 jmpl 0x106
  ...
106: 90 nop
gcc:
   0: e9 00 01 jmp 0x103
  ...
103: 90 nop

When experimenting with compiling GRUB2 with clang using integrated as,
I found out that it generates a 16-bit code bigger than gas counterpart
and result gets too big for size constraints of bootsector. This was
traced mainly to 2 problems.

...

32-bit access to 16-bit addresses.
clang:
    7cbc: 67 66 8b 1d 5c 7c 00 00 addr32 mov 0x7c5c,%ebx
gas:
    7cbc: 66 8b 1e 5c 7c mov 0x7c5c,%ebx

32-bit jump.
clang:
+ 7cb5: 66 0f 83 07 01 00 00 jae 7dc3 <L_floppy_probe>
gas:
- 7cb5: 0f 83 0a 01 jae 7dc3 <L_floppy_probe>

To a large extent, those are the *same* problem. We don't know that it's
eventually going to fit into a 16-bit offset, so we emit it with a fixup
record which can cope with 32 bits.

Arguably, the jump is *particularly* gratuitous in many cases... but in
'big real' mode is the IP *really* limited to 16 bits?

We could make it default to 16-bit, as gas does. But then we'd be
screwed in the cases where we really *do* need 32-bit.

What we actually need to do is implement handling for the explicit
addr32 prefix. Then we can do what gas does and default to 16-bit but
*also* have a way to do 32-bit when it's needed.

When experimenting with compiling GRUB2 with clang using integrated as,
I found out that it generates a 16-bit code bigger than gas counterpart
and result gets too big for size constraints of bootsector. This was
traced mainly to 2 problems.

...

32-bit access to 16-bit addresses.
clang:
    7cbc: 67 66 8b 1d 5c 7c 00 00 addr32 mov 0x7c5c,%ebx
gas:
    7cbc: 66 8b 1e 5c 7c mov 0x7c5c,%ebx

32-bit jump.
clang:
+ 7cb5: 66 0f 83 07 01 00 00 jae 7dc3 <L_floppy_probe>
gas:
- 7cb5: 0f 83 0a 01 jae 7dc3 <L_floppy_probe>

To a large extent, those are the *same* problem. We don't know that it's
eventually going to fit into a 16-bit offset, so we emit it with a fixup
record which can cope with 32 bits.

All labels are local to the source file. If I use %eax instead of %ebx
in first example I get the short code. For the second example how does
clang detect that offset fits into one byte for issuing EB XX sequence
which is issued in resulting file in several places. Can we use the same
mechanism to detect when issuing 16-bit reference and keep 32-bit one
for external references?

It's been a while since I looked at this... but I think for the short
jumps we just emit the 8-bit version and there's a fixup which can go
back and re-emit the instruction in 32-bit mode if it finds it doesn't
fit?

Do we just need to support a similar fixup for promoting 16-bit to
32-bit relocations?

OK, the term I was looking for was 'relaxation'. Look in
lib/Target/X86/MCTargetDesc/X86AsmBackend.cpp for
X86AsmBackend::relaxInstruction() and related methods.

Observe that it will cope with 'relaxing' 8-bit PC-relative relocations
to 32-bit PC-relative, but it doesn't cope with anything else.

Your task, should you choose to accept it, is to make it cope with other
forms of relaxation where necessary.

Note that the existing cases end up emitting a new instruction with a
*new* opcode. In your case it won't be doing that. It's the *same*
opcode, but you'll have to set a flag to tell the emitter to use the
32-bit addressing mode (for data and/or addr as appropriate) this time.

And while you're doing that, you should note that that's the *same* flag
that'll be needed to support explicit addr32/data32 prefixes in the asm
source. So you might as well support those too. I might suggest doing
them *first*, in fact.

Your task, should you choose to accept it, is to make it cope with other
forms of relaxation where necessary.

And if not, please open a bug :slight_smile:

There are a few other missing cases that cause MC to produce code that
is more "relaxed" than it needs to be.

Cheers,
Rafael

http://llvm.org/bugs/show_bug.cgi?id=22662

FWIW I could reproduce the 'movl foo, %ebx' one but a relative jump
*was* using 16 bits (although gas uses 8):

$ cat foo.S
.code16
  jae foo
  movl (foo), %ebx
foo:
$ gcc -c -oa.out foo.S ; llvm-objdump -d -triple=i686-pc-linux-code16

a.out: file format ELF64-x86-64

Disassembly of section .text:
.text:
       0: 73 05 jae 5
       2: 66 8b 1e 00 00 movl 0, %ebx
$ llvm-mc -filetype=obj foo.S | llvm-objdump -d -triple=i686-pc-linux-code16 -

<stdin>: file format ELF64-x86-64

Disassembly of section .text:
.text:
       0: 0f 83 08 00 jae 8
       4: 67 66 8b 1d 00 00 00 00 movl 0, %ebx

Does gas really relax from 16-bit addresses to 32-bit address as necessary? I played around briefly and it looks like gas will only emit 16-bit addresses in 16-bit mode unless addr32 is prefixed. Even for an external symbol it only emitted a 16-bit relocation type until I added addr32.

I wonder if we shouldn’t fix the x86 encoder to use 16-bit addresses in 16-bit mode. (Actually I think we’re emitting 0x67 prefix because the displacement size check in Is16BitMemOperand doesn’t like cases where displacement isExpr instead of isImm). And maybe override the mode in SubTargetInfo around the EmitInstruction call for any that specifies “addr32” in 16-bit mode or “addr16” in 32-bit mode?

That doesn’t help with jumps though since they do need their opcode switched to JMP_2 instead of JMP_4. Again I can’t prove that gas will further relax 2-byte to 4-byte without addr32. I think we either need to again change SubTargetInfo and pass it into relaxInstruction OR we could create new 16-bit mode only 1-byte jumps that we can parse based on mode and relax to the 2 byte form.

Does gas really relax from 16-bit addresses to 32-bit address as
necessary? I played around briefly and it looks like gas will only
emit 16-bit addresses in 16-bit mode unless addr32 is prefixed. Even
for an external symbol it only emitted a 16-bit relocation type until
I added addr32.

I believe you are correct. My use of 32-bit relocations in LLVM was
mostly because we didn't yet support addr32. Having code which is
correct but slightly larger than needed was better than having some
things which you just *couldn't* build, in the short term.

I wonder if we shouldn't fix the x86 encoder to use 16-bit addresses
in 16-bit mode.

Yes, we should. Having implemented the addr32 prefix first.

(Actually I think we're emitting 0x67 prefix because the displacement
size check in Is16BitMemOperand doesn't like cases where displacement
isExpr instead of isImm). And maybe override the mode in SubTargetInfo
around the EmitInstruction call for any that specifies "addr32" in
16-bit mode or "addr16" in 32-bit mode?

That doesn't help with jumps though since they do need their opcode
switched to JMP_2 instead of JMP_4. Again I can't prove that gas will
further relax 2-byte to 4-byte without addr32. I think we either need
to again change SubTargetInfo and pass it into relaxInstruction OR we
could create new 16-bit mode only 1-byte jumps that we can parse based
on mode and relax to the 2 byte form.

I don't think we need a new opcode for a 16-bit mode 1-byte jump, do we?
The mode is already stored in the MCInst because I needed to do that to
fix PR18303.