Representing MIPS ABI information in the triple as ARM/X86 do for EABI/EABIHF/X32

Hi,

Having recently enabled IAS by default for the MIPS O32 ABI, I'm now trying to do the same thing for the MIPS N64 ABI. Unfortunately, it is not currently possible to enable IAS by default for the N64 ABI without also enabling it for the N32 ABI because this information is not reflected in the triple and that's the only information MipsMCAsmInfo has. This would be fine if it N32 was also in a good state but the current N32 ABI support for IAS is badly broken and will likely take considerable effort to fix (and fixing it also requires solving the same key problem as enabling IAS for just N64). I therefore want to separate the two ABI's so that I can finish off N64 and then fix N32 afterwards.

I've posted a series of patches that allow us to distinguish N32 and N64 so that we can enable IAS for only N64 (D21465, D21467, D21069 for LLVM along with D21070, D21072 for clang). During the review of D21467, Rafael asked me to explain my approach here. I'm hoping that people will agree that this is an acceptable approach to take and that it's no different from how other targets handle certain ABI's and how we already handle -m32/-m64/-EL/-EB/etc.

The approach I'm using is to encode the ABI information into the triple in the same way that ARM does for EABI and hardfloat, and the same way that X86 does for X32. These targets define variants of Triple::GNU for each of these ABI's in the form of the Triple::GNUEABI, Triple::GNUEABIHF, and Triple::GNUX32 values of the environment component of the triple and it's up to the frontend and/or API-user to pass the right thing to the backend.

For MIPS, I'm defining Triple::GNUABI32, Triple::GNUABIN32, and Triple::GNUABI64. All three of these are supported by tools like binutils (by virtue of a wildcard match '*linux-gnu*') and of these three, Triple::GNUABI32 is not in general use, Triple::GNUABIN32 will be used by Debian if/when we do an N32 port, and Triple::GNUABI64 is currently used by Debian for our N64 port. Once the ABI is in our backend triples, it can be used by MipsMCAsmInfo to enable IAS by default for N64 without also enabling it for N32. In later patches, such as D1292 this same information will also be used to fix the N32 support for IAS.

At this point, you may be wondering what happens to Triple::GNU for triples like mips-linux-gnu and mips64-linux-gnu. The answer to this is that the API-user (e.g. clang) is expected to normalize such triples down to one of Triple::GNUABI32/GNUABIN32/GNUABI64. This can be as simple as adding:
  Triple ABITriple = TT.getABIVariant(ABIName); // ABIName can be the empty string to get the default ABI.
  if (ABITriple.getArch() != Triple::UnknownArch) {
    TT = ABITriple;
    ABIName = ""; // <- Only needed if this would end up in MCTargetOptions::ABIName.
  }
to the appropriate place in the caller. This is the same way clang handles the -m32, -m64, -EL, and -EB options with those options using get32BitArchVariant(), get64BitArchVariant(), getLittleEndianArchVariant(), and getBigEndianArchVariant() respectively to transform the triple for the backend.

In summary, human end users of tools such as clang will continue using the same triples as they always have. These tools will then resolve them down to the effective triple (for example, in functions like clang's ComputeEffectiveClangTriple(), ComputeLLVMTriple(), and computeTargetTriple()). The MIPS backend will only see the triples that have a specific known ABI. I'd like to stress that this process is the same as the one we already use today for other options. For example 'x86_64-linux-gnu-clang -m32' transforms the triple with get32BitArchVariant() and passes 'i386-linux-gnu' to the backend.

So far I've only been talking about Linux/GNU but the patch series also covers Linux/Musl, Linux/Android, and FreeBSD. I haven't talked about those because they are handled in the same way as the Linux/GNU case described above except that they use Triple::ABI32/ABIN32/ABI64 (or Android32/Android64 in the case of Android) instead of Triple::GNUABI32/GNUABIN32/GNUABI64.

Why is the Triple approach better for MIPS than MCTargetOptions::ABIName?

The approach I'm using is to encode the ABI information into the triple in the same way that ARM does for EABI and hardfloat, and the same way that X86 does for X32. These targets define variants of Triple::GNU for each of these ABI's in the form of the Triple::GNUEABI, Triple::GNUEABIHF, and Triple::GNUX32 values of the environment component of the triple and it's up to the frontend and/or API-user to pass the right thing to the backend.

Hi Daniel,

The precedent is there, so I don't think this should be a contentious issue.

For MIPS, I'm defining Triple::GNUABI32, Triple::GNUABIN32, and Triple::GNUABI64. All three of these are supported by tools like binutils (by virtue of a wildcard match '*linux-gnu*') and of these three, Triple::GNUABI32 is not in general use, Triple::GNUABIN32 will be used by Debian if/when we do an N32 port, and Triple::GNUABI64 is currently used by Debian for our N64 port. Once the ABI is in our backend triples, it can be used by MipsMCAsmInfo to enable IAS by default for N64 without also enabling it for N32. In later patches, such as D1292 this same information will also be used to fix the N32 support for IAS.

This "should" work.

In summary, human end users of tools such as clang will continue using the same triples as they always have. These tools will then resolve them down to the effective triple (for example, in functions like clang's ComputeEffectiveClangTriple(), ComputeLLVMTriple(), and computeTargetTriple()). The MIPS backend will only see the triples that have a specific known ABI. I'd like to stress that this process is the same as the one we already use today for other options. For example 'x86_64-linux-gnu-clang -m32' transforms the triple with get32BitArchVariant() and passes 'i386-linux-gnu' to the backend.

We do similar, but not completely.

For instance, "-target arm-linux-gnueabi -mthumb -mfloat-abi=hard
-mcpu=cortex=a15" will change the triple to "thumbv7--linux-gnueabi"
not the expected "thumbv7--linux-gnueabihf".

Though, that's probably a bug. :slight_smile:

cheers,
--renato

Hi,

Having recently enabled IAS by default for the MIPS O32 ABI, I'm now trying to do the same thing for the MIPS N64 ABI. Unfortunately, it is not currently possible to enable IAS by default for the N64 ABI without also enabling it for the N32 ABI because this information is not reflected in the triple and that's the only information MipsMCAsmInfo has.

What would it take to pass more information to MipsMCAsmInfo?

For MIPS, I'm defining Triple::GNUABI32, Triple::GNUABIN32, and Triple::GNUABI64. All three of these are supported by tools like binutils (by virtue of a wildcard match '*linux-gnu*') and of these three, Triple::GNUABI32 is not in general use, Triple::GNUABIN32 will be used by Debian if/when we do an N32 port, and Triple::GNUABI64 is currently used by Debian for our N64 port. Once the ABI is in our backend triples, it can be used by MipsMCAsmInfo to enable IAS by default for N64 without also enabling it for N32. In later patches, such as D1292 this same information will also be used to fix the N32 support for IAS.

I would be perfectly fine with this if instead of just hitting an
wildcard these different triples actually meant different things to
binutils.

Also, do you plan to deprecate the other ways of passing mips abi info
and use only a normalized triple? If so I am OK with adding these
extra triples.

Eric, what do you think?

Cheers,
Rafael

Thanks Renato and Rafael.

> Hi,
>
> Having recently enabled IAS by default for the MIPS O32 ABI, I'm now
trying to do the same thing for the MIPS N64 ABI. Unfortunately, it is not
currently possible to enable IAS by default for the N64 ABI without also
enabling it for the N32 ABI because this information is not reflected in the
triple and that's the only information MipsMCAsmInfo has.

What would it take to pass more information to MipsMCAsmInfo?

All callers of createMCAsmInfo() and LLVMCreateDisasmCPUFeatures() would need to specify the ABI. In the case of LLVMCreateDisasmCPUFeatures(), they'd move to a new variant of this function and some backward compatibility fixups would be left in the pre-existing version to maintain its current level of functionality where only the default ABI for the triple is available. IRObjectFile would need some means of determining the ABI otherwise IRObjectFile::CollectAsmUndefinedRefs() would produce a slightly different result for some assembly inputs. This is because a small number of directives introduce references on particular ABI's and not on others. I think this information would be best delivered via the LLVM-IR but adding the ABI in LLVM-IR files has been strongly opposed in previous discussions. Also, IRObjectFile may need the CPU due to some changes to relocation rules for MIPS32R6 that prevent elimination of relocations that can be evaluated locally which would lead to the same issue.

The other callers of createMCAsmInfo() either already have the ABI by one means or another, or would be able to figure it out if we extend ObjectFile to detect the ABI from the object*. Some of the tools that call createMCAsmInfo() link multiple object files (e.g. llvm-rtdyld, and llvm-dwp) and may therefore need to implement the ABI inter-linking rules.

Looking beyond enabling IAS for N64 towards fixing N32 support for IAS, the same requirements would apply to createMCSubtargetInfo(), createAsmPrinter() (because the Mips, PowerPC, AMDGPU, and WebAssembly targets call createMCSubtargetInfo() in their implementations), createMCCodeEmitter(), and createMCAsmBackend(). IRLinker would also need to handle the incompatibility between the O32 and N32/N64 ABI's IR-level since frontends can generate different IR for different calling conventions. Other targets handle this incompatibility by having different triples. I think N32 and N64 are compatible at the LLVM-IR level but I haven't proven this and it's moot since they aren't compatible with O32 which should also be allowed on MIPS64.

* This bit is slightly different from the description in http://reviews.llvm.org/D20916 where I believed many tools needed to provide the ABI but lacked the information to do so. Since writing that, I have added a facility to ObjectFile that allows llvm-objdump to detect the feature bits from the Object header. The same technique can be used for other tools that handle ObjectFiles.

> For MIPS, I'm defining Triple::GNUABI32, Triple::GNUABIN32, and
> Triple::GNUABI64. All three of these are supported by tools like binutils (by
> virtue of a wildcard match '*linux-gnu*') and of these three,
> Triple::GNUABI32 is not in general use, Triple::GNUABIN32 will be used by
> Debian if/when we do an N32 port, and Triple::GNUABI64 is currently used
> by Debian for our N64 port. Once the ABI is in our backend triples, it can be
> used by MipsMCAsmInfo to enable IAS by default for N64 without also
> enabling it for N32. In later patches, such as D1292 this same information will
> also be used to fix the N32 support for IAS.

I would be perfectly fine with this if instead of just hitting an
wildcard these different triples actually meant different things to
binutils.

That doesn't seem necessary since 'gnueabi' and 'gnueabihf' seem to be matched by a 'gnu*' wildcard too but I'm in favour of defining these triples in binutils if that's necessary to fix LLVM. I'd appreciate it if this patch series didn't have to wait until the triples are present in the binutils source though. Are you ok with us going ahead with this patch series while a corresponding binutils patch is being put together?

I should also mention that Debian's binutils package already contains downstream patches to define 'gnuabi64' and select the appropriate default behaviour. I assume this will be upstreamed in the near future.

Also, do you plan to deprecate the other ways of passing mips abi info
and use only a normalized triple? If so I am OK with adding these
extra triples.

Yes, ⚙ D21069 [mips] Require that ABI's are passed in the triple within LLVM. makes it mandatory to pass the ABI in the triple.

All callers of createMCAsmInfo() and LLVMCreateDisasmCPUFeatures() would need to specify the ABI. In the case of LLVMCreateDisasmCPUFeatures(), they'd move to a new variant of this function and some backward compatibility fixups would be left in the pre-existing version to maintain its current level of functionality where only the default ABI for the triple is available. IRObjectFile would need some means of determining the ABI otherwise IRObjectFile::CollectAsmUndefinedRefs() would produce a slightly different result for some assembly inputs. This is because a small number of directives introduce references on particular ABI's and not on others. I think this information would be best delivered via the LLVM-IR but adding the ABI in LLVM-IR files has been strongly opposed in previous discussions. Also, IRObjectFile may need the CPU due to some changes to relocation rules for MIPS32R6 that prevent elimination of relocations that can be evaluated locally which would lead to the same issue.

Yes, that seems worse than just adding a few extra triples.

I should also mention that Debian's binutils package already contains downstream patches to define 'gnuabi64' and select the appropriate default behaviour. I assume this will be upstreamed in the near future.

Thanks, that is another strong indication for going with the triples.

Also, do you plan to deprecate the other ways of passing mips abi info
and use only a normalized triple? If so I am OK with adding these
extra triples.

Yes, ⚙ D21069 [mips] Require that ABI's are passed in the triple within LLVM. makes it mandatory to pass the ABI in the triple.

Awesome.

Thanks,
Rafael

Hi Daniel,

While I’m sympathetic to the amount of work associated this appears to be attempting to work around an API problem by changing triples needlessly. There’s already some ABI support in MCTargetOptions that I used when fixing up the ARM port a while back and this use can either be extended or changed depending on what’s necessary. It may require plumbing, but that’s expected with an API being expanded for new uses. It appears that the command line syntax is sufficient for the needs of enumerating target plus abi and if we need to add more defaults then it shouldn’t be too bad to do so.

Even if it weren’t an API problem I’m not sure it’d be a good idea. I think we could add a with-abi configuration option to match the current gcc one rather than tacking it onto triple parsing.

That said, I’m more than happy to help you detangle the existing API and come up with a good strategy for the MC level API.

Thanks!

-eric

ps. It also doesn’t appear that Matthew (who took over my gcc responsibilities, thanks!) has added support for this in gcc yet either.

echristo@dzur ~/b/build-gcc-mips64abin64> …/configure --target=mips64-linux-gnuabi64 --enable-languages=c

echristo@dzur ~/tmp> ~/builds/build-gcc-mips64abin64/gcc/cc1 foo.c -o -
.file 1 “foo.c”
.section .mdebug.abiN32

Hi Eric,

It’s the unsolved problems on the pass-MCTargetOptions-everywhere path that are my main concern with that approach rather than the amount of work. The first problem is that the result of IRObjectFile::CollectAsmUndefinedRefs() depends on the ABI but IRObjectFile doesn’t know it. How would you deliver the ABI to IRObjectFile? The second problem is that IRLinker will link incompatible IR together because it doesn’t know the ABI’s involved in the link. It thinks it’s sufficient to check that the triples are the same. How would you prevent it from linking IR intended for O32 with IR intended for N64?

I’m also worried about the risk of undiscovered problems on the pass-MCTargetOptions-everywhere path but it’s difficult to discuss those for obvious reasons. However, I think it’s safe to say that there could be some major problems waiting to be discovered given that we don’t have any targets that implement a major ABI with this approach.

There’s already some ABI support in MCTargetOptions that I used when fixing up the ARM port a while back and this use can either be extended or changed depending on what’s necessary

That’s correct and r224492 is quite similar to what I’m trying to do in this patch series (see below). I should mention that even that commit relies on the ABI being encoded in the triple though. In computeTargetABI() from ARMTargetMachine.cpp, MCTargetOptions::ABIName specifies the ABI if it is a non-empty string but the code otherwise falls back on extracting the ABI from the triple. The clang driver has similar code, it determines the value for the –target-abi option from –mabi if it’s given and falls back on calculating it from the triple using the same code as computeTargetABI(). The overall effect is that the ABI information is delivered to some parts of LLVM via both MCTargetOptions::ABIName and the triple, and other parts solely via the triple (e.g. ARMAsmPrinter and ARMISelLowering). Both copies of this information always agree when –mabi is not given but they may differ when –mabi is given.

As I mentioned above, I’m trying to do a very similar thing to r224492 but I have an additional problem to deal with. In my scenario, clang doesn’t start off with a triple that clearly specifies the ABI (mips64-linux-gnu can be N32 or N64 depending on –mabi) which makes it impossible for the bits of LLVM that can’t see MCTargetOptions to know what to do. I therefore start by normalizing the triple in clang to one that does specify the ABI clearly and is consistent with the –mabi option. With that problem solved, the rest of the patch series is consistent with r224492 except that MCTargetOptions::ABIName and the triple can never disagree. Personally, I would stop the patch series at this point and not go on to duplicate the ABI information into MCTargetOptions::ABIName (because this permits inconsistency and it doesn’t make sense to put N64 code inside an O32 ELF) but if you feel strongly about this then I can add another patch that causes clang to pass the information in both the triple and the –target-abi option and have the Mips backend accept this. With that additional patch, this patch series will be completely consistent with r224492.

Even if it weren’t an API problem I’m not sure it’d be a good idea. I think we could add a with-abi configuration option to match the current gcc one rather than tacking it onto triple parsing.

I thought you were strongly against configure-time options based on previous discussions. Has this changed?

ps. It also doesn’t appear that Matthew (who took over my gcc responsibilities, thanks!) has added support for this in gcc yet either.

echristo@dzur ~/b/build-gcc-mips64abin64> …/configure --target=mips64-linux-gnuabi64 --enable-languages=c

echristo@dzur ~/tmp> ~/builds/build-gcc-mips64abin64/gcc/cc1 foo.c -o -

.file 1 “foo.c”

.section .mdebug.abiN32

It’s true that upstream GCC doesn’t know about this triple yet but that’s because downstream users of GCC don’t need to push their changes upstream when they redefine triples. Fedora and Gentoo both use --with-abi while Debian uses source patches. Upstream GCC only contains the default-defaults which may be different from the actual-defaults that are compiled in to the toolchain. In contrast, LLVM requires all the (often conflicting) meanings of a triple to be upstream. This is part of a separate problem that I plan to address in a later discussion so I’ll leave this for another thread.

Outlook removed llvm-dev again. Re-added it.

I still do not believe this is the correct direction. I’ll reply to the thread when i am able.

Thanks.

cc1as. The main difference is that X86 is adjusting the triple in the driver while I’m doing it in cc1 and cc1as.

Rereading your email it looks like there was a bit of confusion.

As long as the interface to the driver is still --target mips-linux-gnu -mabi=XX I care a bit less that you’re mangling the abi information into the triple. I still don’t believe it’s the correct long term direction for the project, but it appears to unblock you with a minimum of work.

Please do, however, make this consistent for every port in the tree. Mips is not enough of a special snowflake that it needs different ABI handling than any other port. :slight_smile:

Thanks.

-eric

cc1as. The main difference is that X86 is adjusting the triple in the driver while I’m doing it in cc1 and cc1as.
As long as the interface to the driver is still --target mips-linux-gnu -mabi=XX I care a bit
less that you’re mangling the abi information into the triple. I still don’t believe it’s the
correct long term direction for the project, but it appears to unblock you with a minimum
of work.

Thanks. Yep, the interface to the driver is unchanged. Do you have a preference as to whether the driver, cc1+cc1as, or both adjust the triple?

Please do, however, make this consistent for every port in the tree. Mips is not enough of
a special snowflake that it needs different ABI handling than any other port. :slight_smile:

Sure, I’ll make sure ARM, Mips, and X86 work the same way and I’ll have another look through the others in case I’ve missed any that do the same thing.

Thanks again.

cc1as. The main difference is that X86 is adjusting the triple in the driver while I’m doing it in cc1 and cc1as.

As long as the interface to the driver is still --target mips-linux-gnu -mabi=XX I care a bit
less that you’re mangling the abi information into the triple. I still don’t believe it’s the
correct long term direction for the project, but it appears to unblock you with a minimum
of work.

Thanks. Yep, the interface to the driver is unchanged. Do you have a preference as to whether the driver, cc1+cc1as, or both adjust the triple?

Not in particular here. Do what’s the easiest to maintain and undo later :slight_smile:

Please do, however, make this consistent for every port in the tree. Mips is not enough of
a special snowflake that it needs different ABI handling than any other port. :slight_smile:

Sure, I’ll make sure ARM, Mips, and X86 work the same way and I’ll have another look through the others in case I’ve missed any that do the same thing.

Thanks again.

Cool.

Thanks!

-eric