Getting command line options to affect subtarget features

The problem I'm trying to solve: Invoking clang on PowerPC with
-fno-altivec has no effect.

From what I've been able to piece together, PPC.td specifies various

CPUs and the processor features available on each. So for example we
have:

def FeatureAltivec : SubtargetFeature<"altivec","HasAltivec", "true",
                                        "Enable Altivec instructions">;

def : Processor<"pwr7", G5Itineraries,
                  [DirectivePwr7, FeatureAltivec,
                   FeatureMFOCRF, FeatureFSqrt, FeatureSTFIWX,
                   FeatureISEL, Feature64Bit /*, Feature64BitRegs */]>;

During compilation the subtarget class (such as PPCSubtarget) is
instantiated with the strings obtained from -mcpu and -mattr.
ParseSubtargetFeatures then uses the -mcpu and -mattr values to
determine which features to use for code generation.

What seems problematic is that -mattr is the only way that the static
features of the target processor can be overridden.

It seems there must be a way for command-line parameters such as
-fno-altivec to be used to override which processor features are
exploited. But I can't see immediately how this should work. I don't
know whether or how the command line flags from Clang are made available
to LLVM for inspection.

I'm sure there must be an obvious way to do this, but peeking at code
from other targets hasn't enlightened me. Any advice for how I can
accomplish this?

Thanks! I appreciate any help.

Bill

From: "Bill Schmidt" <wschmidt@linux.vnet.ibm.com>
To: llvmdev@cs.uiuc.edu
Sent: Thursday, January 31, 2013 9:26:15 AM
Subject: [LLVMdev] Getting command line options to affect subtarget features

The problem I'm trying to solve: Invoking clang on PowerPC with
-fno-altivec has no effect.

From what I've been able to piece together, PPC.td specifies various
CPUs and the processor features available on each. So for example we
have:

def FeatureAltivec : SubtargetFeature<"altivec","HasAltivec",
"true",
                                        "Enable Altivec
                                        instructions">;

def : Processor<"pwr7", G5Itineraries,
                  [DirectivePwr7, FeatureAltivec,
                   FeatureMFOCRF, FeatureFSqrt, FeatureSTFIWX,
                   FeatureISEL, Feature64Bit /*, Feature64BitRegs
                   */]>;

During compilation the subtarget class (such as PPCSubtarget) is
instantiated with the strings obtained from -mcpu and -mattr.
ParseSubtargetFeatures then uses the -mcpu and -mattr values to
determine which features to use for code generation.

What seems problematic is that -mattr is the only way that the static
features of the target processor can be overridden.

It seems there must be a way for command-line parameters such as
-fno-altivec to be used to override which processor features are
exploited. But I can't see immediately how this should work. I
don't
know whether or how the command line flags from Clang are made
available
to LLVM for inspection.

I'm sure there must be an obvious way to do this, but peeking at code
from other targets hasn't enlightened me. Any advice for how I can
accomplish this?

Off of the top of my head...

Within the current infrastructure, I think that we need to alter the Clang PowerPC toolchain code (Clang::AddPPCTargetArgs in Tools.cpp) to do something like this:

if (Args.hasFlag(options::OPT_fno_altivec, options::OPT_faltivec, true)) {
  CmdArgs.push_back("-backend-option");
  CmdArgs.push_back("-mattr=-altivec");
}

Why this not done currently I don't know.

-Hal

> From: "Bill Schmidt" <wschmidt@linux.vnet.ibm.com>
> To: llvmdev@cs.uiuc.edu
> Sent: Thursday, January 31, 2013 9:26:15 AM
> Subject: [LLVMdev] Getting command line options to affect subtarget features
>
> The problem I'm trying to solve: Invoking clang on PowerPC with
> -fno-altivec has no effect.
>
> From what I've been able to piece together, PPC.td specifies various
> CPUs and the processor features available on each. So for example we
> have:
>
> def FeatureAltivec : SubtargetFeature<"altivec","HasAltivec",
> "true",
> "Enable Altivec
> instructions">;
>
> def : Processor<"pwr7", G5Itineraries,
> [DirectivePwr7, FeatureAltivec,
> FeatureMFOCRF, FeatureFSqrt, FeatureSTFIWX,
> FeatureISEL, Feature64Bit /*, Feature64BitRegs
> */]>;
>
> During compilation the subtarget class (such as PPCSubtarget) is
> instantiated with the strings obtained from -mcpu and -mattr.
> ParseSubtargetFeatures then uses the -mcpu and -mattr values to
> determine which features to use for code generation.
>
> What seems problematic is that -mattr is the only way that the static
> features of the target processor can be overridden.
>
> It seems there must be a way for command-line parameters such as
> -fno-altivec to be used to override which processor features are
> exploited. But I can't see immediately how this should work. I
> don't
> know whether or how the command line flags from Clang are made
> available
> to LLVM for inspection.
>
> I'm sure there must be an obvious way to do this, but peeking at code
> from other targets hasn't enlightened me. Any advice for how I can
> accomplish this?

Off of the top of my head...

Within the current infrastructure, I think that we need to alter the Clang PowerPC toolchain code (Clang::AddPPCTargetArgs in Tools.cpp) to do something like this:

if (Args.hasFlag(options::OPT_fno_altivec, options::OPT_faltivec, true)) {
  CmdArgs.push_back("-backend-option");
  CmdArgs.push_back("-mattr=-altivec");
}

Perfect, Hal, thanks! That makes good sense. I'll fix this and also
bring some of the CPU handling more up-to-date (several missing
processors that ought to be in there).

Thanks,
Bill

> > From: "Bill Schmidt" <wschmidt@linux.vnet.ibm.com>
> > To: llvmdev@cs.uiuc.edu
> > Sent: Thursday, January 31, 2013 9:26:15 AM
> > Subject: [LLVMdev] Getting command line options to affect subtarget features
> >
> > The problem I'm trying to solve: Invoking clang on PowerPC with
> > -fno-altivec has no effect.
> >
> > From what I've been able to piece together, PPC.td specifies various
> > CPUs and the processor features available on each. So for example we
> > have:
> >
> > def FeatureAltivec : SubtargetFeature<"altivec","HasAltivec",
> > "true",
> > "Enable Altivec
> > instructions">;
> >
> > def : Processor<"pwr7", G5Itineraries,
> > [DirectivePwr7, FeatureAltivec,
> > FeatureMFOCRF, FeatureFSqrt, FeatureSTFIWX,
> > FeatureISEL, Feature64Bit /*, Feature64BitRegs
> > */]>;
> >
> > During compilation the subtarget class (such as PPCSubtarget) is
> > instantiated with the strings obtained from -mcpu and -mattr.
> > ParseSubtargetFeatures then uses the -mcpu and -mattr values to
> > determine which features to use for code generation.
> >
> > What seems problematic is that -mattr is the only way that the static
> > features of the target processor can be overridden.
> >
> > It seems there must be a way for command-line parameters such as
> > -fno-altivec to be used to override which processor features are
> > exploited. But I can't see immediately how this should work. I
> > don't
> > know whether or how the command line flags from Clang are made
> > available
> > to LLVM for inspection.
> >
> > I'm sure there must be an obvious way to do this, but peeking at code
> > from other targets hasn't enlightened me. Any advice for how I can
> > accomplish this?
>
> Off of the top of my head...
>
> Within the current infrastructure, I think that we need to alter the Clang PowerPC toolchain code (Clang::AddPPCTargetArgs in Tools.cpp) to do something like this:
>
> if (Args.hasFlag(options::OPT_fno_altivec, options::OPT_faltivec, true)) {
> CmdArgs.push_back("-backend-option");
> CmdArgs.push_back("-mattr=-altivec");
> }

Perfect, Hal, thanks! That makes good sense. I'll fix this and also
bring some of the CPU handling more up-to-date (several missing
processors that ought to be in there).

Well, I spoke too soon. I tried this and received the following
objection:

~/src/llvm> $LLVM_INSTALL/bin/clang vectest.c -S -O3 -o vectest.s -fno-altivec
clang: Unknown command line argument '-mattr=-altivec'. Try: 'clang -help'
clang: Did you mean '-mxgot=-altivec'?
~/src/llvm>

Yet I can do the following:

~/src/llvm> $LLVM_INSTALL/bin/llc -O3 -mattr=-altivec vectest.ll
~/src/llvm>

The general mechanism seems valid since code like this exists:

  if (Args.hasFlag(options::OPT_mstackrealign, options::OPT_mno_stackrealign,
                   false)) {
    CmdArgs.push_back("-backend-option");
    CmdArgs.push_back("-force-align-stack");
  }

But it doesn't like -mattr. Anyone have thoughts why?

Thanks,
Bill

>
> > > From: "Bill Schmidt" <wschmidt@linux.vnet.ibm.com>
> > > To: llvmdev@cs.uiuc.edu
> > > Sent: Thursday, January 31, 2013 9:26:15 AM
> > > Subject: [LLVMdev] Getting command line options to affect subtarget features
> > >
> > > The problem I'm trying to solve: Invoking clang on PowerPC with
> > > -fno-altivec has no effect.
> > >
> > > From what I've been able to piece together, PPC.td specifies various
> > > CPUs and the processor features available on each. So for example we
> > > have:
> > >
> > > def FeatureAltivec : SubtargetFeature<"altivec","HasAltivec",
> > > "true",
> > > "Enable Altivec
> > > instructions">;
> > >
> > > def : Processor<"pwr7", G5Itineraries,
> > > [DirectivePwr7, FeatureAltivec,
> > > FeatureMFOCRF, FeatureFSqrt, FeatureSTFIWX,
> > > FeatureISEL, Feature64Bit /*, Feature64BitRegs
> > > */]>;
> > >
> > > During compilation the subtarget class (such as PPCSubtarget) is
> > > instantiated with the strings obtained from -mcpu and -mattr.
> > > ParseSubtargetFeatures then uses the -mcpu and -mattr values to
> > > determine which features to use for code generation.
> > >
> > > What seems problematic is that -mattr is the only way that the static
> > > features of the target processor can be overridden.
> > >
> > > It seems there must be a way for command-line parameters such as
> > > -fno-altivec to be used to override which processor features are
> > > exploited. But I can't see immediately how this should work. I
> > > don't
> > > know whether or how the command line flags from Clang are made
> > > available
> > > to LLVM for inspection.
> > >
> > > I'm sure there must be an obvious way to do this, but peeking at code
> > > from other targets hasn't enlightened me. Any advice for how I can
> > > accomplish this?
> >
> > Off of the top of my head...
> >
> > Within the current infrastructure, I think that we need to alter the Clang PowerPC toolchain code (Clang::AddPPCTargetArgs in Tools.cpp) to do something like this:
> >
> > if (Args.hasFlag(options::OPT_fno_altivec, options::OPT_faltivec, true)) {
> > CmdArgs.push_back("-backend-option");
> > CmdArgs.push_back("-mattr=-altivec");
> > }
>
> Perfect, Hal, thanks! That makes good sense. I'll fix this and also
> bring some of the CPU handling more up-to-date (several missing
> processors that ought to be in there).

Well, I spoke too soon. I tried this and received the following
objection:

~/src/llvm> $LLVM_INSTALL/bin/clang vectest.c -S -O3 -o vectest.s -fno-altivec
clang: Unknown command line argument '-mattr=-altivec'. Try: 'clang -help'
clang: Did you mean '-mxgot=-altivec'?
~/src/llvm>

Yet I can do the following:

~/src/llvm> $LLVM_INSTALL/bin/llc -O3 -mattr=-altivec vectest.ll
~/src/llvm>

The general mechanism seems valid since code like this exists:

  if (Args.hasFlag(options::OPT_mstackrealign, options::OPT_mno_stackrealign,
                   false)) {
    CmdArgs.push_back("-backend-option");
    CmdArgs.push_back("-force-align-stack");
  }

But it doesn't like -mattr. Anyone have thoughts why?

I found some ideas to try in
http://lists.cs.uiuc.edu/pipermail/cfe-dev/2011-September/017087.html.
Perhaps I can try pushing these tokens onto CmdArgs and then enable
altivec as a feature in PPCTargetInfo::setFeatureEnabled():

-Xclang -target-feature -Xclang -altivec

I'll give that a whirl and see what happens.

Bill

> >
> > > > From: "Bill Schmidt" <wschmidt@linux.vnet.ibm.com>
> > > > To: llvmdev@cs.uiuc.edu
> > > > Sent: Thursday, January 31, 2013 9:26:15 AM
> > > > Subject: [LLVMdev] Getting command line options to affect subtarget features
> > > >
> > > > The problem I'm trying to solve: Invoking clang on PowerPC with
> > > > -fno-altivec has no effect.
> > > >
> > > > From what I've been able to piece together, PPC.td specifies various
> > > > CPUs and the processor features available on each. So for example we
> > > > have:
> > > >
> > > > def FeatureAltivec : SubtargetFeature<"altivec","HasAltivec",
> > > > "true",
> > > > "Enable Altivec
> > > > instructions">;
> > > >
> > > > def : Processor<"pwr7", G5Itineraries,
> > > > [DirectivePwr7, FeatureAltivec,
> > > > FeatureMFOCRF, FeatureFSqrt, FeatureSTFIWX,
> > > > FeatureISEL, Feature64Bit /*, Feature64BitRegs
> > > > */]>;
> > > >
> > > > During compilation the subtarget class (such as PPCSubtarget) is
> > > > instantiated with the strings obtained from -mcpu and -mattr.
> > > > ParseSubtargetFeatures then uses the -mcpu and -mattr values to
> > > > determine which features to use for code generation.
> > > >
> > > > What seems problematic is that -mattr is the only way that the static
> > > > features of the target processor can be overridden.
> > > >
> > > > It seems there must be a way for command-line parameters such as
> > > > -fno-altivec to be used to override which processor features are
> > > > exploited. But I can't see immediately how this should work. I
> > > > don't
> > > > know whether or how the command line flags from Clang are made
> > > > available
> > > > to LLVM for inspection.
> > > >
> > > > I'm sure there must be an obvious way to do this, but peeking at code
> > > > from other targets hasn't enlightened me. Any advice for how I can
> > > > accomplish this?
> > >
> > > Off of the top of my head...
> > >
> > > Within the current infrastructure, I think that we need to alter the Clang PowerPC toolchain code (Clang::AddPPCTargetArgs in Tools.cpp) to do something like this:
> > >
> > > if (Args.hasFlag(options::OPT_fno_altivec, options::OPT_faltivec, true)) {
> > > CmdArgs.push_back("-backend-option");
> > > CmdArgs.push_back("-mattr=-altivec");
> > > }
> >
> > Perfect, Hal, thanks! That makes good sense. I'll fix this and also
> > bring some of the CPU handling more up-to-date (several missing
> > processors that ought to be in there).
>
> Well, I spoke too soon. I tried this and received the following
> objection:
>
> ~/src/llvm> $LLVM_INSTALL/bin/clang vectest.c -S -O3 -o vectest.s -fno-altivec
> clang: Unknown command line argument '-mattr=-altivec'. Try: 'clang -help'
> clang: Did you mean '-mxgot=-altivec'?
> ~/src/llvm>
>
> Yet I can do the following:
>
> ~/src/llvm> $LLVM_INSTALL/bin/llc -O3 -mattr=-altivec vectest.ll
> ~/src/llvm>
>
> The general mechanism seems valid since code like this exists:
>
> if (Args.hasFlag(options::OPT_mstackrealign, options::OPT_mno_stackrealign,
> false)) {
> CmdArgs.push_back("-backend-option");
> CmdArgs.push_back("-force-align-stack");
> }
>
> But it doesn't like -mattr. Anyone have thoughts why?

I found some ideas to try in
http://lists.cs.uiuc.edu/pipermail/cfe-dev/2011-September/017087.html.
Perhaps I can try pushing these tokens onto CmdArgs and then enable
altivec as a feature in PPCTargetInfo::setFeatureEnabled():

-Xclang -target-feature -Xclang -altivec

I'll give that a whirl and see what happens.

Nope, that's no good either (as I probably should have expected):

wschmidt@bns:~/src/llvm> $LLVM_INSTALL/bin/clang vectest.c -S -O3 -o vectest.s -fno-altivec
error: unknown argument: '-Xclang -target-feature'
error: unknown argument: '-Xclang -altivec'
wschmidt@bns:~/src/llvm>

I'm officially out of ideas...

From: "Bill Schmidt" <wschmidt@linux.vnet.ibm.com>
To: "Hal Finkel" <hfinkel@anl.gov>
Cc: llvmdev@cs.uiuc.edu
Sent: Thursday, January 31, 2013 11:40:24 AM
Subject: Re: [LLVMdev] Getting command line options to affect subtarget features

> > >
> > > > > From: "Bill Schmidt" <wschmidt@linux.vnet.ibm.com>
> > > > > To: llvmdev@cs.uiuc.edu
> > > > > Sent: Thursday, January 31, 2013 9:26:15 AM
> > > > > Subject: [LLVMdev] Getting command line options to affect
> > > > > subtarget features
> > > > >
> > > > > The problem I'm trying to solve: Invoking clang on PowerPC
> > > > > with
> > > > > -fno-altivec has no effect.
> > > > >
> > > > > From what I've been able to piece together, PPC.td
> > > > > specifies various
> > > > > CPUs and the processor features available on each. So for
> > > > > example we
> > > > > have:
> > > > >
> > > > > def FeatureAltivec :
> > > > > SubtargetFeature<"altivec","HasAltivec",
> > > > > "true",
> > > > > "Enable Altivec
> > > > > instructions">;
> > > > >
> > > > > def : Processor<"pwr7", G5Itineraries,
> > > > > [DirectivePwr7, FeatureAltivec,
> > > > > FeatureMFOCRF, FeatureFSqrt,
> > > > > FeatureSTFIWX,
> > > > > FeatureISEL, Feature64Bit /*,
> > > > > Feature64BitRegs
> > > > > */]>;
> > > > >
> > > > > During compilation the subtarget class (such as
> > > > > PPCSubtarget) is
> > > > > instantiated with the strings obtained from -mcpu and
> > > > > -mattr.
> > > > > ParseSubtargetFeatures then uses the -mcpu and -mattr
> > > > > values to
> > > > > determine which features to use for code generation.
> > > > >
> > > > > What seems problematic is that -mattr is the only way that
> > > > > the static
> > > > > features of the target processor can be overridden.
> > > > >
> > > > > It seems there must be a way for command-line parameters
> > > > > such as
> > > > > -fno-altivec to be used to override which processor
> > > > > features are
> > > > > exploited. But I can't see immediately how this should
> > > > > work. I
> > > > > don't
> > > > > know whether or how the command line flags from Clang are
> > > > > made
> > > > > available
> > > > > to LLVM for inspection.
> > > > >
> > > > > I'm sure there must be an obvious way to do this, but
> > > > > peeking at code
> > > > > from other targets hasn't enlightened me. Any advice for
> > > > > how I can
> > > > > accomplish this?
> > > >
> > > > Off of the top of my head...
> > > >
> > > > Within the current infrastructure, I think that we need to
> > > > alter the Clang PowerPC toolchain code
> > > > (Clang::AddPPCTargetArgs in Tools.cpp) to do something like
> > > > this:
> > > >
> > > > if (Args.hasFlag(options::OPT_fno_altivec,
> > > > options::OPT_faltivec, true)) {
> > > > CmdArgs.push_back("-backend-option");
> > > > CmdArgs.push_back("-mattr=-altivec");
> > > > }
> > >
> > > Perfect, Hal, thanks! That makes good sense. I'll fix this
> > > and also
> > > bring some of the CPU handling more up-to-date (several missing
> > > processors that ought to be in there).
> >
> > Well, I spoke too soon. I tried this and received the following
> > objection:
> >
> > ~/src/llvm> $LLVM_INSTALL/bin/clang vectest.c -S -O3 -o vectest.s
> > -fno-altivec
> > clang: Unknown command line argument '-mattr=-altivec'. Try:
> > 'clang -help'
> > clang: Did you mean '-mxgot=-altivec'?
> > ~/src/llvm>
> >
> > Yet I can do the following:
> >
> > ~/src/llvm> $LLVM_INSTALL/bin/llc -O3 -mattr=-altivec vectest.ll
> > ~/src/llvm>
> >
> > The general mechanism seems valid since code like this exists:
> >
> > if (Args.hasFlag(options::OPT_mstackrealign,
> > options::OPT_mno_stackrealign,
> > false)) {
> > CmdArgs.push_back("-backend-option");
> > CmdArgs.push_back("-force-align-stack");
> > }
> >
> > But it doesn't like -mattr. Anyone have thoughts why?
>
> I found some ideas to try in
> http://lists.cs.uiuc.edu/pipermail/cfe-dev/2011-September/017087.html.
> Perhaps I can try pushing these tokens onto CmdArgs and then enable
> altivec as a feature in PPCTargetInfo::setFeatureEnabled():
>
> -Xclang -target-feature -Xclang -altivec
>
> I'll give that a whirl and see what happens.

Nope, that's no good either (as I probably should have expected):

wschmidt@bns:~/src/llvm> $LLVM_INSTALL/bin/clang vectest.c -S -O3 -o
vectest.s -fno-altivec
error: unknown argument: '-Xclang -target-feature'
error: unknown argument: '-Xclang -altivec'
wschmidt@bns:~/src/llvm>

I'm officially out of ideas...

This is indeed a bit convoluted. mattr is declared in llvm/CodeGen/CommandFlags.h. So is code-model, so looking at how that works...

Frontend/CompilerInvocation.cpp pulls out -mcode-model into an options structure:
  Opts.CodeModel = Args.getLastArgValue(OPT_mcode_model)

Then in CodeGen/BackendUtil.cpp, this is transformed into an argument to createTargetMachine

  // FIXME: Parse this earlier.
  llvm::CodeModel::Model CM;
  if (CodeGenOpts.CodeModel == "small") {
    CM = llvm::CodeModel::Small;
  } else if (CodeGenOpts.CodeModel == "kernel") {
    CM = llvm::CodeModel::Kernel;
...
  TargetMachine *TM = TheTarget->createTargetMachine(Triple, TargetOpts.CPU,
                                                     FeaturesStr, Options,
                                                     RM, CM, OptLevel);

I think that what we want for altivec is actually to put things into FeaturesStr. This is populated by:

  std::string FeaturesStr;
  if (TargetOpts.Features.size()) {
    SubtargetFeatures Features;
    for (std::vector<std::string>::const_iterator
           it = TargetOpts.Features.begin(),
           ie = TargetOpts.Features.end(); it != ie; ++it)
      Features.AddFeature(*it);
    FeaturesStr = Features.getString();
  }

the code that puts things into Features is in Basic/Targets.cpp (to get defaults) and Driver/Tools.cpp (to get command-line options). Unfortunately, the code in Driver/Tools.cpp is in Clang::AddX86TargetArgs (and, thus, is X86 specific). It seems like some refactoring might be in order.

-Hal