RFC - Stop ignoring -fprofile-generate and -fprofile-use

The flags -fprofile-generate and -fprofile-use are currently ignored for GCC compatibility. I would like to enable them and give them similar semantics to GCC. These flags are baked pretty deeply into our build environment, so supporting them at the driver level will make our lives a lot simpler.

From https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html:

`-fprofile-generate`
`-fprofile-generate=`
Enable options usually used for instrumenting application to produce profile useful for later recompilation with profile feedback based optimization. You must use both when compiling and when linking your program.

The following options are enabled: , , .

If is specified, GCC looks at the to find the profile feedback data files. See .

`-fprofile-use`
`-fprofile-use=`
Enable profile feedback-directed optimizations, and the following optimizations which are generally profitable only with profile feedback available: , , , , , , and .

By default, GCC emits an error message if the feedback profiles do not match the source code. This error can be turned into a warning by using . Note this may result in poorly optimized code.

If is specified, GCC looks at the to find the profile feedback data files. See .

Note that the argument to -fprofile-generate and -fprofile-use is not a file name. It is a path prefix used to store the generated profile (in GCC’s case, each object file in the binary generates its own profile file). In Clang, the flags would do the following:

  1. -fprofile-generate=path-prefix would cause the instrumented binary to write the profile /default.profraw. If does not exist, it is created by the runtime.
  2. -fprofile-use=path-prefix would cause the compiler to read from /default.profile.
  3. I could also add support for -fprofile-dir, but we don’t really use it internally.
    As with -fprofile-instr-generate, LLVM_PROFILE_FILE would override the path prefix and name of the profile file.

Does this sound reasonable?

Thanks. Diego.

+cfe-dev

The flags -fprofile-generate and -fprofile-use are currently ignored
for GCC compatibility. I would like to enable them and give them
similar semantics to GCC. These flags are baked pretty deeply into
our build environment, so supporting them at the driver level will
make our lives a lot simpler.

This seems like a fairly reasonable idea. We'll have to make sure it's
clear (in the docs or whatever) that you can't use clang to generate a
profile and gcc to consume it and vice versa, but that doesn't seem like
a big deal.

From https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html:

-fprofile-generate
-fprofile-generate=path
Enable options usually used for instrumenting application to produce
profile useful for later recompilation with profile feedback based
optimization. You must use -fprofile-generate both when compiling
and when linking your program.
The following options are enabled: -fprofile-arcs, -fprofile-values, -fvpt.

If path is specified, GCC looks at the path to find the profile
feedback data files. See -fprofile-dir.

-fprofile-use
-fprofile-use=path
Enable profile feedback-directed optimizations, and the following
optimizations which are generally profitable only with profile
feedback available: -fbranch-probabilities, -fvpt, -funroll-loops,
-fpeel-loops, -ftracer, -ftree-vectorize, and
ftree-loop-distribute-patterns.
By default, GCC emits an error message if the feedback profiles do
not match the source code. This error can be turned into a warning
by using -Wcoverage-mismatch. Note this may result in poorly
optimized code.

If path is specified, GCC looks at the path to find the profile
feedback data files. See -fprofile-dir.

Note that the argument to -fprofile-generate and -fprofile-use is
not a file name. It is a path prefix used to store the generated
profile (in GCC's case, each object file in the binary generates its
own profile file). In Clang, the flags would do the following:

  • -fprofile-generate=path-prefix would cause the instrumented
    binary to write the profile <path-prefix>/default.profraw. If
    <path-prefix> does not exist, it is created by the runtime.
  • -fprofile-use=path-prefix would cause the compiler to read
    from <path-prefix>/default.profile.

The path-prefix versions are a bit of an odd fit for the instrprof
stuff - I'd prefer if they dealt with the path to the file. I can see
how that might be difficult to do and still be compatible though.

Maybe we could make the -fprofile-use= version accept a file or
directory, and if a directory is specified then look for the default?

The other thing that would be nice would be if -fprofile-use=path could
autodetect which kind of profile the path is - that is, It'd be nice to
accept both instrprof and sample profiles with this option. I'd limit
this to the path-to-a-file case though, I think it's too magical when
pointing at a directory.

  • I could also add support for -fprofile-dir, but we don't
    really use it internally.

I don't think it's necessary unless someone actually wants to use it.

The path-prefix versions are a bit of an odd fit for the instrprof
stuff - I'd prefer if they dealt with the path to the file. I can see
how that might be difficult to do and still be compatible though.

Maybe we could make the -fprofile-use= version accept a file or
directory, and if a directory is specified then look for the default?

That sounds reasonable.

The other thing that would be nice would be if -fprofile-use=path could
autodetect which kind of profile the path is - that is, It'd be nice to
accept both instrprof and sample profiles with this option. I'd limit
this to the path-to-a-file case though, I think it's too magical when
pointing at a directory.

Agree.

David

> • -fprofile-generate=path-prefix would cause the instrumented
> binary to write the profile <path-prefix>/default.profraw. If
> <path-prefix> does not exist, it is created by the runtime.
> • -fprofile-use=path-prefix would cause the compiler to read
> from <path-prefix>/default.profile.

The path-prefix versions are a bit of an odd fit for the instrprof
stuff - I'd prefer if they dealt with the path to the file. I can see
how that might be difficult to do and still be compatible though.

Maybe we could make the -fprofile-use= version accept a file or
directory, and if a directory is specified then look for the default?

Sure. Sounds reasonable.

The other thing that would be nice would be if -fprofile-use=path could
autodetect which kind of profile the path is - that is, It'd be nice to
accept both instrprof and sample profiles with this option.

Sure. But this may have a wrinkle. The problem here is that GCC already
has a different option for sample profiles (-fauto-profile). Someone using
the same flags for both compilers (or migrating off of GCC) might find this
problematic. Although, I can't really think a scenario where this would
happen.

In a sense, anything that is a superset of GCC's current behaviour should
be fine. So, I suppose this is OK.

I'd limit
this to the path-to-a-file case though, I think it's too magical when
pointing at a directory.

Agreed.

Diego.

> • -fprofile-generate=path-prefix would cause the instrumented
> binary to write the profile <path-prefix>/default.profraw. If
> <path-prefix> does not exist, it is created by the runtime.
> • -fprofile-use=path-prefix would cause the compiler to read
> from <path-prefix>/default.profile.

The path-prefix versions are a bit of an odd fit for the instrprof
stuff - I'd prefer if they dealt with the path to the file. I can see
how that might be difficult to do and still be compatible though.

Maybe we could make the -fprofile-use= version accept a file or
directory, and if a directory is specified then look for the default?

Sure. Sounds reasonable.

The other thing that would be nice would be if -fprofile-use=path could
autodetect which kind of profile the path is - that is, It'd be nice to
accept both instrprof and sample profiles with this option.

Sure. But this may have a wrinkle. The problem here is that GCC already has
a different option for sample profiles (-fauto-profile). Someone using the
same flags for both compilers (or migrating off of GCC) might find this
problematic. Although, I can't really think a scenario where this would
happen.

In a sense, anything that is a superset of GCC's current behaviour should be
fine. So, I suppose this is OK.

yes -- you are not removing the old behavior.

Diego Novillo <dnovillo@google.com> writes:

  • -fprofile-generate=path-prefix would cause the instrumented
    binary to write the profile <path-prefix>/default.profraw. If
    <path-prefix> does not exist, it is created by the runtime.
  • -fprofile-use=path-prefix would cause the compiler to read
    from <path-prefix>/default.profile.

The path-prefix versions are a bit of an odd fit for the instrprof
stuff - I'd prefer if they dealt with the path to the file. I can see
how that might be difficult to do and still be compatible though.

Maybe we could make the -fprofile-use= version accept a file or
directory, and if a directory is specified then look for the default?

Sure. Sounds reasonable.

The other thing that would be nice would be if -fprofile-use=path could
autodetect which kind of profile the path is - that is, It'd be nice to
accept both instrprof and sample profiles with this option.

Sure. But this may have a wrinkle. The problem here is that GCC already has a
different option for sample profiles (-fauto-profile). Someone using the same
flags for both compilers (or migrating off of GCC) might find this
problematic. Although, I can't really think a scenario where this would
happen.

Convince gcc to do this too? :wink:

I'm not terribly attached to this idea if you think it's a bad precedent
to set or something, but it does seem like it'd be convenient enough to
be worthwhile.