Accuracy of -ftime-report

Hi,

When I try ‘-ftime-report’ the numbers differ from ‘time’, either higher or lower, e.g:

time clang -cc1 lib/VMCore/Dominators.cpp -Iinclude -Ilib/VMCore -D_DEBUG -D_GNU_SOURCE -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -ftime-report -emit-llvm-bc -o t.o

[LLVM pass timers]

That can be due to lots of things. Just consider this difference:

  • Wall time: preception of time form the user. That is, if you run your program at time T and it ends at time T+D, then the wall time is D normally
  • Processor time: time allocated on the processor, which can even be greater than the wall time if the program uses several processors (if you have two processors, you can in theory use 2*D processor time for a wall time of D).

I don’t know how the times are computed but it could be related to these points. And perhaps the time needed to do all the initialisation and loading of the executable is counted somewhere.

Amaury Pouly

2010/8/5 Argyrios Kyrtzidis <kyrtzidis@apple.com>

That can be due to lots of things. Just consider this difference:

  • Wall time: preception of time form the user. That is, if you run your program at time T and it ends at time T+D, then the wall time is D normally
  • Processor time: time allocated on the processor, which can even be greater than the wall time if the program uses several processors (if you have two processors, you can in theory use 2*D processor time for a wall time of D).

I don’t know how the times are computed but it could be related to these points. And perhaps the time needed to do all the initialisation and loading of the executable is counted somewhere.

Yes but the difference between the ‘time’ utility and the internal timers is not constant, with ‘-fsyntax-only’ the internal timers report lower numbers but with ‘-emit-llvm-bc’ they report higher ones.

In general is ‘time’ utility the more reliable one ?

-Argiris

One would need to check but other things are important. For example, do the timers encompass the time during which clang/llvm read/write files (not even parsing them but just open/read/write/close) ?

As a matter of fact, on linux, llvm obtains time statistics with gettimeofday which has an unspecified precision. So it’s hard to compare with time which gives you the time reported by the kernel.
I would say that if you want the total time need by the process, pick the one of ‘time’, if you need the time per phase, then pick the one of the output and use relative figures only (percentage of the total time).

Amaury Pouly

2010/8/5 Argyrios Kyrtzidis <kyrtzidis@apple.com>

That can be due to lots of things. Just consider this difference:
- Wall time: preception of time form the user. That is, if you run your
program at time T and it ends at time T+D, then the wall time is D normally
- Processor time: time allocated on the processor, which can even be greater
than the wall time if the program uses several processors (if you have two
processors, you can in theory use 2*D processor time for a wall time of D).

I don't know how the times are computed but it could be related to these
points. And perhaps the time needed to do all the initialisation and loading
of the executable is counted somewhere.

Yes but the difference between the 'time' utility and the internal timers is
not constant, with '-fsyntax-only' the internal timers report lower numbers
but with '-emit-llvm-bc' they report higher ones.
In general is 'time' utility the more reliable one ?

The individual timers are just as reliable as 'time', for tracking
user time, but you have to know what they mean (i.e., exactly what
they are counting).

What is happening here is that some of the timers overlap, I suspect.
The Clang frontend timer probably is ending up including the LLVM IR
generation timer.

- Daniel

That can be due to lots of things. Just consider this difference:
- Wall time: preception of time form the user. That is, if you run your
program at time T and it ends at time T+D, then the wall time is D normally
- Processor time: time allocated on the processor, which can even be greater
than the wall time if the program uses several processors (if you have two
processors, you can in theory use 2*D processor time for a wall time of D).

I don't know how the times are computed but it could be related to these
points. And perhaps the time needed to do all the initialisation and loading
of the executable is counted somewhere.

Yes but the difference between the 'time' utility and the internal timers is
not constant, with '-fsyntax-only' the internal timers report lower numbers
but with '-emit-llvm-bc' they report higher ones.
In general is 'time' utility the more reliable one ?

The individual timers are just as reliable as 'time', for tracking
user time, but you have to know what they mean (i.e., exactly what
they are counting).

What is happening here is that some of the timers overlap, I suspect.
The Clang frontend timer probably is ending up including the LLVM IR
generation timer.

How do you suggest timing the phases ? If the timers overlap, the percentages are not reliable, right ?
Use 'time' on separate executions (time -fsyntax-only, time -emit-llvm, etc) ?

-Argiris

That can be due to lots of things. Just consider this difference:
- Wall time: preception of time form the user. That is, if you run your
program at time T and it ends at time T+D, then the wall time is D normally
- Processor time: time allocated on the processor, which can even be greater
than the wall time if the program uses several processors (if you have two
processors, you can in theory use 2*D processor time for a wall time of D).

I don't know how the times are computed but it could be related to these
points. And perhaps the time needed to do all the initialisation and loading
of the executable is counted somewhere.

Yes but the difference between the 'time' utility and the internal timers is
not constant, with '-fsyntax-only' the internal timers report lower numbers
but with '-emit-llvm-bc' they report higher ones.
In general is 'time' utility the more reliable one ?

The individual timers are just as reliable as 'time', for tracking
user time, but you have to know what they mean (i.e., exactly what
they are counting).

What is happening here is that some of the timers overlap, I suspect.
The Clang frontend timer probably is ending up including the LLVM IR
generation timer.

How do you suggest timing the phases ? If the timers overlap, the percentages are not reliable, right ?

Does that matter? The absolute values are still comparable. Presumably
you are wanting to check before-and-after numbers?

That said, I personally almost always use something like 'time' for my
measurements, once I have a test case that demonstrates the
performance of the component I am change.

- Daniel