Nightly Testers

Would those of you running the nightly tester make sure they are using the NEW default script "/nightlytest/NightlyTestAccept.php" (was /nightlytest/NightlyTestAccept.cgi",) This would require that you have an updated llvm/utils/NewNightlyTest.pl and that you are not overriding with a different script using -submit-script=.

Thank you.

Cheers,

-- Jim

Jim,

How are the nightly test machines identified? I went to look at my
machine's test results and couldn't find any (for a while) because a new
machine id had been set. I'd like to avoid creating new machine ids so
could you please indicate what factors lead to one run of nightly test
being viewed as a different machine? If you explain it, I'll add the
explanation to the documentation.

Thanks,

Reid.

I don't recall the details, but I believe that all of the architecture, base compiler version, nickname and probably other stuff factor into it.

Upgrading the system compiler (in particular) can have a significant impact on the performance results, so I think this behavior makes sense. If you strongly disagree and want to change it, go for it.

-Chris

Hi Chris,

> How are the nightly test machines identified? I went to look at my
> machine's test results and couldn't find any (for a while) because a new
> machine id had been set. I'd like to avoid creating new machine ids so
> could you please indicate what factors lead to one run of nightly test
> being viewed as a different machine? If you explain it, I'll add the
> explanation to the documentation.

I don't recall the details, but I believe that all of the architecture,
base compiler version, nickname and probably other stuff factor into it.

I talked with Jim about it off-list. He indicated that the factors are:

Username, hardware, OS, machine name, nickname and gcc version.

Upgrading the system compiler (in particular) can have a significant
impact on the performance results, so I think this behavior makes sense.
If you strongly disagree and want to change it, go for it.

I had upgraded my Linux kernel from 2.6.17 to 2.6.18. I suggested to Jim
that changes in the minor revision number of the operating system should
not count as a new machine. He said he'd look into it when he has time.
Its not a huge issue, I just didn't want to be incrementing the machine
numbers needlessly and needed to know what it was based on.

Reid.

Ok, seems reasonable. The nightly tester script probably shouldn't be in the business of trying to grok OS version #'s. Maybe OS version # shouldn't be included at all.

-Chris

That's an easy fix. However, I do need to merge data (correctly) based on the change. So it will take 3 or 4 wags.

Cheers,

-- Jim

I think it needs to include it. Suppose someone used the same user ID,
same nickname but changed the machine from Linux to Windows. Furthermore
even a change like Linux 2.6 to Linux 2.8 could impact performance
significantly so it should be regarded as a new machine. The complexity
of dealing with the OS version #s is why Jim tabled the work for now. I
agree, its an ugly mess that NightlyTest shouldn't be involved in. But I
don't think the correct thing to do is remove OS version from the
comparison altogether. I'd rather have the current functionality than OS
not mattering at all.

Reid.

Too late. Made the change. I can revert, but past records have been merged.

But why wouldn't you use a different nickname between windows and linux?

Transitioning to a new OS major version, will only take the one day hit. You wouldn't have a prior set to compare anyway, so no loss of info. Potentially, you get whine data for hits the OS caused (ex., 30 tests newly fail because of header changes.)

Cheers,

-- Jim