Really nasty remat bug [LONG]

Ok, I've been tracking down a remat bug for over a week. I think I finally
understand what is going on. This happens in specfp2006 leslie3d
compiled with our frontend and optimizer. Unfortunately, getting a testcase
is going to be impossible due to restrictions on SPEC redistribution and
the complexity of the sequence of events leading to the bug.

The bug is extremely subtle and nasty. Therefore this explanation is very
long and complicated. I'm looking for a check on my analysis and suggestions
of how best to fix the problem.

This is with 2.3 plus a few extra patches (from upstream) that fix bugs
in the regalloc/spill area. I believe the bug exists in TOT, however.
There's nothing I see in the logs that indicates anything has been changed
to fix this problem.

My comments delimited with #####. The rest is llvm debug output (lots
of stuff added by me so you won't get the same output with upstream
llvm -debug).

Hooray! I have a fix for this. I filed PR2634 as a placeholder and history
marker so I can refer to it in the commit log once management signs
off on sending the patch.


Thanks Dave!