[RFC] Removal of Pretokenized Headers

Hi all-

I recently found a bug compiling Boost Math library with their version of “PCH” (which actually by -XClang flag uses PTH instead, though I have a bug report filed to switch them over to PCH instead). Upon asking on IRC, it was noted that this feature is considered by many as a failed expired experiment and I was encouraged to remove it. See the patch to do so here: https://reviews.llvm.org/D54547 I’ve found another few modifications to IdentifierInfo/IdentifierTable that Bruno Ricci and I are still digging into that we suspect will provide significant performance benefits as well.

Research has shown that this functionality has been unmaintained for roughly 9 years, and has had significant bit-rot ever since. The bug I’d found makes it broken for about 1/3 of possible tokens (which, by shear happenstance “works” with Boost’s usage of __is_signed, at least until we add a token in the wrong place!), though many more are likely. Research has also shown that the Boost::Build (and only Boost::Math seems to use this) is the only user of PTH that I could find, and I believe they use it erroneously.

I was encouraged to announce this on this mailing list before acceptance of the patch.

Are there any issues anyone has with this?