Fix for preprocessor paste bug

Clang Frontend
#1

Hi,

Attached is a patch fixing a preprocessor paste bug.

The bug manifests itself by swallowing up the remainder of the file
following an invalid paste operation. In terms of technical details,
TokenLexer::PasteTokens was returning tok::eof in this case, effectively
stopping any further processing. The bug includes a test-case demonstrating
the fix. No other regressions are noted.

Cheers
Andy

P.S. For Richard Smith, who is keeping my wits sharp on the "-verify"
enhancement, this solves one of the unnecessary test-case changes! ;o)

paste-fix.diff (1 KB)

#2

Hi Andy,

Attached is a patch fixing a preprocessor paste bug.

Great, thanks!

The bug manifests itself by swallowing up the remainder of the file
following an invalid paste operation. In terms of technical details,
TokenLexer::PasteTokens was returning tok::eof in this case, effectively
stopping any further processing. The bug includes a test-case demonstrating
the fix. No other regressions are noted.

In -fms-extension -Wno-invalid-token-paste mode, the patch regresses this test, because we’re no longer invoking LookUpIdentifierInfo on the pasted ‘ab’ token:

#define foo a ## b ## = 0
int foo;

Switching out the ‘return true’ for a ‘break’ should resolve that, I think. Your fix should also apply in the AsmPreprocessor case.

Can you merge your test case into the existing file?

#3

Hi again,

In -fms-extension -Wno-invalid-token-paste mode, the patch
regresses this test, because we're no longer invoking
LookUpIdentifierInfo on the pasted 'ab' token:

#define foo a ## b ## = 0
int foo;

Switching out the 'return true' for a 'break' should resolve
that, I think. Your fix should also apply in the
AsmPreprocessor case.

Your wish is my command! Attached is a revised patch.

Cheers
Andy

paste-fix.diff (1.49 KB)

#4

Many thanks, r158412. [I slightly tweaked your tests.]