Hi,
In cling we have line-by-line input that comes from a terminal-like prompt.
We do incremental compilation of the input. The input lines come as llvm::MemoryBuffers
We compile each memory buffer by passing them to clang. However, when clang
parses a buffer it sees EOF in the end and destroys it's current lexer and what not.
For example cling can have:
[cling$] extern "C" int printf(const char* fmt, ...);
[cling$] int i = 12;
[cling$] printf("%d\n", i);
Every line comes in memory buffer containing \0 in the end. Clang considers that as
an EOF. Ideally I want to tell the parser that the parsing of the translation unit is not
yet done, but that it should be suspended until next user's input.
Correct me if I am wrong, but the best way of doing that would be to implement a
'suspend' token. When the lexer and parser see that 'suspend' token they would stop
as if it was EOF token but without deleting/cleaning anything, so that the parsing could
be restarted later with the same state.
If this is the right approach what would be the best way to represent the 'suspend'
token? i.e. which ascii char that would trigger the suspension? (I'd really like to have
"$" but it is already part of an extension).
Vassil
Hi,
In cling we have line-by-line input that comes from a terminal-like prompt.
We do incremental compilation of the input. The input lines come as
llvm::MemoryBuffers
We compile each memory buffer by passing them to clang. However, when clang
parses a buffer it sees EOF in the end and destroys it's current lexer
and what not.
For example cling can have:
[cling$] extern "C" int printf(const char* fmt, ...);
[cling$] int i = 12;
[cling$] printf("%d\n", i);
Every line comes in memory buffer containing \0 in the end. Clang
considers that as
an EOF. Ideally I want to tell the parser that the parsing of the
translation unit is not
yet done, but that it should be suspended until next user's input.
Correct me if I am wrong, but the best way of doing that would be to
implement a
'suspend' token. When the lexer and parser see that 'suspend' token
they would stop
as if it was EOF token but without deleting/cleaning anything, so that
the parsing could
be restarted later with the same state.
'suspend' should probably just be a special handling of the 'eof' token, so that the parser/lexer doesn't tear everything down (but otherwise acts exactly the same). We don't want the parser or preprocessor to have to check 'is this eof or suspend?' every time it currently checks for eof.
If this is the right approach what would be the best way to represent
the 'suspend'
token? i.e. which ascii char that would trigger the suspension? (I'd
really like to have
"$" but it is already part of an extension).
I suggest looking at how the code-completion token is created. It uses \0 + a file offset to distinguish between a \0 at the end of the buffer and an embedded \0 that is the code completion point.
- Doug