The index into the tokens list of the current token (next token to consume). tokens[p] should be LT(1). p=-1 indicates need to initialize with first token. The ctor doesn't get a token. First call to LT(1) or whatever gets the first token and sets p=0;
Record every single token pulled from the source so we can reproduce chunks of it later. The buffer in LookaheadStream overlaps sometimes as its moving window moves through the input. This list captures everything so we can access complete input text.
How deep have we gone?
Move the input pointer to the next incoming token. The stream must become active with LT(1) available. consume() simply moves the input pointer so that LT(1) points at the next input symbol. Consume at least one token.
Walk past any token not on the channel the parser is listening to.
add n elements to buffer
Make sure index i in tokens has a token.
Generated using TypeDoc
Buffer all input tokens but do on-demand fetching of new tokens from lexer. Useful when the parser or lexer has to set context/mode info before proper lexing of future tokens. The ST template parser needs this, for example, because it has to constantly flip back and forth between inside/output templates. E.g.,<names:{hi, <it>}> has to parse names
as part of an expression but "hi, <it>" as a nested template.
You can't use this stream if you pass whitespace or other off-channel tokens to the parser. The stream can't ignore off-channel tokens. (UnbufferedTokenStream is the same way.)
This is not a subclass of UnbufferedTokenStream because I don't want to confuse small moving window of tokens it uses for the full buffer.