The index into the tokens list of the current token (next token to consume). tokens[p] should be LT(1). p=-1 indicates need to initialize with first token. The ctor doesn't get a token. First call to LT(1) or whatever gets the first token and sets p=0;
Record every single token pulled from the source so we can reproduce chunks of it later. The buffer in LookaheadStream overlaps sometimes as its moving window moves through the input. This list captures everything so we can access complete input text.
Skip tokens on any channel but this one; this is how we skip whitespace...
How deep have we gone?
Always leave p on an on-channel token.
add n elements to buffer
Given a starting index, return the index of the first on-channel token.
Make sure index i in tokens has a token.
Generated using TypeDoc
The most common stream of tokens is one where every token is buffered up and tokens are prefiltered for a certain channel (the parser will only see these tokens and cannot change the filter channel number during the parse).
TODO: how to access the full token stream? How to track all tokens matched per rule?