class Crass::TokenScanner
Like {Scanner}, but for tokens!
def collect
Executes the given block, collects all tokens that are consumed during its
def collect start = @pos yield @tokens[start...@pos] || [] end
def consume
Consumes the next token and returns it, advancing the pointer. Returns
def consume @current = @tokens[@pos] @pos += 1 if @current @current end
def initialize(tokens)
def initialize(tokens) @tokens = tokens.to_a reset end
def peek
Returns the next token without consuming it, or `nil` if there is no next
def peek @tokens[@pos] end
def reconsume
Reconsumes the current token, moving the pointer back one position.
def reconsume @pos -= 1 if @pos > 0 end
def reset
def reset @current = nil @pos = 0 end