class Syntax::Token
lexeme.
itself, decorated with a ‘group’ attribute to identify the type of the
A single token extracted by a tokenizer. It is simply the lexeme
def initialize( text, group, instruction = :none )
Create a new Token representing the given text, and belonging to the
def initialize( text, group, instruction = :none ) super text @group = group @instruction = instruction end