class Syntax::Tokenizer

used to specify optional behavior.
not) publish any options, but if a tokenizer does those options may be
Specify a set of tokenizer-specific options. Each tokenizer may (or may

def set( opts={} )
  ( @options ||= Hash.new ).update opts
end