class Crass::Scanner
def marked
Returns the substring between {#marker} and {#pos}, without altering the
def marked # Extract the marked text using byte offsets rather than character # offsets. Slicing the original string by character offset is O(n) on # multi-byte input (Ruby must translate the character index into a byte # index), which makes tokenizing non-ASCII input superlinear. Byte slicing # is O(length) regardless of how far into the string we are. @string.byteslice(@byte_marker, @scanner.pos - @byte_marker) || '' end