module ERB::Util

:nodoc:
nobody should use it.
Tokenizes a line of ERB. This is really just for error reporting and

def self.tokenize(source) # :nodoc:
  require "strscan"
  source = StringScanner.new(source.chomp)
  tokens = []
  start_re = /<%(?:={1,2}|-|\#|%)?/m
  finish_re = /(?:[-=])?%>/m
  while !source.eos?
    pos = source.pos
    source.scan_until(/(?:#{start_re}|#{finish_re})/)
    raise NotImplementedError if source.matched.nil?
    len = source.pos - source.matched.bytesize - pos
    case source.matched
    when start_re
      tokens << [:TEXT, source.string[pos, len]] if len > 0
      tokens << [:OPEN, source.matched]
      if source.scan(/(.*?)(?=#{finish_re}|\z)/m)
        tokens << [:CODE, source.matched] unless source.matched.empty?
        tokens << [:CLOSE, source.scan(finish_re)] unless source.eos?
      else
        raise NotImplementedError
      end
    when finish_re
      tokens << [:CODE, source.string[pos, len]] if len > 0
      tokens << [:CLOSE, source.matched]
    else
      raise NotImplementedError, source.matched
    end
  end
  tokens
end

# => "<< Accept & Checkout"
html_escape_once('<< Accept & Checkout')

# => "1 < 2 & 3"
html_escape_once('1 < 2 & 3')

A utility method for escaping HTML without affecting existing escaped entities.

def html_escape_once(s)
  ActiveSupport::Multibyte::Unicode.tidy_bytes(s.to_s).gsub(HTML_ESCAPE_ONCE_REGEXP, HTML_ESCAPE).html_safe
end

might override +to_json+ to bypass Active Support's encoder).
JSON gem, do not provide this kind of protection by default; also some gems
is recommended that you always apply this helper (other libraries, such as the
is enabled, or if you are unsure where your JSON string originated from, it
Therefore, when you are unsure if +ActiveSupport.escape_html_entities_in_json+

applied even if +ActiveSupport.escape_html_entities_in_json+ is already true.
set to true. Because this transformation is idempotent, this helper can be
Active Support JSON encoder when +ActiveSupport.escape_html_entities_in_json+ is
The escaping performed in this method is identical to those performed in the

will happily eval() that string as JavaScript.
+current_user.to_json+ in the example above with user input instead, the browser
will open up serious XSS vulnerabilities. For example, if you replace the
WARNING: this helper only works with valid JSON. Using this on non-JSON values



automatically escaped for you:
like this, as any unsafe characters (including quotation marks) will be
If you need to output JSON elsewhere in your HTML, you can just do something

content returned by your JSON.
If that is the case, be sure to +html_escape+ or +sanitize+ any user-generated
whether or not it is being inserted via html(). Most jQuery plugins do this.
If your JSON is being used downstream for insertion into the DOM, be aware of

use inside HTML attributes.
automatically flag the result as HTML safe, since the raw value is unsafe to
don't get converted to " entities. +json_escape+ doesn't
It is necessary to +raw+ the result of +json_escape+, so that quotation marks


var currentUser = <%= raw json_escape(current_user.to_json) %>;
\"}"
json = JSON.generate({ name: ""})

JSON value, the output will have equivalent meaning when parsed:
context of a JSON string, so assuming the input is a valid and well-formed
These sequences have identical meaning as the original characters inside the
escaped as they are treated as newline characters in some JavaScript engines.
\u0026, \u003e, and \u003c. The Unicode sequences \u2028 and \u2029 are also
&, > and < characters are replaced with their equivalent unicode escaped form -
A utility method for escaping HTML entities in JSON strings. Specifically, the

def json_escape(s)
  result = s.to_s.dup
  result.gsub!(">", '\u003e')
  result.gsub!("<", '\u003c')
  result.gsub!("&", '\u0026')
  result.gsub!("\u2028", '\u2028')
  result.gsub!("\u2029", '\u2029')
  s.html_safe? ? result.html_safe : result
end

It follows the requirements of the specification: https://www.w3.org/TR/REC-xml/#NT-Name

# => "1___2___3"
xml_name_escape('1 < 2 & 3')

A utility method for escaping XML names of tags and names of attributes.

def xml_name_escape(name)
  name = name.to_s
  return "" if name.blank?
  return name if name.match?(SAFE_XML_TAG_NAME_REGEXP)
  starting_char = name[0]
  starting_char.gsub!(INVALID_TAG_NAME_START_REGEXP, TAG_NAME_REPLACEMENT_CHAR)
  return starting_char if name.size == 1
  following_chars = name[1..-1]
  following_chars.gsub!(INVALID_TAG_NAME_FOLLOWING_REGEXP, TAG_NAME_REPLACEMENT_CHAR)
  starting_char << following_chars
end