tokenize

This is the quintessential function. Given a string containing C++ code and a filename, fills output with the tokens in the file. Warning - don't use temporaries for input and filename because the resulting tokens contain StringPiece objects pointing into them.

  1. CppLexer.Token[] tokenize(string input, string initialFilename)
    tokenize
    (
    string input
    ,
    string initialFilename = null
    )
  2. void tokenize(string input, string initialFilename, CppLexer.Token[] t)

Meta