| Extracted from Pike v7.8 release 700 at 2013-06-08.

[Top]
Parser
Parser._parser
Parser._parser._C

# Method Parser._parser._C.tokenize()

Method tokenize

  `array(array(string)|string) tokenize(string code)`

Description

  Tokenize a string of C tokens.

Note

  Don't use this function directly. Use Parser.C.tokenize() instead.

Returns

  Returns an array with an array with C-level tokens, and the remainder (a partial token), if any.