(PHP 8)
This class provides an alternative to token_get_all(). While the function returns tokens either as a single-character string, or an array with a token ID, token text and line number, PhpToken::tokenize() normalizes all tokens into PhpToken objects, which makes code operating on tokens more memory efficient and readable.
One of the T_* constants, or an ASCII codepoint representing a single-char token.
The textual content of the token.
The starting line number (1-based) of the token.
The starting position (0-based) in the tokenized string.