classMySqlTokenizer

MySqlGenerationpackages/sql-faker/src/MySql/Generation/Tokenization/MySqlTokenizer.php:18
final class MySqlTokenizer
Restricted visibility: declared "@visibility root". Code outside that scope must not name this declaration.

Reads MySQL text back into the parser tokens the server would produce.

This diagnostic tokenizer supports the declared generated spellings and selected scanner contracts. It is not a substitute for an independent native parser verdict.

Methods§

public function __construct(
    private array<string, string> $symbolTokens,
    private array<string, string> $functionTokens,
    private bool $dollarQuotedStrings,
)
100%source§

Parameters

$symbolTokensarray<string, string>Terminal name by upper-cased keyword or operator
$functionTokensarray<string, string>Terminal name by upper-cased function name
$dollarQuotedStringsboolWhether the server build accepts `$$ ... $$` strings
Test cases 59
public function tokenize(string $sql): list<string>
88%source§

Reads SQL text into the tokens MySQL's own lexer would produce.

Parameters

$sqlstringText to read

Returns

list<string> Parser token names, in order

Throws

LexicalException When the text holds something the lexer cannot read
Test cases 35
Called from 1
Calls 6
public function tokenAt(string $sql, int &$offset, list<string> $emitted): string|null
100%source§

Reads the one token that starts at the offset, advancing past it.

Parameters

$sqlstringText being read
$offsetintWhere to read from; moved past what was consumed
$emittedlist<string>Tokens read so far, for the rules that depend on them

Returns

string|null The token name, or null when nothing here is a token

Throws

LexicalException When a quoted or dollar-quoted run never closes
Test cases 34
Called from 1
Calls 5
public function quotedTokenAt(string $sql, int &$offset): string|null
100%source§

Reads a backtick, single-quoted or dollar-quoted run.

Parameters

$sqlstringText being read
$offsetintWhere to read from; moved past what was consumed

Returns

string|null The token name, or null when no quote opens here

Throws

LexicalException When the run never closes
Test cases 35
Called from 1
Calls 4
public function markerTokenAt(string $sql, int &$offset, list<string> $emitted): string|null
100%source§

Reads a parameter marker, a user-variable sigil, or the hostname after one.

A hostname is only a hostname because an @ came before it; the same characters anywhere else are an ordinary identifier, which is why the tokens read so far decide this.

Parameters

$sqlstringText being read
$offsetintWhere to read from; moved past what was consumed
$emittedlist<string>Tokens read so far

Returns

string|null The token name, or null when none of these start here
Test cases 32
Called from 1
Calls 3
public function numericTokenAt(string $sql, int &$offset): string|null
89%source§

Reads a numeric literal in any of the spellings MySQL accepts.

Parameters

$sqlstringText being read
$offsetintWhere to read from; moved past what was consumed

Returns

string|null The token name, or null when no number starts here

Throws

LexicalException When a prefixed quoted literal never closes
Test cases 29
Called from 1
Calls 8
public function wordTokenAt(string $sql, int &$offset): string|null
100%source§

Reads a keyword, function name, charset introducer or identifier.

Parameters

$sqlstringText being read
$offsetintWhere to read from; moved past what was consumed

Returns

string|null The token name, or null when no word starts here
Test cases 22
Called from 1
Calls 5
public function operatorTokenAt(string $sql, int &$offset): string|null
100%source§

Reads an operator or a single punctuation character.

Parameters

$sqlstringText being read
$offsetintWhere to read from; moved past what was consumed

Returns

string|null The token name, or null when nothing operator-like starts here
Test cases 6
Called from 1
Calls 2
public function skipTrivia(string $sql, int &$offset): bool
100%source§

Consumes whitespace or one comment.

Parameters

$sqlstringText being read
$offsetintWhere to read from; moved past what was consumed

Returns

bool True when something was skipped

Throws

LexicalException When a block comment never closes
Test cases 35
Called from 1
Calls 5
public function skipQuoted(string $sql, int &$offset, string $quote): void
100%source§

Consumes a quoted run, doubling being the way a quote escapes itself.

Parameters

$sqlstringText being read
$offsetintOffset of the opening quote; moved past the closing one
$quotestringThe quote character

Throws

LexicalException When the run never closes
Test cases 7
Called from 3
Calls 2
public function merged(list<string> $tokens): list<string>
100%source§

Joins the token pairs MySQL's parser sees as one.

Parameters

$tokenslist<string>Tokens as read

Returns

list<string> Tokens with `WITH ROLLUP` collapsed
Test cases 33
Called from 1

Private surface 5§

Implementation details, listed for orientation only.

private const OPERATORS = ['<=>', '->>', '&&', '<=', '<>', '!=', '>=', '<<', '>>', ':=', '->', '||']
private const PUNCTUATION = '!%&()*+,-./:;@^{}|~=<>'
private array<string, string> $symbolTokens
private array<string, string> $functionTokens
private bool $dollarQuotedStrings

Test cases 68§

Test cases that cover or call this symbol, from the coverage report and from the analyzed test sources.

Dedicated tests 61
Other tests reaching this symbol 7

Relations§

Instantiated in 1
Method calls 1
Type declarations 1