classScanner

packages/bison-parser/src/Scanner/Scanner.php:20

final class Scanner

Restricted visibility: declared "@visibility root". Code outside that scope must not name this declaration.

Splits a grammar file into the tokens Bison's own scanner produces.

This follows scan-gram.l: an identifier is looked past to see whether a colon or a bracketed name follows, a %name-prefix may be followed by =, a stray comma is whitespace, and everything after the second %% is one epilogue token.

Constants§

public const IDENTIFIER = '[.A-Za-z_][-.A-Za-z0-9_]*'

source §

Pattern of an identifier: a letter, underscore or dot, then also digits and hyphens.

Methods§

public function __construct(
    private CodeReader $code = new \BisonParser\Scanner\CodeReader(),
    private Escapes $escapes = new \BisonParser\Scanner\Escapes(),
)

source §

Parameters

`$code`	`CodeReader`	Reads host code
`$escapes`	`Escapes`	Decodes literals

Calls 2

new CodeReader line 32
new Escapes line 33

public function scan(string $source): list<Token>

source §

Tokenizes a grammar file.

Parameters

$source string Contents of the file

Returns

list<Token> The tokens, ending with the end-of-file token

Throws

SyntaxException When the file holds something the grammar language does not allow

Test cases 57

Called from 1

method-call BisonParser\Parser::parse() packages/bison-parser/src/Parser.php:52

Calls 12

new Cursor line 48
method-call Scanner::skipTrivia() line 52
method-call Cursor::eof() line 53
new Token line 54
class-const TokenKind::End() line 54
method-call Cursor::location() line 54
method-call Scanner::next() line 58
function-call count line 61
class-const TokenKind::Section() line 62
class-const TokenKind::Epilogue() line 64
method-call Cursor::take() line 64
function-call strlen line 64

public function skipTrivia(Cursor $cursor): void

source §

Skips whitespace, comments, stray commas and #line directives.

Parameters

$cursor Cursor Cursor to advance

Throws

SyntaxException When a comment never closes

Test cases 2

Called from 5

Calls 6

method-call Cursor::eof() line 81
method-call Cursor::match() line 82
method-call Cursor::startsWith() line 85
method-call Cursor::location() line 86
method-call Cursor::takeUntil() line 87
static-call SyntaxException::unterminated() line 88

public function next(Cursor $cursor): list<Token>

source §

Reads the token, or the tokens, that begin at the cursor.

Parameters

$cursor Cursor Cursor positioned on a token

Returns

list<Token> One token, or an identifier and its bracketed name

Throws

SyntaxException When no token begins here or it never closes

Test cases 2

ScannerTest::testNext calls
ScannerTest::testNextRejectsAnInvalidCharacter calls

Called from 1

method-call BisonParser\Scanner\Scanner::scan() packages/bison-parser/src/Scanner/Scanner.php:58

Calls 16

method-call Cursor::location() line 108
method-call Cursor::peek() line 109
method-call Cursor::match() line 111
new Token line 113
class-const TokenKind::Line() line 113
method-call Scanner::percent() line 117
class-const TokenKind::Code() line 120
method-call CodeReader::braced() line 120
function-call preg_match line 122
method-call Cursor::startsWith() line 122
method-call Scanner::identifier() line 123
function-call ctype_digit line 125
method-call Scanner::integer() line 126
method-call Scanner::literal() line 128
method-call Scanner::punctuation() line 128
static-call SyntaxException::invalid() line 130

public function percent(Cursor $cursor, Location $location): Token

source §

Reads what begins with a percent sign: a prologue, a section marker, a predicate or a directive.

Parameters

`$cursor`	`Cursor`	Cursor positioned on the percent sign
`$location`	`Location`	Where it is

Returns

Token The token

Throws

SyntaxException When the directive is unknown or the code never closes

Test cases 3

Called from 1

method-call BisonParser\Scanner\Scanner::next() packages/bison-parser/src/Scanner/Scanner.php:117

Calls 16

method-call Cursor::startsWith() line 148
new Token line 149
class-const TokenKind::Prologue() line 149
method-call CodeReader::prologue() line 149
method-call Cursor::take() line 152
class-const TokenKind::Section() line 154
method-call Cursor::match() line 156
class-const TokenKind::Predicate() line 157
method-call CodeReader::braced() line 157
class-const Scanner::IDENTIFIER() line 159
static-call Directives::canonical() line 160
function-call substr line 160
static-call SyntaxException::invalid() line 162
function-call in_array line 164
class-const Directives::EQUAL_OPTIONAL() line 164
class-const TokenKind::Directive() line 168

public function identifier(Cursor $cursor, Location $location): list<Token>

source §

Reads an identifier and looks past it for a bracketed name and a colon.

Parameters

`$cursor`	`Cursor`	Cursor positioned on the identifier
`$location`	`Location`	Where it is

Returns

list<Token> The identifier, followed by its bracketed name when one follows it

Throws

SyntaxException When a bracketed name is malformed

Test cases 1

ScannerTest::testIdentifier calls

Called from 1

method-call BisonParser\Scanner\Scanner::next() packages/bison-parser/src/Scanner/Scanner.php:123

Calls 8

method-call Cursor::match() line 183
class-const Scanner::IDENTIFIER() line 183
method-call Scanner::skipTrivia() line 184
method-call Cursor::peek() line 186
method-call Scanner::bracketed() line 187
class-const TokenKind::IdentifierColon() line 190
class-const TokenKind::Identifier() line 190
new Token line 191

public function bracketed(Cursor $cursor): Token

source §

Reads a bracketed name such as [left].

Parameters

$cursor Cursor Cursor positioned on the opening bracket

Returns

Token The bracketed identifier

Throws

SyntaxException When the brackets hold anything but one identifier

Test cases 4

Called from 2

Calls 11

method-call Cursor::location() line 210
method-call Cursor::take() line 211
method-call Scanner::skipTrivia() line 212
method-call Cursor::match() line 213
class-const Scanner::IDENTIFIER() line 213
static-call SyntaxException::invalid() line 215
method-call Cursor::peek() line 218
static-call SyntaxException::unexpected() line 219
method-call Cursor::eof() line 219
new Token line 223
class-const TokenKind::BracketedIdentifier() line 223

public function integer(Cursor $cursor, Location $location): Token

source §

Reads a decimal or hexadecimal integer.

Parameters

`$cursor`	`Cursor`	Cursor positioned on the first digit
`$location`	`Location`	Where it is

Returns

Token The integer, its text in decimal

Throws

SyntaxException When letters follow the digits

Test cases 6

Called from 1

method-call BisonParser\Scanner\Scanner::next() packages/bison-parser/src/Scanner/Scanner.php:126

Calls 11

method-call Cursor::match() line 238
method-call Cursor::lookingAt() line 239
static-call SyntaxException::invalid() line 240
function-call str_starts_with line 242
function-call strtolower line 242
function-call ltrim line 243
function-call substr line 243
function-call strlen line 244
function-call hexdec line 244
new Token line 249
class-const TokenKind::Integer() line 249

public function literal(Cursor $cursor, Location $location): Token|null

source §

Reads a character literal, a string, a translatable string, or a tag.

Parameters

`$cursor`	`Cursor`	Cursor positioned on the literal
`$location`	`Location`	Where it is

Returns

Token|null The token, or null when no literal begins here

Throws

SyntaxException When the literal never closes or a character literal is not one byte

Test cases 5

Called from 1

method-call BisonParser\Scanner\Scanner::next() packages/bison-parser/src/Scanner/Scanner.php:128

Calls 13

method-call Cursor::peek() line 264
method-call Cursor::match() line 266
static-call SyntaxException::unterminated() line 268
method-call Escapes::decode() line 270
function-call substr line 270
static-call SyntaxException::invalid() line 272
function-call strlen line 274
new Token line 278
class-const TokenKind::CharLiteral() line 278
method-call Cursor::startsWith() line 280
class-const TokenKind::TranslatableString() line 288
class-const TokenKind::String() line 288
method-call Scanner::tag() line 291

public function tag(Cursor $cursor, Location $location): Token

source §

Reads a tag, which may nest angle brackets as a C++ template does.

Parameters

`$cursor`	`Cursor`	Cursor positioned on the opening angle bracket
`$location`	`Location`	Where it is

Returns

Token The tag

Throws

SyntaxException When the tag never closes

Test cases 2

ScannerTest::testTag calls
ScannerTest::testTagRejectsAnUnterminatedTag calls

Called from 1

method-call BisonParser\Scanner\Scanner::literal() packages/bison-parser/src/Scanner/Scanner.php:291

Calls 10

method-call Cursor::startsWith() line 309
method-call Cursor::take() line 310
new Token line 312
class-const TokenKind::TagAny() line 312
class-const TokenKind::TagNone() line 317
method-call Cursor::eof() line 322
method-call Cursor::match() line 323
class-const TokenKind::Tag() line 326
function-call strlen line 330
static-call SyntaxException::unterminated() line 335

public function punctuation(Cursor $cursor, Location $location): Token|null

source §

Reads one punctuation byte.

Parameters

`$cursor`	`Cursor`	Cursor positioned on the byte
`$location`	`Location`	Where it is

Returns

Token|null The token, or null when the byte is not punctuation the language has

Throws

SyntaxException When a bracketed name is malformed

Test cases 1

ScannerTest::testPunctuation calls

Called from 1

method-call BisonParser\Scanner\Scanner::next() packages/bison-parser/src/Scanner/Scanner.php:128

Calls 8

method-call Cursor::peek() line 350
method-call Scanner::bracketed() line 352
class-const TokenKind::Colon() line 355
class-const TokenKind::Equal() line 356
class-const TokenKind::Pipe() line 357
class-const TokenKind::Semicolon() line 358
method-call Cursor::take() line 364
new Token line 366

Private surface 2§

Implementation details, listed for orientation only.

private CodeReader $code = new \BisonParser\Scanner\CodeReader()

private Escapes $escapes = new \BisonParser\Scanner\Escapes()

Test cases 86§

Test cases that cover or call this symbol, from the coverage report and from the analyzed test sources.

Dedicated tests 29

Other tests reaching this symbol 57

Relations§

Instantiated in 1

new BisonParser\Parser::__construct() packages/bison-parser/src/Parser.php:36

Method calls 1

method-call BisonParser\Parser::parse() packages/bison-parser/src/Parser.php:52

Type declarations 1

type BisonParser\Parser packages/bison-parser/src/Parser.php:36