classTokenGenerator
final class TokenGenerator
Derives grammar terminals without choosing spellings or pruning unsupported lexemes.
Properties§
public ?ScopedGeneration $lastScope = nullLexical scopes belonging only to the latest derivation.
Methods§
public function __construct(
private Grammar $grammar,
private Generator $faker,
private Closure(string): bool $nonOutput,
)Fixes grammar and production choices; marker metadata affects emptiness, never handler availability.
Parameters
$grammar | Grammar | |
$faker | Generator | |
$nonOutput | Closure(string): bool |
Calls 2
public function generate(
string $root,
GenerationPlan<bool> $plan,
Closure(int, non-empty-list<Production>): ?int|null $choose = null,
): TerminalSequenceProduces the selected terminal occurrences and their grammar provenance.
Parameters
$root | string | |
$plan | GenerationPlan<bool> | |
$choose | Closure(int, non-empty-list<Production>): ?int|null |
Returns
Throws
GenerationException When the grammar cannot complete the planLogicException When derivation failed to retain its traceTest cases 5
TokenGeneratorTest::testDeriveRetainsOriginalRuleIdentityForRepeatedChoicescallsTokenGeneratorTest::testGenerateCanChooseNullableChildrenWhenAnotherSiblingProvidesOutputcallsTokenGeneratorTest::testGenerateReservesVisibleOutputWithoutConsultingLexicalHandlerAvailabilitycallsTokenGeneratorTest::testMinimumExpansionsIncludesEveryPlannedListItemcallsTokenGeneratorTest::testPrepareKeepsPlansAndUnconstrainedCallsIndependentcalls
Called from 1
Calls 3
- method-call TokenGenerator::prepare() line 60
- method-call GenerationPlan::rules() line 61
- method-call TokenGenerator::derive() line 66
public function derive(
string $root,
GenerationPlan<bool> $plan,
Closure(int, non-empty-list<Production>): ?int|null $choose = null,
): TerminalSequenceRuns the shared derivation engine on the selected grammar.
Parameters
$root | string | |
$plan | GenerationPlan<bool> | |
$choose | Closure(int, non-empty-list<Production>): ?int|null |
Returns
Throws
GenerationException When the grammar cannot complete the planLogicException When derivation failed to retain its traceCalled from 1
Calls 3
- new Derivation line 78
- method-call Derivation::of() line 79
- new
LogicExceptionline 80
public function minimumExpansions(string $root, GenerationPlan<bool> $plan): intUses the very same specialized grammar when choosing a feasible byte-plan budget.
Parameters
$root | string | |
$plan | GenerationPlan<bool> |
Returns
intTest cases 2
Called from 1
Calls 7
- method-call TokenGenerator::prepare() line 89
- method-call GenerationPlan::rules() line 90
- method-call ConstrainedCompletion::minimum() line 92
- new ConstrainedCompletion line 92
- new NonTerminal line 93
- method-call GenerationPlan::requiresNonEmpty() line 96
- method-call GenerationPlan::expansionBudget() line 97
public function prepare(string $root, GenerationPlan<bool> $plan): voidKeeps one preparation, avoiding unbounded caches as fuzz inputs change their constraints.
Parameters
$root | string | |
$plan | GenerationPlan<bool> |
Called from 2
Calls 4
- method-call GenerationPlan::rules() line 107
- method-call GrammarCompiler::compile() line 110
- new GrammarCompiler line 110
- new
selfline 111
Private surface 9§
Implementation details, listed for orientation only.
private TerminationAnalyzer $terminationprivate CompletionCosts $completionprivate array<string, RulePlan> $plannedRules = []private ?string $plannedRoot = nullprivate ?PreparedGrammar $prepared = nullprivate ?self $planned = nullprivate Grammar $grammarprivate Generator $fakerprivate Closure(string): bool $nonOutputTest cases 23§
Test cases that cover or call this symbol, from the coverage report and from the analyzed test sources.
Dedicated tests 6
TokenGeneratorTest::testDeriveRetainsOriginalRuleIdentityForRepeatedChoicescallsTokenGeneratorTest::testGenerateCanChooseNullableChildrenWhenAnotherSiblingProvidesOutputcallsTokenGeneratorTest::testGenerateReservesVisibleOutputWithoutConsultingLexicalHandlerAvailabilitycallsTokenGeneratorTest::testMinimumExpansionsIncludesEveryPlannedListItemcallsTokenGeneratorTest::testMinimumExpansionsUsesTheCurrentRootPlanAndExplicitBudgetcallsTokenGeneratorTest::testPrepareKeepsPlansAndUnconstrainedCallsIndependentcalls
Other tests reaching this symbol 17
BytePlanCompilerTestcallsBytePlanEncoderTestcallsPlanBuilderTestcallsDerivationTraceTestcallsLexemeSequenceTestcallsCandidateResolverTestcallsSqlGeneratorTestcallsTerminalSequenceTestcallsGenerationContextTestcallsSqlGeneratorFactoryTestcallsMySqlProviderTestcallsGenerationContextTestcallsSqlGeneratorFactoryTestcallsPostgreSqlProviderTestcallsGenerationContextTestcallsSqlGeneratorFactoryTestcallsSqliteProviderTestcalls