Golden Tests¶
Cluster: Uncategorised | Type: component | MCP Tools: None
Overview¶
Capability-gate regression suite comprising deterministic end-to-end scenarios that verify core G6 subsystem integration. Built-in scenarios cover grounded goal planning, safety-gated actions, CEGIS synthesis, self-model introspection, and domain agent workflows. End users can extend the suite with JSON/YAML declarative workflow scenarios or Python module manifests and generate starter templates from the CLI, so design partners can encode their own domain workflow checks without editing package internals. Each scenario produces structured GoldenResult reports with per-assertion pass/fail detail. Supports CI mode (JSONL output, non-zero exit on failure), interactive use, scenario listing, and template generation.
When to use:
- Running integration gate checks before deployment
- Verifying subsystem wiring after refactoring
- Extending with custom domain-specific golden scenarios
- CI regression gating with structured JSONL output
Works well with: goal_engine, align_csf, formal_methods (cegis), self_training, all job_* components
Public API¶
ScenarioSpec¶
| Field | Type | Default |
|---|---|---|
name | str | required |
module | str | '' |
description | str | '' |
source | str | 'builtin' |
kind | str | 'module' |
config | dict[str, Any] \| None | None |
AssertionResult¶
| Field | Type | Default |
|---|---|---|
name | str | required |
passed | bool | required |
expected | object | required |
actual | object | required |
GoldenResult¶
| Field | Type | Default |
|---|---|---|
scenario_name | str | required |
passed | bool | required |
assertions | list[AssertionResult] | field(default_factory=list) |
warnings | list[str] | field(default_factory=list) |
duration_ms | float | 0.0 |
error | str \| None | None |
Methods:
to_dict() -> dict¶
GoldenTestsInput(BaseModel)¶
| Field | Type | Default |
|---|---|---|
op | str | required |
parameters | dict[str, Any] | Field(default_factory=dict) |
GoldenTestsOutput(BaseModel)¶
| Field | Type | Default |
|---|---|---|
op | str | '' |
result | dict[str, Any] | Field(default_factory=dict) |
message | str | '' |
completion_state | Literal['verified', 'qualified-draft', 'blocked-escalated'] | 'qualified-draft' |
warning_card | dict[str, Any] | Field(default_factory=dict) |
evidence | dict[str, Any] | Field(default_factory=dict) |
request_id | str | '' |
task_id | str | '' |
run_id | str | '' |
degraded | bool | False |
degradation_reason | str | '' |
GoldenTestsBlock(AIBlock)¶
AIBlock wrapper for golden test output comparison and listing.
Methods: