Domain Quality¶

mvp.domain_quality — Expert Loop domain-quality / surrogate-review engine.

Cluster: Uncategorised | Type: component | MCP Tools: None

Overview¶

Public API¶

`DomainQualityDecisionError(ValueError)`¶

The LLM did not produce a usable, validated domain-quality critique.

`DomainQualityCritique`¶

Validated, tightening-only critique of a domain-QA verdict.

Field	Type	Default
`added_deficiencies`	`list[str]`	field(default_factory=list)
`proposed_truth_state`	`str \\| None`	None
`proposed_decision`	`str \\| None`	None
`requires_human_review`	`bool`	False
`rationale`	`str`	''
`confidence`	`float`	0.0
`degraded`	`bool`	False
`raw_response`	`str`	''

`LLMDomainQualityRuntime`¶

Provider-neutral domain-quality critique runtime over G6's LLM caller.

Constructor:

Parameter	Type	Default
`llm`	`LLMCaller \\| None`	None

Methods:

`critique(artifact: str, domain: str, baseline: DomainQAResult, grounded_evidence: dict | None = None) -> DomainQualityCritique`¶

`DomainQualityPlanner`¶

Runtime-first critique facade with the deterministic four-valued floor.

Constructor:

Parameter	Type	Default
`runtime`	`DomainQualityRuntime \\| None`	None

Methods:

`verdict(artifact: str, domain: str, baseline: DomainQAResult) -> DomainQAResult`¶

Produce the grounded verdict (floor baseline -> LLM critique -> floor -> guard).

`CitationGroundingResult(BaseModel)`¶

Result of citation grounding verification (EXPERT_LOOP.md §13).

Field	Type	Default
`claim_count`	`int`	0
`cited_claim_count`	`int`	0
`unsupported_claim_count`	`int`	0
`citation_coverage_ratio`	`float`	0.0
`unsupported_claims`	`list[str]`	Field(default_factory=list)
`overall_status`	`Literal['pass', 'warn', 'fail', 'unknown']`	'unknown'

`DomainQualityBlock(AIBlock[DomainQualityInput, DomainQualityOutput, None])`¶

Public facade for the domain-quality / surrogate-review engine.

Field	Type	Default
`name`	`str`	'domain_quality'

Methods:

`infer(data: DomainQualityInput) -> Result[DomainQualityOutput]`¶

`LeakageReport(BaseModel)`¶

Overlap measurement between a candidate answer and an exemplar (§21).

Field	Type	Default
`exact_overlap_score`	`float`	0.0
`ngram_overlap_score`	`float`	0.0
`uncommon_phrase_overlap_score`	`float`	0.0
`prohibited_term_hits`	`int`	0
`overall_leakage_risk`	`float`	0.0
`decision`	`Literal['leakage_pass', 'leakage_warning', 'leakage_fail', 'unusable_case_due_to_contamination']`	'leakage_pass'

`DomainQualityPatternReview`¶

Structured one-way tightening review for a domain-QA verdict.

Field	Type	Default
`tightened`	`bool`	required
`reason`	`str`	required
`truth_state`	`str`	required
`decision`	`str`	required
`added_deficiencies`	`tuple[str, ...]`	()
`requires_human_review`	`bool`	False
`warnings`	`tuple[str, ...]`	()
`reflection`	`str`	''

Methods:

`to_metadata() -> dict[str, Any]`¶

`DomainQualityPatternRuntime`¶

Stateless executable mechanisms for domain_quality's applied patterns.

Methods:

`tighten(truth_state: str, decision: str, critical_failures: tuple[str, ...] | list[str] = (), human_review_required: bool = False) -> DomainQualityPatternReview`¶

One-way strictness guard (hook-based-safety-guard-rails).

`GroundedRunResult(BaseModel)`¶

Outcome of a grounded agentic run.

Field	Type	Default
`goal`	`str`	required
`final_output`	`str`	''
`verdict`	`DomainQAResult \\| None`	None
`accepted`	`bool`	False
`escalated`	`bool`	False
`revisions`	`int`	0
`transcript`	`list[str]`	Field(default_factory=list)
`degraded`	`bool`	False

`GroundedAgenticRuntime`¶

Run a domain task through grounded QA with a critique/revise loop.

Constructor:

Parameter	Type	Default
`domain`	`str`	required
`task_type`	`str`	''
`qa_block`	`DomainQualityBlock \\| None`	None
`generate`	`GenerateFn \\| None`	None
`risk_tier`	`str`	'R2'
`require_citations`	`bool`	True
`expert_review_required`	`bool`	False
`max_revisions`	`int`	1

Methods:

`run(goal: str, initial_output: str | None = None) -> GroundedRunResult`¶

`QualityPack(BaseModel)`¶

Everything needed to QA a domain-specific component (EXPERT_LOOP.md §8).

Field	Type	Default
`domain`	`str`	required
`task_type`	`str`	required
`version`	`str`	'0.1.0'
`intended_output_type`	`str`	''
`rubric_dimensions`	`list[dict[str, Any]]`	Field(default_factory=list)
`checklist_items`	`list[dict[str, Any]]`	Field(default_factory=list)
`citation_policy`	`dict[str, Any]`	Field(default_factory=dict)
`standards_mappings`	`list[str]`	Field(default_factory=list)
`templates`	`list[dict[str, Any]]`	Field(default_factory=list)
`failure_modes`	`list[str]`	Field(default_factory=list)
`outcome_proxy_spec`	`dict[str, Any]`	Field(default_factory=dict)
`provenance`	`dict[str, Any]`	Field(default_factory=dict)
`source_list`	`list[str]`	Field(default_factory=list)
`source_hashes`	`tuple[str, ...]`	()
`config_hash`	`str`	''
`grounding_corpus_hash`	`str`	''
`created_at`	`str`	''
`generated_by`	`str`	'domain_quality'
`confidence_level`	`str`	'low'
`review_status`	`str`	'draft'
`expert_review_status`	`ExpertReviewStatus`	'domain_expert_unavailable'
`review_record`	`str`	''
`limitations`	`list[str]`	Field(default_factory=lambda: ['surrogate validation only; not domain-expert certified'])

Methods:

`fingerprint() -> str`¶

Deterministic content fingerprint for reproducibility/lineage (§18).

`DomainQAResult(BaseModel)`¶

Four-valued result of running domain QA on an output (EXPERT_LOOP.md §14).

Field	Type	Default
`truth_state`	`Literal['T', 'F', 'B', 'N']`	required
`completion_state`	`str`	required
`decision`	`QADecision`	required
`grade`	`str`	''
`passed`	`bool \\| None`	None
`overall_confidence`	`float`	0.0
`evidence_level`	`str`	'E0'
`risk_tier`	`str`	'R2'
`modal_frame`	`str`	'deontic'
`critical_failures`	`list[str]`	Field(default_factory=list)
`deficiency_count`	`int`	0
`evidence_for`	`list[str]`	Field(default_factory=list)
`evidence_against`	`list[str]`	Field(default_factory=list)
`recommended_remediations`	`list[str]`	Field(default_factory=list)
`expert_review_status`	`ExpertReviewStatus`	'domain_expert_unavailable'
`human_review_required`	`bool`	False
`citation_status`	`str`	'not_checked'
`citation_coverage_ratio`	`float \\| None`	None
`residual_risk`	`str`	''
`degraded`	`bool`	False

`DomainQualityInput(BaseModel)`¶

Input to DomainQualityBlock (EXPERT_LOOP.md §6 public interface).

Field	Type	Default
`operation`	`DomainQualityOp`	'build_pack'
`domain`	`str`	''
`task_type`	`str`	''
`grounding_sources`	`list[str]`	Field(default_factory=list)
`options`	`dict[str, Any]`	Field(default_factory=dict)
`job_output`	`dict[str, Any]`	Field(default_factory=dict)
`jurisdiction`	`str`	''
`risk_tier`	`str`	'R2'
`expert_review_required`	`bool`	False
`require_citations`	`bool`	True
`run_mode`	`str`	'beta'
`reviewer_signature`	`str`	''

`DomainQualityOutput(BaseModel)`¶

Output from DomainQualityBlock.

Field	Type	Default
`operation`	`str`	''
`ok`	`bool`	True
`error`	`str`	''
`pack`	`QualityPack \\| None`	None
`qa_result`	`DomainQAResult \\| None`	None
`degraded`	`bool`	False
`degradation_reason`	`str`	''
`agentic_evidence`	`dict \\| None`	None

`DomainQualitySkill`¶

Field	Type	Default
`name`	`str`	required
`pattern_slug`	`str`	required
`description`	`str`	required
`executable`	`bool`	required
`mechanism`	`str`	required
`capabilities`	`tuple[str, ...]`	required
`triggers`	`tuple[str, ...]`	required
`risk_notes`	`tuple[str, ...]`	required

Methods:

`compact() -> dict[str, Any]`¶

`DomainQualitySkillCatalog`¶

Maps each applied pattern slug to a domain-quality-gating skill record.

Methods:

`list_skills() -> list[DomainQualitySkill]`¶

`executable_skills() -> list[DomainQualitySkill]`¶

`get(slug: str) -> DomainQualitySkill | None`¶

Functions¶

`summarize_domain_quality_agentic_evidence(decisions: list[dict[str, Any]]) -> dict[str, Any]`¶

Summarise runtime-vs-fallback domain-quality decisions with path redaction.

`agentic_planner_enabled(default_enabled: bool) -> bool`¶

Decide whether the agentic domain_quality critique should be used.

`verify_citations(text: str, required_citation: bool = True, min_claim_words: int = 3) -> CitationGroundingResult`¶

Segment claims, detect citation markers, and grade coverage.

`detect_leakage(candidate: str, exemplar: str, prohibited_terms: tuple[str, ...] | list[str] = (), ngram_n: int = 3) -> LeakageReport`¶

Score how much candidate leaks from exemplar.

`applied_agentic_patterns() -> list[dict[str, Any]]`¶

Return compact metadata for domain_quality-applied vendored patterns.

`apply_qa_floor(candidate: DomainQAResult, baseline: DomainQAResult) -> DomainQAResult`¶

Clamp candidate so it is never less strict than baseline per axis.

`pack_filename(pack: QualityPack) -> str`¶

Deterministic filename: domain + version + content fingerprint.

`save_pack(pack: QualityPack, base_dir: str | Path) -> Path`¶

Write a pack to base_dir under its deterministic filename; returns the path.

`load_pack(path: str | Path) -> QualityPack`¶

Load a pack from disk. Raises FileNotFoundError if missing.

`qa_verdict_from_standards(std_out: Any, risk_tier: str = 'R2', expert_review_required: bool = False, expert_review_status: ExpertReviewStatus = 'domain_expert_unavailable', citation: Any | None = None) -> DomainQAResult`¶

Map a StandardsVerificationOutput to a four-valued DomainQAResult.

Domain Quality¶

Overview¶

Public API¶

DomainQualityDecisionError(ValueError)¶

DomainQualityCritique¶

LLMDomainQualityRuntime¶

critique(artifact: str, domain: str, baseline: DomainQAResult, grounded_evidence: dict | None = None) -> DomainQualityCritique¶

DomainQualityPlanner¶

verdict(artifact: str, domain: str, baseline: DomainQAResult) -> DomainQAResult¶

CitationGroundingResult(BaseModel)¶

DomainQualityBlock(AIBlock[DomainQualityInput, DomainQualityOutput, None])¶

infer(data: DomainQualityInput) -> Result[DomainQualityOutput]¶

LeakageReport(BaseModel)¶

DomainQualityPatternReview¶

to_metadata() -> dict[str, Any]¶

DomainQualityPatternRuntime¶

tighten(truth_state: str, decision: str, critical_failures: tuple[str, ...] | list[str] = (), human_review_required: bool = False) -> DomainQualityPatternReview¶

GroundedRunResult(BaseModel)¶

GroundedAgenticRuntime¶

run(goal: str, initial_output: str | None = None) -> GroundedRunResult¶

QualityPack(BaseModel)¶

fingerprint() -> str¶

DomainQAResult(BaseModel)¶

DomainQualityInput(BaseModel)¶

DomainQualityOutput(BaseModel)¶

DomainQualitySkill¶

compact() -> dict[str, Any]¶

DomainQualitySkillCatalog¶

list_skills() -> list[DomainQualitySkill]¶

executable_skills() -> list[DomainQualitySkill]¶

get(slug: str) -> DomainQualitySkill | None¶

Functions¶

summarize_domain_quality_agentic_evidence(decisions: list[dict[str, Any]]) -> dict[str, Any]¶

agentic_planner_enabled(default_enabled: bool) -> bool¶

verify_citations(text: str, required_citation: bool = True, min_claim_words: int = 3) -> CitationGroundingResult¶

detect_leakage(candidate: str, exemplar: str, prohibited_terms: tuple[str, ...] | list[str] = (), ngram_n: int = 3) -> LeakageReport¶

applied_agentic_patterns() -> list[dict[str, Any]]¶

apply_qa_floor(candidate: DomainQAResult, baseline: DomainQAResult) -> DomainQAResult¶

pack_filename(pack: QualityPack) -> str¶

save_pack(pack: QualityPack, base_dir: str | Path) -> Path¶

load_pack(path: str | Path) -> QualityPack¶

qa_verdict_from_standards(std_out: Any, risk_tier: str = 'R2', expert_review_required: bool = False, expert_review_status: ExpertReviewStatus = 'domain_expert_unavailable', citation: Any | None = None) -> DomainQAResult¶

get_skill_catalog() -> DomainQualitySkillCatalog¶

`DomainQualityDecisionError(ValueError)`¶

`DomainQualityCritique`¶

`LLMDomainQualityRuntime`¶

`critique(artifact: str, domain: str, baseline: DomainQAResult, grounded_evidence: dict | None = None) -> DomainQualityCritique`¶

`DomainQualityPlanner`¶

`verdict(artifact: str, domain: str, baseline: DomainQAResult) -> DomainQAResult`¶

`CitationGroundingResult(BaseModel)`¶

`DomainQualityBlock(AIBlock[DomainQualityInput, DomainQualityOutput, None])`¶

`infer(data: DomainQualityInput) -> Result[DomainQualityOutput]`¶

`LeakageReport(BaseModel)`¶

`DomainQualityPatternReview`¶

`to_metadata() -> dict[str, Any]`¶

`DomainQualityPatternRuntime`¶

`tighten(truth_state: str, decision: str, critical_failures: tuple[str, ...] | list[str] = (), human_review_required: bool = False) -> DomainQualityPatternReview`¶

`GroundedRunResult(BaseModel)`¶

`GroundedAgenticRuntime`¶

`run(goal: str, initial_output: str | None = None) -> GroundedRunResult`¶

`QualityPack(BaseModel)`¶

`fingerprint() -> str`¶

`DomainQAResult(BaseModel)`¶

`DomainQualityInput(BaseModel)`¶

`DomainQualityOutput(BaseModel)`¶

`DomainQualitySkill`¶

`compact() -> dict[str, Any]`¶

`DomainQualitySkillCatalog`¶

`list_skills() -> list[DomainQualitySkill]`¶

`executable_skills() -> list[DomainQualitySkill]`¶

`get(slug: str) -> DomainQualitySkill | None`¶

`summarize_domain_quality_agentic_evidence(decisions: list[dict[str, Any]]) -> dict[str, Any]`¶

`agentic_planner_enabled(default_enabled: bool) -> bool`¶

`verify_citations(text: str, required_citation: bool = True, min_claim_words: int = 3) -> CitationGroundingResult`¶

`detect_leakage(candidate: str, exemplar: str, prohibited_terms: tuple[str, ...] | list[str] = (), ngram_n: int = 3) -> LeakageReport`¶

`applied_agentic_patterns() -> list[dict[str, Any]]`¶

`apply_qa_floor(candidate: DomainQAResult, baseline: DomainQAResult) -> DomainQAResult`¶

`pack_filename(pack: QualityPack) -> str`¶

`save_pack(pack: QualityPack, base_dir: str | Path) -> Path`¶

`load_pack(path: str | Path) -> QualityPack`¶

`qa_verdict_from_standards(std_out: Any, risk_tier: str = 'R2', expert_review_required: bool = False, expert_review_status: ExpertReviewStatus = 'domain_expert_unavailable', citation: Any | None = None) -> DomainQAResult`¶

`get_skill_catalog() -> DomainQualitySkillCatalog`¶