Skip to content

Csf Auto Model

CSF auto-model component - includes 12 tools.

Cluster: Uncategorised | Type: component | MCP Tools: 14

Overview

Automatically derives a formal BoundedAgent model from the CsfAuditLog action history. Closes the gap between observed agent behaviour and the formal agent specification required by the Computational Safety Framework for probabilistic safety proofs.

When to use:

  • Constructing a formal agent model without hand-coding transition kernels
  • Bootstrapping CSF verification from production audit logs
  • Updating the agent model incrementally as new actions are observed

Calibration caveat

csf_auto_model bootstraps a safety model from recent audit history. It is not a calibrated safety certificate. The model uses a coarse Markov abstraction of observed audit states, so low-sample results should be treated as an operational confidence signal, not proof that a workflow is safe in every context. Check result.coverage, result.confidence, and result.warnings before relying on the generated model for production gating.

Classifier semantic weakness

The <risk_class> segment of each <risk_class>:<tier>:<verdict> state label comes from a coarse keyword classifier over the free-text action string, NOT a formally validated scheme (the reworded-action eval measures ~54% risk-class accuracy). The tier and verdict segments are taken verbatim from the audit row. Transition probabilities are row-order empirical estimates without confidence intervals, and classifier_confidence is an advisory signal, not calibrated statistical risk.

production_ready is a heuristic, not a certificate

production_ready is computed from threshold comparisons (30 observations / 80% coverage / 0.7 confidence). Every build_model output stamps production_ready_basis="threshold_heuristic" and statistical_certificate=False. It is gated behind a fail-closed reviewer-signature production gate and is never auto-promoted to the canonical verified reliability state. A formal promotion contract would additionally require transition confidence intervals, representativeness checks, and end-to-end sign-off (deferred).

build_model returns metadata, not a serialized BoundedAgent

The AIBlock/MCP build_model result returns model metadata (coverage, confidence, num_states, transitions, warnings, initial_state) only. The constructed BoundedAgent is internal to AgentModelResult; consumers who need the agent for downstream CSF verification call build_agent_from_log in-process. There is no public safe BoundedAgent serialization/export op yet — capabilities reports source_capabilities.bounded_agent_export = "capability_pending".

Operations (12): build_model, get_info, list_strategies, explain_safety_floor, plus read-only decision-point inspection ops classify_states, derive_transition_kernel, derive_resource_bounds, assess_readiness, estimate_confidence, inspect_transition_support, validate_candidate_model, and capabilities. Every op stamps the canonical envelope (completion_state, warning_card, evidence, request_id, run_id); the read-only inspection ops are deterministic projections of the same pipeline and never relax a safety floor.

Example:

from mvp.csf_auto_model import build_agent_from_log
from mvp.csf_audit import CsfAuditLog

log = CsfAuditLog(db_path="csf_audit.db")
result = build_agent_from_log(log)
# result.agent -> BoundedAgent ready for CSFBlock.verify()
# result.coverage/result.confidence tell you whether the audit sample is mature.

Works well with: csf, csf_audit, csf_gate

Public API

AgentModelResult

Field Type Default
agent BoundedAgent required
coverage str required
num_observations int required
num_states int required
num_transitions int 0
min_transition_observations int 0
confidence str 'none'
warnings tuple[str, ...] ()
agentic_evidence dict[str, Any] field(default_factory=dict)

CsfAutoModelInput(BaseModel)

Field Type Default
op str required
parameters dict[str, Any] Field(default_factory=dict)
run_mode str 'beta'
reviewer_signature str ''

CsfAutoModelOutput(BaseModel)

Field Type Default
op str ''
result dict[str, Any] Field(default_factory=dict)
message str ''
n_observations int 0
min_observations_required int 30
coverage_pct float 0.0
min_coverage_required_pct float 80.0
classifier_confidence float 0.0
min_confidence_required float 0.7
production_ready bool False
review_status str 'unreviewed'
agentic_evidence dict[str, Any] Field(default_factory=dict)
production_ready_basis str 'threshold_heuristic'
statistical_certificate bool False
completion_state str _QUALIFIED_DRAFT
warning_card dict[str, Any] Field(default_factory=dict)
evidence dict[str, Any] Field(default_factory=dict)
request_id str ''
run_id str ''

CsfAutoModelBlock(AIBlock)

AIBlock wrapper for CSF auto-model builder.

Constructor:

Parameter Type Default
planner Any \| None None

Methods:

infer(input: CsfAutoModelInput) -> Result[CsfAutoModelOutput]

CsfAutoModelMCPBlock(AIBlock[CsfAutoModelMCPInput, CsfAutoModelMCPOutput, dict])

Dispatcher for 12 operations (CSF auto-model).

Field Type Default
name str 'csf_auto_model_mcp'
resource_bounds ResourceBounds \| None None
usage ResourceUsage field(default_factory=ResourceUsage)

Methods:

infer(data: CsfAutoModelMCPInput) -> Result[CsfAutoModelMCPOutput]

Functions

build_agent_from_log_deterministic_fallback(audit_log: Any, resources: ResourceTriple | None = None, limit: int = _DEFAULT_HISTORY_LIMIT) -> AgentModelResult

Derive a BoundedAgent from the audit log's action history.

build_agent_from_log(audit_log: Any, resources: ResourceTriple | None = None, limit: int = _DEFAULT_HISTORY_LIMIT, planner: Any | None = None) -> AgentModelResult

Derive a BoundedAgent, optionally through the grounded agentic planner.

MCP Tools

Operation Source
ops csf_auto_model_mcp
help csf_auto_model_mcp
build_model csf_auto_model_mcp
get_info csf_auto_model_mcp
list_strategies csf_auto_model_mcp
explain_safety_floor csf_auto_model_mcp
classify_states csf_auto_model_mcp
derive_transition_kernel csf_auto_model_mcp
derive_resource_bounds csf_auto_model_mcp
assess_readiness csf_auto_model_mcp
estimate_confidence csf_auto_model_mcp
inspect_transition_support csf_auto_model_mcp
validate_candidate_model csf_auto_model_mcp
capabilities csf_auto_model_mcp