Tutorial: Safety-Guarded Goals¶

This tutorial shows how to run agent goals with CSF (Computational Safety Framework) safety guards, so G6 checks modeled risk before execution and pauses for a human when it matters. You drive everything through G6's surfaces — the GUI, Claude Code (MCP), or REST — by attaching declarative controls to your goal. No coding required.

What you will build¶

A safety-guarded goal that:

1. Declares resource bounds (token / time / cost limits) for the run
2. Adds a guardrail that halts if a modeled CSF hazard exceeds its budget
3. Adds a checkpoint that logs a quality metric without halting
4. Adds a breakpoint that pauses for human approval before a high-stakes step
5. Lets you review the safety decision that G6 records for the run

How CSF decides¶

G6 ships with default hazard priors and a safety threshold (epsilon). You can see them any time by asking G6 for its safety status (the system_status tool reports the active CSF signature):

The epsilon budget (default 0.20) is the maximum acceptable combined hazard. A plan is approved by the gate when its modeled hazard stays under epsilon. Two strategies are available:

So a step that calls an LLM (0.05) and an external API (0.08) has a combined bound of 0.13 < 0.20 → approved. Add code execution (0.10) and file writes (0.01) and the bound becomes 0.24 > 0.20 → blocked.

Step 1: Declare safety controls on your goal¶

G6 reads three kinds of declarative control from your goal's JSON. These are evaluated by the engine at runtime — you don't write Python:

A goal with all three:

{
  "goal": "Train and deploy an ML classifier",
  "resource_bounds": {
    "max_execution_seconds": 600,
    "max_tokens_per_hour": 200000
  },
  "guardrails": [
    {
      "name": "csf_code_execute",
      "predicate": "csf_hazard",
      "params": {"action": "code_execute", "max_prob": 0.10},
      "message": "Halt if code-execution hazard exceeds 10%"
    }
  ],
  "checkpoints": [
    {
      "name": "f1_quality_gate",
      "predicate": "metric_above",
      "params": {"metric": "f1", "threshold": 0.90},
      "description": "Log a warning if F1 drops below 0.90"
    }
  ],
  "breakpoints": [
    {
      "name": "pre_deploy_review",
      "description": "Pause for human review before deploying to production",
      "active": true
    }
  ],
  "subtasks": []
}

Predicate registry¶

Each guardrail and checkpoint references a predicate key:

Step 2: Run the goal with safety verification¶

Submit the goal through whichever surface you use — the controls travel with it and the engine enforces them.

Use the G6 run_safety_pipeline tool to run this goal with its guardrails,
checkpoints, and breakpoints: <paste the goal JSON above>

Use G6 to check the CSF safety of a step that does an LLM call plus a code execution

curl -X POST http://localhost:8000/invoke/csf \
  -H "Content-Type: application/json" \
  -d '{"operation": "verify", "params": {"hazards": {"llm_call": 0.05, "external_api": 0.08}, "epsilon": 0.20, "strategy": "union_bound"}}'

Step 3: Review the safety decision¶

Every safety check returns a structured decision you can inspect (in the tool result, the GUI run summary, or the REST response):

{
  "is_safe": true,
  "bound_value": 0.13,
  "strategy": "union_bound",
  "details": {
    "hazards": {"llm_call": 0.05, "external_api": 0.08},
    "epsilon": 0.20,
    "combined_bound": 0.13,
    "margin": 0.07
  }
}

If the combined bound exceeds epsilon, is_safe is false and the guarded step does not run — the decision and its reason are recorded in the run's audit trail.

When to use which control¶

For extension developers¶

If you are building your own component with the extension SDK (the g6ext.* namespace; see Custom Component), the same CSF primitives are available in code so your component can verify its own actions. The csf_guarded decorator wraps a function so it only runs when the modeled hazard stays under epsilon:

# Inside a g6ext.* extension component — import path per the extension SDK reference
@csf_guarded(hazards=["llm_call", "external_api"])
def research_with_web_search(query: str) -> str:
    """Combined hazard 0.13 < 0.20 (epsilon) -> runs.
    If the combined hazard exceeded epsilon, the call is blocked instead."""
    ...

See the Custom Component tutorial for how to build and register an extension.

Next steps¶

• Research Pipeline — build a full research pipeline
• Custom Component — create your own CSF-aware extension component
• Security Architecture — deep dive into the CSF framework