Skip to content

Self-Improving Agent

Production caveat

Treat this use case as a guided reliability workflow, not a promise of autonomous self-training. For launch or customer-facing workflows, EvoSkill proposals should be reviewed, applied inside a bounded train path, and then replayed through the learning_layer with mutation disabled on holdout or frozen data. The claim to show users is not "the agent improved itself"; it is "this candidate passed the recorded validation checks."

Learning-layer evaluation depends on the scorer. Exact-match scoring is only valid for short deterministic outputs; open-ended work needs a concrete rubric/criteria field and incurs an extra LLM-judge call per row. Always validate on held-out examples before exporting a harness.

Before a customer pilot or production workflow, run the gated live-backend smoke test on the deployment machine using the same G6_WORKSPACE, model/provider, credentials, and environment:

$env:G6_WORKSPACE="C:\path\to\pilot-workspace"
$env:G6_LEARNING_LAYER_LIVE_SMOKE="1"
python -m pytest -q tests/mvp/learning_layer/test_live_llm_smoke.py

Build a bounded self-improvement loop that uses counterexample-guided inductive synthesis (CEGIS) to synthesise new skills, available formal checks to verify explicitly specified properties, and drift detection (DDM) to trigger re-optimisation — all within strict resource bounds.

Illustrative end-to-end example

A worked illustration of how these components compose — every component and operation named below is real, but it is not a copy-paste script. The steps run offline against built-in defaults; the formal_methods proof needs a solver backend (the built-in propositional solver, or Z3) available in your deployment.

GoalInput

{
  "goal": "Run a bounded EvoSkill self-improvement loop with CEGIS synthesis and drift detection",
  "context": "The agent handles structured data extraction from financial documents. Current accuracy is 0.74 — target is 0.92. Use CEGIS to synthesise extraction functions from sketches with holes, formal methods to verify correctness invariants, and adapt_healing for DDM-based drift detection to trigger re-evolution when accuracy degrades.",
  "constraints": [
    "Accuracy must reach 0.92 within 10 evolution iterations",
    "CEGIS sketch convergence must occur within 8 iterations per skill",
    "CSF epsilon must not exceed 0.20 for any self-modification action",
    "Drift detection must trigger at DDM warning threshold",
    "All synthesised functions must pass configured tests and available formal checks before deployment"
  ],
  "resource_bounds": {
    "max_execution_seconds": 900,
    "max_tokens_per_hour": 300000,
    "max_disk_bytes": 134217728
  },
  "guardrails": [
    {
      "name": "csf_code_execute",
      "predicate": "csf_hazard",
      "params": {"action": "code_execute", "max_prob": 0.10},
      "message": "Halt if code-execution hazard exceeds 10% during self-modification"
    }
  ],
  "checkpoints": [
    {
      "name": "accuracy_gate",
      "predicate": "metric_above",
      "params": {"metric": "accuracy", "threshold": 0.92},
      "description": "Log warning if accuracy drops below 0.92 during evolution"
    }
  ],
  "breakpoints": [
    {
      "name": "pre_deploy_synthesis",
      "description": "Pause for human review before deploying synthesised functions to production",
      "active": true
    }
  ],
  "subtasks": [
    {
      "goal": "Baseline the current extraction agent on validation data",
      "context": "Use align_evals to compute accuracy, precision, recall, and F1 on the validation set. This establishes the starting fitness for the evolution frontier.",
      "constraints": ["Compute all 4 core metrics", "Record baseline for drift comparison"]
    },
    {
      "goal": "Synthesise improved extraction functions via CEGIS",
      "context": "Use cegis with sketch-based synthesis. Sketch: extraction function with ?? holes for regex patterns and field mappings. Oracle: ground truth extraction results. The CEGIS loop fills holes via counterexample-guided refinement with progressive LLM agency reduction (CODE_AGENT -> PROCESSOR).",
      "constraints": [
        "Max CEGIS iterations: 8",
        "Agency decay: start at CODE_AGENT (level 5), reduce each iteration",
        "Sketch must include holes for regex patterns and field mappings"
      ],
      "resource_bounds": {
        "max_execution_seconds": 300
      }
    },
    {
      "goal": "Formally verify correctness of synthesised functions",
      "context": "Use formal_methods (propositional DPLL) to verify: (input_valid AND schema_matched AND all_fields_extracted) -> extraction_correct. Use align_csf to bound hazard probability of code_execute actions during synthesis.",
      "constraints": [
        "Propositional formula must return SAT",
        "CSF code_execute hazard <= 0.10",
        "CSF epsilon <= 0.20"
      ]
    },
    {
      "goal": "Run the EvoSkill evolution loop",
      "context": "Evolve the extraction agent: attempt tasks with current skill set, analyse failures, generate new skills via CEGIS synthesis, evaluate against validation data, select top performers for the frontier. Max 10 evolution iterations with frontier size 3.",
      "constraints": [
        "Max iterations: 10",
        "Frontier size: 3",
        "Target accuracy: 0.92",
        "Early stop: if accuracy >= 0.92 for 3 consecutive iterations"
      ],
      "resource_bounds": {
        "max_execution_seconds": 600,
        "max_tokens_per_hour": 200000
      }
    },
    {
      "goal": "Configure drift detection for production monitoring",
      "context": "Use adapt_healing with DDM (Drift Detection Method) to monitor extraction accuracy in production. When DDM warning threshold is breached, trigger re-evolution. Use adapt_healing assisted patching for transient errors, with verification and human review before deployment.",
      "constraints": [
        "DDM warning level triggers re-evaluation",
        "DDM drift level triggers full re-evolution",
        "Auto-patch transient errors (ImportError, JSONDecodeError)"
      ]
    }
  ]
}

Pipeline Diagram

graph TD
    A[align_evals<br/>baseline metrics] -->|fitness scores| B[cegis<br/>sketch synthesis]
    B -->|candidate functions| C[formal_methods<br/>DPLL verification]
    C -->|verified functions| D[align_csf<br/>hazard bounding]
    D -->|safe candidates| E[EvoSkill Loop<br/>attempt → analyse → generate → evaluate → select]
    E -->|frontier update| F{accuracy >= 0.92?}
    F -->|no| B
    F -->|yes| G[adapt_healing<br/>DDM drift monitor]
    G -->|drift detected| B
    G -->|stable| H((Production Agent))

What You Need

  • Tier: Builder
  • Components: cegis, formal_methods, align_csf, align_evals, adapt_healing, agent_claude

Step-by-Step

Step 1: Establish Baseline

{
  "component": "align_evals",
  "operation": "evaluate",
  "params": {
    "predictions": ["...current agent outputs..."],
    "ground_truth": ["...validation labels..."],
    "metrics": ["accuracy", "precision", "recall", "f1"]
  }
}

Record the baseline metrics. These define the starting point for the evolution frontier and the reference for drift detection.

Step 2: CEGIS Sketch Synthesis

{
  "component": "cegis",
  "operation": "synthesize",
  "params": {
    "sketch_src": "def extract_amount(text):\n    pattern = ??\n    match = re.search(pattern, text)\n    return float(match.group(??)) if match else ??",
    "oracle_src": "def oracle(text):\n    # ground truth extraction\n    return {'$1,234.56': 1234.56, '$99.00': 99.0}[text]",
    "max_iterations": 8,
    "agency_start": 5,
    "agency_decay": 1
  }
}

How CEGIS Works

CEGIS (Counterexample-Guided Inductive Synthesis) implements Algorithm 1: (1) propose a candidate that fills the ?? holes, (2) verify against the oracle, (3) if a counterexample is found, add it to the test set and re-synthesise. Agency level starts at CODE_AGENT (5) and decays each iteration — the system progressively relies less on LLM generation and more on programmatic refinement.

Step 3: Formal Verification

{
  "component": "formal_methods",
  "operation": "verify",
  "params": {
    "formula": "(input_valid AND schema_matched AND all_fields_extracted) -> extraction_correct",
    "solver": "propositional"
  }
}

Then bound the hazard. align_csf checks one operation at a time (input is operation + n_steps + epsilon, not a list of actions); call it once per action and confirm the union bound stays within ε:

{
  "component": "align_csf",
  "operation": "check",
  "params": {
    "operation": "code_execute",
    "n_steps": 1,
    "epsilon": 0.20
  }
}

Step 4: Evolution Loop

The EvoSkill cycle runs the 5-stage loop:

  1. Attempt — run the current agent on validation tasks
  2. Analyse — identify failure patterns and error categories
  3. Generate — use CEGIS to synthesise new skills targeting the failure patterns
  4. Evaluate — score each variant against validation data via align_evals
  5. Select — top 3 performers enter the frontier

The loop terminates when accuracy reaches 0.92 for 3 consecutive iterations or after 10 iterations.

Step 5: Drift Detection

{
  "component": "adapt_healing",
  "operation": "detect",
  "params": {
    "metric_name": "extraction_accuracy",
    "current_value": 0.88,
    "baseline_value": 0.92,
    "method": "ddm"
  }
}

DDM Thresholds

The Drift Detection Method (DDM) tracks the running mean and standard deviation of a metric. Warning level: mean + 2σ deviation from baseline — triggers re-evaluation. Drift level: mean + 3σ — triggers full re-evolution via CEGIS. This is the same approach used in production ML monitoring (from Self_Healing_ML patterns).

Auto-patch transient errors:

{
  "component": "adapt_healing",
  "operation": "diagnose",
  "params": {
    "error_type": "JSONDecodeError",
    "func_name": "extract_amount",
    "context_json": "{\"input\": \"malformed JSON...\"}"
  }
}

What Happened

G6 ran a bounded self-improvement loop:

  1. align_evals established baseline accuracy (0.74)
  2. cegis synthesised extraction functions from sketches with ?? holes, verified by oracle counterexamples
  3. formal_methods proved correctness invariants; align_csf bounded hazard probabilities
  4. EvoSkill evolved the agent over 10 iterations, selecting top-3 frontier candidates each round
  5. adapt_healing configured DDM drift monitoring to trigger re-evolution when accuracy degrades

The key differentiator: CEGIS provides a structured generate-check-refine loop. When the specification is formal and the backend supports the property, it can produce machine-checkable evidence; otherwise treat the result as verified against oracle tests and recorded checks, not as a universal proof of correctness.

Why G6 Over a Bare LLM

A capable LLM can refactor code and suggest improvements when prompted. G6 adds a structured evolution loop — CEGIS for counterexample-guided synthesis, available formal checks for specified properties, drift detection for production monitoring, and bounded resource constraints at every step. Prebuilt templates turn ad-hoc prompting into a deterministic improve-check-deploy cycle triggered by one GoalInput JSON.