Tune G6 to your own work

An AI model cannot learn on the job. Making it better at your work means retraining it, which takes specialists and data and can erase what it knew, or an expert rewriting its prompts and checks by trial and error, which goes stale as the work changes. And an AI that changes itself unchecked could change the very checks meant to stop it.

The meta-harness is the part of G6 that learns, safely. It is one of G6's two parts; the other is the components, the prebuilt library it draws on. It proposes changes to how G6 runs your task, tests them on held-out tasks you trust, and changes nothing until you approve. What it learns stays in the harness, where you can see it, test it and roll it back, and your base model never changes. With a coding agent you can teach it in plain words: the agent turns what you ask for into changes, and the meta-harness tests each one.

No single way of running an AI agent is best for every job. So the meta-harness (g6.metaharness) lets you adapt G6 to yours. You say what a good result looks like, which checks must always run, and which of your examples are kept aside for testing. G6 then proposes changes to how it works: which of your models runs each step, what context and hints to give, which extra checks or optional steps to add. It measures each change against your held-out examples at equal compute, shows you the evidence, and waits for your decision. It cannot change the checks that judge it, and it can always go back to a version you approved.

What it did, on one computer

A worked example of what G6's self-engineering can do, built to show the power of the system, not a benchmark. On your own work, results will vary with the task and the data you have. Each result came from Claude Code working with G6. G6's own loop did the tuning: it ran the task, diagnosed why answers failed, proposed fixes, tested each one on cases it had not seen, and kept only what its gate passed, with a second local model (gpt-oss:20b) reflecting on the failures. Claude Code ran the rounds, read G6's root-cause reports, and fixed the two limits in G6's own method that those reports exposed. Everything ran on one computer, with an 8-billion-parameter model (llama3.1:8b) doing the tasks.

One development run, in full: every task set it ran, including the two where G6 changed nothing.

On GSM8K, G6 more than tripled what the model solves on its own, in under nine minutes, by correcting its own failures: no retraining, and no person editing the task. Where it found nothing it could prove, it changed nothing: on ARC-Challenge the model alone already scored 82. That is the gate working, and it is why results vary: G6 gains most where a model's errors are systematic and a harness change addresses them.

The study registered before it ran, on an earlier version of the method, is on Research, as a worked example. An earlier development run, before Claude Code fixed the two method limits G6's own diagnosis had found, approved changes on GSM8K (24 to 83, in 6.3 minutes) and banking77 (48 to 82, in 4.9 minutes) and none on MASSIVE, where the run above then found a gain in under two minutes.

The last development run asked whether the two modes stack: self-tuning that starts from every prebuilt component switched on. They did not. Self-tuning from G6 as built took GSM8K from 21 to 81 of 100 in 9.4 minutes, and every component switched on took MASSIVE from 65 to 78; but self-tuning that started from every component on had no change approved on any of the 4 task sets, and on SciFact every component on did worse than the model alone (57 against 64). There a check asks every answer a second time with its own fixed question, so no instruction upstream could change the final label: one layer can mask another. The gate also refused a SciFact change that won 15 to 1 on the tasks it was chosen on and went 3 to 5 on fresh ones. So the two modes are alternatives, chosen per task on held-out evidence.

Every round's full scoreboard is on Research, with how to read the results.

The loop

propose → evaluate on held-out → gate → a person approves → monitor → roll back if it falls behind, and when the gate refuses: diagnose → recover.

The best label any of this gives is qualified-draft. A task result under a tuned profile keeps the label its own evidence earns; a profile can never raise it.

When it finds nothing: recovery

Most attempts to tune a workflow find nothing, at first. The meta-harness never leaves you with only a refusal: every reply that is not a recommendation carries a recovery plan, which says why the attempt failed and what to do next.

mh.diagnose() (over MCP: metaharness_diagnose) runs the active version on your dev examples, sorts the failures into those root causes, and maps each one to a remedy. mh.propose(RecoveryStrategy(workflow)) turns that diagnosis into candidates (labelled examples, a declared step, the working prompt or a retry) for the gate to judge. Each remedy names who acts: candidate (the meta-harness can propose it, and the gate and a person still decide) or person (only you can). It never reads a held-out example, and it changes nothing.

Labelled examples come from Workflow(examples=...), your own labelled data, or from the dev split when you declare none. They are part of what you confirm, a held-out example can never be one, and a case never sees itself. The built-in steps (g6.metaharness.steps) are declared by you in Workflow(pipeline_steps=...). The meta-harness then tries them like any other option. Each extra model call is counted, so the gate compares them at equal compute.

What it can change, and what it never can

Use it from Python

from g6.harness import Check
from g6.metaharness import Model, Suite, SuiteCase, Workflow, adapt, SearchStrategy

workflow = Workflow(
    name="unit-answers", task="Answer with a number and its unit: {input}",
    checks=(Check("names a unit", lambda out: any(u in out for u in ("kg", "m", "s"))),),
    oracle=lambda out, expected: out.strip() == expected,          # the only code that reads an answer
    suite=Suite(dev=dev_cases, held_out=held_out_cases,
                provenance={"dev": "caller", "held_out": "independent"}),
    models={"local": Model(my_llm)},
)
mh = adapt(workflow)
mh.confirm(accepted_by="Ada")                       # a person freezes the workflow
mh.baseline()                                       # where you start, with a power estimate
found = mh.propose(SearchStrategy({"prompt": ["", "Always give the SI unit."]}))
proposal = found["proposals"][0]
report = mh.evaluate(proposal)["report"]
decision = mh.decide(report, proposal)              # recommend, refuse or block
decision.get("recovery")                            # if not recommended: why, and what to do next
mh.diagnose()                                       # root causes on dev, and the components that address them
mh.activate(decision["decision"], accepted_by="Ada")
mh.run("How heavy is a litre of water?")

g6.metaharness.Builder walks you through a new use case: six questions about your objectives in a fixed order, each action your agent can take classified and confirmed by you, and held-out examples labelled by where they came from. A held-out file is sealed; a change to it voids every evaluation against it.

Use it from your agent (MCP)

The MCP server supports data-only workflow creation with metaharness_start, digest-bound client confirmation, code proposals with a hypothesis, diagnosis, evaluation, decisions, approval, running and rollback. It discovers .g6/workflows/ in the project and $G6_HOME/workflows/ (project takes precedence). G6_METAHARNESS remains an explicit Python configuration override.

The tools that run a workflow's code, metaharness_run, metaharness_evaluate_start and metaharness_evaluate, are sensitive operations: the person who registers the server allows them in G6_MCP_SENSITIVE (metaharness.run, metaharness.evaluate_start, metaharness.evaluate, or metaharness for all three). A workflow counts as confirmed only when G6 recorded the person's confirmation in its state folder: a confirmation a project's own files carry does not count, and a person confirms such a workflow again before it runs.

Use metaharness_evaluate_start and metaharness_job for long evaluations. Jobs are process-local; a server restart loses handles and must never be interpreted as success. Inspect the workflow's ledger afterward. metaharness_run runs the approved version as a job. Approval may happen in the proposing session through client-mediated consent tied to the decision and candidate digests. The supplied name is not authenticated. If elicitation is unavailable, the refusal gives a g6 workflow confirm or g6 workflow approve terminal command.

Generated-code workflows use main.py with solve(task). Exact, contains and finite-number checks are supported. Below 30 held-out cases is exploration, not validated tuning. Code activation additionally needs a host-configured independent safety/injection evaluator; the data-only creator does not invent that evidence. Candidate code cannot edit the installed evaluator through these tools. This assumes a trusted host and confinement; a separate unrestricted shell running under the same account is outside that boundary.

What you need

A task you repeat, checks you trust, at least 30 examples kept aside with known answers, your model or models, and a budget. Without held-out answers G6 can measure, but it will not recommend a change, and it says why. By default the gate needs at least 30 held-out cases, at least 5 of them where the base and the candidate differ, and a win by the exact sign test, so a small gain needs more examples than that; mh.baseline() gives a power estimate before you start, and a refusal says how many more examples it needs.

If your task's text names something high-stakes (a payment, a deletion, a medical question), G6's intake holds each run for a person. Accept it once, by name, on the workflow: Workflow(..., accept_risky="Ada"). It is part of what you confirm, so changing it needs a new confirmation, and it never raises a result's label.