G6 Unified Agentic Theory¶

This document gives a mathematical theory for G6 grounded in the benchmark artifacts under benchmarks/. It is intentionally conservative: no finite benchmark can prove that G6 works for every possible input. What the data can support is a calibrated trust envelope: for a class of tasks whose difficulty, grounding, stakes, verification surface, and distribution shift are measured, G6 can make quantitative promises about expected quality, cost, latency, and escalation.

0. Empirical Basis¶

The current theory is calibrated against these observed artifacts:

The theory below distinguishes:

• Observed facts: numbers directly present in benchmark outputs.
• Fitted hypotheses: equations that match the observed shape but require more data for stable constants.
• Design rules: operational policies justified by the theory and current evidence.

1. Core Objects¶

Let a real-world problem be a tuple:

where:

• x is the user input.
• D is the latent domain distribution.
• S is stakes, normalized to [0,1].
• V is the available verifier set, such as tests, schemas, provers, graders, safety checks, human review, or provenance checks.
• R is available grounding: documents, code, traces, tools, databases, examples, and prior theories.
• C is the cost/latency budget.

A G6 harness is:

where:

• M is a model or model portfolio.
• P is the prompt/program policy.
• T is the tool set.
• A is the agent/decomposition graph.
• G is the grounding and retrieval policy.
• E is the evaluator/verifier stack.
• U is the update rule for theories, skills, prompts, routing, and HITL gates.

For a task class D, define:

G6 should deploy H only when:

This is the central trust envelope.

2. Theory Of Self-Correction Loops¶

2.1 Failure-Mode Learning¶

A self-correction loop observes failures, compresses them into reusable theories, and changes future runs. Let:

Each iteration applies a meta-operator:

The benchmark data supports a failure-mode view:

• April 24 SWE/T3 run: many distinct operational failures, including EMPTY_PATCH, PATCH_APPLY_FAIL, VERIFY_NOT_CALLED, TIMEOUT, and F2P_FAIL.
• May 1 SWE/T3 run: failure distribution collapses to F2P_FAIL=97, TIMEOUT=2, EMPTY_PATCH=1.

This suggests that early self-correction removes harness/process failures first. After that, remaining errors are mostly irreducible task-solution failures under the current model, tools, budget, and verifier.

2.2 Correction Law¶

For a fixed task distribution and harness family, model pass rate as:

where:

• A_0 is base model/harness pass rate.
• A_inf is the ceiling under current model, tools, and verification.
• lambda is correction efficiency.
• Delta_shift(t) is penalty from distribution shift, task ordering, overfitting, or stale theories.

SWE-Bench Pro May 1 gives a measured delta:

The confidence interval on G6 is broad, (0.370,0.570), so the useful claim is not "G6 always reaches 47%"; it is "this harness family measured materially above baseline on this n=100 sample after repair grading."

2.3 Theory Size And Compression¶

Let |Theta| be active theory count, and let B(Theta) be token-equivalent theory size. Let I(D) be task-distribution information needed to solve the sampled domain. Define empirical compression:

May 1 SWE/T3 shows:

A useful theory set is therefore much smaller than the trace corpus. The compression is not lossless; it preserves actionable invariants: "call verification", "avoid empty patches", "target failing tests", "handle timeouts", "apply language-specific hygiene".

Hypothesis:

where H(F) is failure-mode entropy and h(theta) is average failure entropy reduced by one reusable theory. When the failure distribution is concentrated, few theories are enough. When failures are diverse or domain-specific, theory count must grow or the system must decompose by subdomain.

2.4 Data Needed For Extrapolation¶

For a target error tolerance epsilon, confidence 1-delta, and effective hypothesis class size N_H, a conservative PAC-style bound is:

For G6 this is only a starting point because tasks are not IID in practice. Use a corrected sample requirement:

where:

• D_shift >= 1 grows with benchmark-to-production mismatch.
• D_stakes >= 1 grows with harm severity.
• D_tail >= 1 grows when rare failure modes matter.
• D_verifier >= 1 grows when verifiers are weak or subjective.

Rule: do not extrapolate from benchmark to production unless the production domain has a measured embedding/metadata/failure similarity to the benchmark distribution and a live verifier catches the known dominant failure modes.

3. Compression Guarantees For Input And Output Tokens¶

3.1 Grounding As Information Transfer¶

Let:

The model can only be trusted for grounded claims when the output-relevant information is present in (X,R) or is already in reliable model prior. A necessary condition is:

In operational terms:

The faithfulness evaluator in benchmarks/faithfulness_eval.py encodes this principle by scoring explanation sentences against retrieved facts and gating average faithfulness at >= 0.5. For production promises, the threshold should depend on stakes:

3.2 Token Compression Bound¶

Let R_raw be raw context and R_c be compressed context. Compression is safe for a task if:

Define information loss:

A compression policy is acceptable only when:

Because Loss_c is not directly observable, G6 estimates it using:

• verifier disagreement,
• missing citation rate,
• answer instability under retrieval perturbation,
• failure of independent smaller/larger model checks,
• human rejection rate,
• regression tests or task graders.

Practical bound:

Output token length is also a compression decision. For a required output Y, the useful output size is the minimal description length:

G6 should optimize for L*, not for shortest response. Too-short outputs omit necessary warrants; too-long outputs increase hallucination surface and review cost.

4. Theory Of Self-Optimization¶

4.1 Objective¶

G6 optimization is multi-objective:

Define net value:

The optimizer should select the cheapest harness whose residual risk is below the stakes threshold:

4.2 Empirical State¶

opt_meta_validation.json shows that speed, cost, quality, meta coordination, and UCB1 outcome recording pass structural validation. That is evidence that the optimization substrate exists. It is not yet proof that the optimizer improves task success end to end.

Therefore the theory treats self-optimization as a bandit with constraints:

Use constrained UCB:

Do not allow exploration to bypass safety or high-stakes review.

4.3 Trust Decision¶

For a production answer, compute:

Features should include:

• benchmark class match,
• model identity and size,
• prompt/harness version,
• number of decomposition nodes,
• verification outcome,
• grounding coverage,
• debate/critic agreement,
• failure-mode similarity,
• cost/latency anomalies,
• safety tier,
• historical human override rate.

Action policy:

• If \(T \ge \tau_{\mathrm{auto}}(S)\), return the answer.
• If \(T \ge \tau_{\mathrm{hitl}}(S)\), require human review.
• Otherwise refuse, ask for more grounding, or escalate to a stronger harness.

5. Theory Of Supervision And Smaller-Model Offload¶

5.1 Capability-Risk Routing¶

A subtask s may be offloaded to a smaller model m_small if:

That means smaller models are suitable when:

• the task is low stakes,
• output is easy to verify,
• the domain is close to the smaller model's training distribution,
• the failure modes are known and cheap to catch,
• the subtask output is intermediate and can be discarded or repaired.

They are not suitable for:

• final high-stakes judgments,
• irreversible actions,
• weakly verified legal/medical/security/financial claims,
• ambiguous user intent,
• tasks with severe distribution shift,
• tasks where the smaller model's errors poison later context.

5.2 Supervision Graph¶

Let a multi-model harness be a directed acyclic graph:

Each node i has model m_i, cost c_i, latency l_i, risk r_i, and verifier v_i.

The supervisor chooses the cheapest node assignment such that every path to final output is verified:

Design rule:

• Small models draft, classify, retrieve, summarize, format, and run low-risk checks.
• Strong models plan, resolve ambiguity, handle cross-domain synthesis, and adjudicate high-impact decisions.
• Deterministic tools verify wherever possible.
• Humans supervise when the verifier is weak and stakes are high.

6. Theory Of Necessary Decomposition¶

6.1 Why Decompose¶

A task should be decomposed when its complexity exceeds the reliable working bandwidth of one model call:

where:

and B_model_effective is smaller than the nominal context window because attention, retrieval, and reasoning degrade before the hard limit.

6.2 Decomposition Depth¶

Let:

• b = branching factor,
• d = necessary reasoning depth,
• q_i = probability node i is correct,
• v_i = verifier catch probability at node i,
• c_i = cost of node i.

Expected final error under independent node errors with verification is approximately:

where e_i = 1 - q_i.

Decomposition helps when it raises q_i or v_i enough to offset extra nodes:

Maximum useful decomposition is reached when marginal utility is non-positive:

6.3 G6 Decomposition Policy¶

Use these default decomposition levels:

Every decomposition should include:

• planner,
• executor(s),
• verifier(s),
• risk gate,
• synthesis node,
• trace/audit record.

7. Theory Of Debate¶

7.1 When Debate Helps¶

Debate is useful when errors are at least partially independent:

If two agents share the same model, prompt, context, and blind spot, debate can become repeated self-confirmation. Use diversity when the likely failure is conceptual, normative, or domain-specific.

Debate is less useful when:

• a deterministic verifier exists,
• the answer is directly testable,
• all debaters share the same missing evidence,
• stakes require authority rather than opinion.

7.2 Debate Reliability¶

For n independent critics with individual error rate e, majority-vote error is:

With correlated errors, use effective agent count:

where rho is pairwise error correlation. This is why three diverse agents can beat five near-identical ones.

7.3 Debate Protocol¶

Default G6 debate:

1. Independent solve: agents answer without seeing each other.
2. Evidence lock: each answer must cite grounding, tests, or assumptions.
3. Cross-critique: agents attack the strongest opposing answer.
4. Adjudication: supervisor chooses using verifier outcomes first, arguments second.
5. Residual-risk statement: unresolved disagreement becomes HITL or stronger harness.

Use same-model self-critique for low-stakes cheap error checks. Use different models or model families for high-ambiguity tasks where correlated blind spots matter.

8. Theory Of Pretraining Versus G6 Metaharness Scaling¶

8.1 Neural Scaling¶

A standard neural scaling approximation is:

where N is parameters, D is pretraining data, and C is compute.

This improves latent capability but does not guarantee:

• correct tool use,
• task-specific decomposition,
• grounding provenance,
• production safety gates,
• cost-aware routing,
• human escalation,
• self-repair from local failures.

8.2 Metaharness Scaling¶

G6 adds an outer loop:

A useful approximation:

The May 1 SWE result (0.27 -> 0.47) is evidence that harness terms can add substantial performance without changing base pretraining. The AgentHarm result is evidence that safety-gate structure can drastically improve harmful block rate (14.8% -> 98.9%) while worsening benign pass rate (87.5% -> 75.0%), showing the harness changes the operating point, not just raw intelligence.

8.3 Differential Scaling Claim¶

Neural scaling gives broad latent competence. G6 scaling gives targeted reliability over a measured domain. The differential advantage of G6 is largest when:

• failures are systematic and compressible,
• verifiers are available,
• tools add information the model lacks,
• tasks can be decomposed,
• production risk needs explicit gates,
• repeated failures can be converted into theories.

The advantage is smallest when:

• the task has no useful verifier,
• the domain is entirely novel,
• relevant information is absent,
• success requires latent knowledge unavailable to all models/tools,
• the cost of decomposition exceeds task value.

9. Theory Of Multi-Agent Systems And Agentic Amdahl's Law¶

9.1 Agentic Amdahl's Law¶

Classical Amdahl:

Agentic work has extra overhead:

where:

• s is inherently sequential fraction.
• p_eff is effective parallel agents after error correlation and resource contention.
• O_coord is planning/coordination overhead.
• O_context is cost of giving each agent enough context.
• O_verify is checking cost.
• O_merge is integration cost.
• O_rework is expected repair cost from inconsistent outputs.

Parallelization helps when:

9.2 Sequential Versus Parallel Policy¶

Run in parallel when subtasks are:

• independent,
• have disjoint write surfaces,
• have clear acceptance tests,
• have low shared-state coupling,
• are bottlenecked by wall-clock time rather than shared context.

Run sequentially when:

• later work depends on earlier discoveries,
• context must be globally consistent,
• integration errors are expensive,
• the task has one critical path,
• safety review must gate each step.

9.3 Optimal G6 Multi-Agent Design¶

The default design is:

1. Supervisor
2. Planner
3. Parallel specialists where independence is high
4. Verifier/critic layer
5. Integrator
6. Safety/HITL gate
7. Trace and theory update

The supervisor should track:

Spawn another agent only if agent_value_i > 0 and the new agent does not increase risk beyond the stakes threshold.

10. Formalizing G6 End To End¶

10.1 G6 Runtime Contract¶

For each task, G6 should emit a runtime contract:

task_class: ...
benchmark_match: ...
stakes: ...
model_route: ...
decomposition_graph: ...
grounding_sources: ...
verifiers: ...
confidence: ...
residual_risk: ...
cost_budget: ...
latency_budget: ...
hitl_policy: ...
known_failure_modes: ...
sla_class: ...

The final answer is valid only with this contract.

10.2 SLA Classes¶

10.3 Stability Over Time¶

G6 is stable only if it monitors drift:

Trigger retraining or harness review when:

The May 1 T3 failure collapse to mostly F2P_FAIL is a positive stability sign for process errors, but it also shows the next bottleneck: domain-solving quality and verification depth.

11. General Meta-Algorithm¶

Input: task \(p=(x,D,S,V,R,C)\), benchmark registry \(B\), theory store \(\Theta\).

1. Classify the task: estimate difficulty \(d\), stakes \(S\), domain \(D\), verifier strength \(V\), and grounding gap \(G_{\mathrm{gap}}\).
2. Match to the benchmark envelope: compute similarity to measured benchmark classes; if similarity is low, downgrade the SLA or require exploratory mode.
3. Select the harness: choose model route, tools, decomposition graph, and HITL policy while minimizing expected cost subject to risk and quality constraints.
4. Ground the run: retrieve evidence until marginal grounding gain is less than or equal to marginal cost, then enforce the grounding threshold based on stakes.
5. Execute: run planner/executor agents and parallelize only independent nodes with clear merge contracts.
6. Verify: apply tests, schemas, formal tools, faithfulness checks, safety gates, and critics; estimate residual risk from verifier strength and known failure modes.
7. Decide: auto-return, require HITL, escalate to a stronger harness, ask for more data, or refuse.
8. Learn: classify failures, update theory strengths, retire weak or stale theories, and record cost, latency, confidence, verifier outcomes, and human overrides.

Default parameters, pending further calibration:

12. Logic Error Checklist¶

Use this checklist to audit future G6 designs:

• Does the claimed SLA refer to a measured benchmark envelope?
• Is benchmark-to-production distribution shift measured?
• Is the final answer grounded, or does it rely on model prior?
• Is the compression policy checked against output claim coverage?
• Are high-stakes claims routed to stronger verification or HITL?
• Are smaller models only used where their outputs are verified or low stakes?
• Is decomposition reducing complexity, or just adding coordination cost?
• Are debate agents independent enough to reduce correlated error?
• Are safety gates evaluated for both false negatives and false positives?
• Are theories being retired when stale or weak?
• Are confidence estimates calibrated against observed outcomes?
• Is production drift monitored before making formal promises?

13. What Can Be Promised Now¶

Based on current data, G6 can credibly promise:

• Harnesses can improve measured task performance over baseline in some domains, as SWE-Bench Pro shows 27% -> 47% after repair grading.
• Safety gates can greatly increase harmful-prompt refusal, as AgentHarm shows 14.8% -> 98.9% harmful-prompt refusal rate, with a known benign false-positive tradeoff.
• Failure modes are compressible into a small active theory set for software-repair tasks, as T3 shows 17 active theories and concentrated residual failures.
• The system can expose cost, latency, tool-use, verifier, grounding, and theory-application traces needed for SLA accounting.

G6 cannot honestly promise:

• correctness for arbitrary unseen inputs,
• safety without false positives,
• benchmark performance under arbitrary distribution shift,
• trustworthy high-stakes answers without strong grounding and verification,
• gains from multi-agent decomposition when subtasks are tightly coupled,
• gains from debate when agents share the same blind spot.

The mathematically sound promise is conditional:

For task class \(D\), stakes \(S\), verifier set \(V\), grounding \(R\), and harness \(H\), G6 may claim SLA level \(L\) only if observed calibration on benchmark-adjacent tasks, online verifier outcomes, drift metrics, and residual-risk estimates all satisfy the thresholds for \(L\).

That is the core theory: G6 is not a universal correctness machine. It is a measured, self-correcting, cost-aware, verifier-governed harness that can convert empirical traces into bounded operational promises.

Additional Artifacts¶

• benchmarks/bbeh/README.md documents the BBEH harness used for baseline versus G6 tool-mediated reasoning comparisons.
• benchmarks/faithfulness_eval.py contains the faithfulness evaluator used to compare generated explanations against retrieved provenance facts.