Skip to content

Failure Modes

A failure mode is a named way an AI step can go wrong — and, in G6, the reason a particular harness exists. Every production harness declares the failure modes it mitigates (P3) and the new failure modes it introduces (P4), so the reliability stack can catch at the next layer what slips past the previous one.

The catalog is maintained in code (mvp.core.failure_modes.FailureModeRegistry) and the per-mode pages in this section are generated from it. Each page records:

  • Signals — what the failure looks like when it happens.
  • Detection — how G6 notices it.
  • Prevention — the guardrail applied before execution.
  • Recovery — what to do once it is detected.
  • Mitigating harnesses — which production harnesses are built to catch it.

Two families

Self-programming failures — ways an agent doing software work goes wrong: breaking existing functionality, attempting too much in one patch, claiming success it cannot prove, missing tests, skipping rollback, or taking an unsafe autonomous action. Detection is signal-driven and recovery is escalate, repair, and re-run targeted verification.

Context failures — ways the context fed to a model goes wrong: an authority conflict between instructions, stale context, memory poisoning, a missing user constraint, over- or under-retrieval, or a compaction that drops a decision or adds an unsourced claim. Prevention applies the authority-hierarchy, provenance, freshness, budget, and bounded-retrieval policy; recovery rebuilds the context packet from authoritative sources.

Severity and human review

Most modes are medium severity. A few are high severity because they can cause irreversible or trust-breaking harm — for example unsafe_autonomous_action and license_or_copyright_risk among self-programming failures, and authority_conflict, missing_user_constraint, memory_poisoning, and subagent_context_leakage among context failures. A subset (such as memory_poisoning and authority_conflict) is flagged to require human review rather than being resolved automatically. Each per-mode page shows its severity and human-review flag.