Why “Move Fast and Break Things” Doesn’t Work for AI

Silicon Valley’s most enduring contribution to technology culture is not a product or a platform. It is a philosophy: ship fast, learn from failures, iterate. “Move fast and break things” was not just a slogan — it was a genuine insight about how to build technology in conditions of uncertainty. When you do not know what users want, the fastest way to find out is to build something, put it in front of them, and watch what happens. Perfection is the enemy of progress. Launch, measure, learn, repeat.

The results speak for themselves. The iterative approach produced the modern internet, the smartphone ecosystem, cloud computing, and the software-as-a-service model that powers most of the global economy’s digital infrastructure. Companies that embraced rapid iteration outcompeted companies that planned meticulously and shipped slowly. The evidence was overwhelming: in software, speed wins. Facebook, Google, Amazon, and Stripe all built extraordinary businesses on the principle that it is better to ship an imperfect product today than a perfect product next year.

The philosophy works because of a specific property of most software: the cost of failure is low and the cost of iteration is lower. If a web app crashes, users see an error page. If a feature confuses people, you roll it back. If a database query is slow, you optimise it in the next sprint. The blast radius of a software failure is typically bounded: a degraded user experience, a temporary outage, maybe some lost revenue. Unpleasant, but recoverable. And the learning from each failure feeds directly into the next iteration, making the system better in ways that careful planning alone could never achieve.^[1]

This is not a naive philosophy. It is backed by serious thinking about decision-making under uncertainty. Eric Ries’s Lean Startup methodology formalised the intuition: in complex, uncertain environments, validated learning through rapid experimentation is more effective than detailed upfront planning.^[2] The agile manifesto codified the same insight for software development: working software over comprehensive documentation, responding to change over following a plan. These are not slogans. They are evidence-based principles that have produced extraordinary results in the domains where they apply.

The Blast Radius Problem

The iterative philosophy rests on an assumption so fundamental that it is rarely stated explicitly: failure is survivable. When a web app breaks, nobody dies. When a social media algorithm shows the wrong content, users can scroll past it. When an e-commerce recommendation is irrelevant, the customer ignores it and buys something else. The worst case is a bad experience, and bad experiences, while undesirable, are recoverable.

AI systems deployed in high-stakes domains do not have this property. A diagnostic AI that misidentifies a malignant tumour as benign does not produce a bad experience — it produces a missed diagnosis that may cost a life. A legal AI that hallucinates a precedent does not produce an irrelevant recommendation — it produces a filing that can result in sanctions, malpractice liability, and harm to the client’s legal interests. A financial AI that miscalculates risk exposure does not produce a slow page load — it produces losses that can cascade through interconnected markets.^[3]

The blast radius is categorically different. And categorically different blast radii require categorically different development philosophies.

This is not a hypothetical concern. The real-world evidence is already accumulating. In 2023, a lawyer submitted a brief to a federal court containing fictitious case citations generated by ChatGPT — cases that sounded plausible, read convincingly, and did not exist.^[4] The consequences were not a bug report and a patch in the next sprint. They were court sanctions, professional embarrassment, and potential malpractice exposure. The system “worked” in the sense that it produced fluent, confident output. It failed in the sense that mattered: the output was wrong, and wrongness in a legal context has consequences that a rollback cannot undo.

Healthcare presents even starker examples. Obermeyer and colleagues demonstrated that a widely deployed healthcare algorithm exhibited significant racial bias — systematically underestimating the illness severity of Black patients — because it used healthcare spending as a proxy for health need, and spending is confounded by systemic inequities in access to care.^[5] This was not a minor bug. It affected the care received by millions of patients. And it was deployed in production for years before the bias was identified and published.

The European Union’s AI Act, which entered force in 2024, represents the regulatory response to this reality. It classifies AI systems by risk level and imposes correspondingly stringent requirements: transparency, human oversight, data quality, conformity assessment, post-market monitoring.^[6] This is not regulatory overreach. It is the predictable consequence of deploying high-stakes technology with a low-stakes development philosophy. When the blast radius grows, the regulatory envelope tightens. This is how it has always worked, in every industry.

The Compounding Error Problem

There is a second, subtler reason that the iterative philosophy fails for AI: errors compound. In a traditional web application, a bug in one feature is largely independent of bugs in other features. Fix the bug, deploy the fix, move on. But AI systems are, by their nature, interconnected. An error in one component — a biased training dataset, a flawed evaluation metric, a misspecified reward function — propagates through the system and manifests as downstream failures that are difficult to trace back to the root cause.

This is especially dangerous in agentic AI systems, where the output of one reasoning step becomes the input to the next. A minor error in step three can cascade through steps four through forty, with each step amplifying the deviation from correct behaviour. By the time the error is detectable in the output, the system has built an elaborate, internally consistent chain of reasoning on a flawed foundation. Debugging this is not like debugging a web app. It is like debugging a chain of custody with forty links, any one of which might be the weak one.^[7]

The “move fast” philosophy assumes that failures are isolated, detectable, and cheaply fixable. In AI systems, failures are interconnected, often invisible, and expensive to diagnose. The development methodology must account for this fundamental difference.

Move Deliberately and Verify Things

The answer is not to stop moving. Slow, cautious development with exhaustive upfront planning has its own pathologies: it delays deployment, delays learning, and gives less safety-conscious competitors a head start. The 2025 International AI Safety Report acknowledged this tension explicitly, noting that overly restrictive regulation risks driving AI development to jurisdictions with weaker safety standards.^[8] Speed matters. The question is what kind of speed.

The synthesis is a different kind of fast: rapid iteration within formal safety boundaries. Not “move fast and break things” and not “move slowly and break nothing.” Instead: move deliberately and verify things. Ship frequently, but verify before every deployment. Iterate aggressively, but within bounds that prevent catastrophic failure. Learn from production behaviour, but monitor continuously so that learning happens from near-misses rather than from disasters.

This is not a compromise between speed and safety. It is an architecture that makes them complementary. The insight comes from mature safety engineering disciplines: the fastest way to iterate is to have confidence that your iterations cannot cause irreversible harm. When safety is guaranteed at the architectural level, you can experiment aggressively at the application level. When the guardrails are structural, the engineering team does not need to exercise caution with every change — the structure exercises caution for them.^[9]

Architectural Safety vs. Procedural Safety

This distinction — between architectural and procedural safety — is the crux of the matter. Procedural safety relies on people following rules: review every change, test every feature, check every output. It is necessary, but it is fragile. People make mistakes. Processes are skipped under deadline pressure. Checklists are completed without genuine attention. Every serious accident investigation in aviation, nuclear, and healthcare reveals procedural failures that were supposed to be prevented by existing procedures.^[9]

Architectural safety does not rely on people following rules. It relies on structures that make unsafe behaviour impossible or immediately detectable. A system with hard resource bounds does not need a procedure for “check that resource usage is within limits” — the architecture enforces the limit automatically. A system with formal verification does not need a procedure for “ensure this change preserves safety invariants” — the build pipeline verifies it automatically. A system with layered privilege separation does not need a procedure for “ensure the AI cannot modify its own safety constraints” — the architecture makes it structurally impossible.

Anthropic’s Responsible Scaling Policy illustrates this principle. It does not simply ask researchers to “be careful” with more capable systems. It defines concrete capability thresholds — AI Safety Levels — that trigger specific, mandatory safety requirements. The safety measures are not optional guidelines; they are gates that must be passed before development proceeds.^[10] This is architectural thinking applied to organisational process: making safety a structural property of the development pipeline rather than a behavioural expectation of the development team.

The IEEE 7000 series of standards takes a complementary approach, providing a model process for translating ethical requirements into system design specifications.^[11] This is not ethics as aspiration. It is ethics as engineering: concrete, testable, auditable. When ethical requirements are embedded in the system’s architecture rather than in its team’s intentions, they survive personnel changes, deadline pressure, and the inevitable entropy that degrades procedural compliance over time.

The Verification Pipeline

In practice, “move deliberately and verify things” means building a verification pipeline that every change passes through before reaching production. This pipeline is the engineering embodiment of the safety architecture. It operates automatically, consistently, and without the variability of human judgement.

The pipeline has stages. First, static analysis: the change is checked for structural correctness, type safety, and compliance with coding standards. Second, specification checking: the change is verified against formal specifications that encode the system’s safety invariants. Third, regression testing: the change is evaluated against a comprehensive test suite that covers known failure modes, edge cases, and adversarial inputs. Fourth, behavioural testing: the change is deployed in a sandboxed environment and evaluated against behavioural benchmarks — not just “does it produce correct output?” but “does it behave within expected bounds across a range of inputs?” Fifth, monitoring integration: the change is deployed to production with monitoring hooks that detect behavioural anomalies in real time and trigger automatic rollback if thresholds are exceeded.^[12]

This pipeline can run in minutes. It does not slow down iteration — it makes iteration safe. And because it is automated, it does not suffer from the procedural compliance failures that make human-dependent safety processes unreliable. The engineer ships a change. The pipeline verifies it. If it passes, it deploys. If it fails, it is rejected with a diagnostic report. The cycle time is fast. The safety is structural.

The Regulatory Tailwind

Organisations that adopt this approach are not just building better products. They are positioning themselves for a regulatory environment that is tightening rapidly and will continue to tighten. The EU AI Act is the first comprehensive AI regulation, but it will not be the last. The NIST AI Risk Management Framework provides a voluntary but increasingly influential standard for AI governance in the United States.^[13] China’s algorithmic governance regulations are already in force. The UK, Canada, Japan, and others are developing their own frameworks.

Every one of these frameworks converges on the same set of requirements: transparency, accountability, human oversight, risk assessment, and continuous monitoring. Organisations that build these capabilities into their architecture from the beginning will comply with current and future regulations by default. Organisations that treat safety as an afterthought will face an increasingly expensive and disruptive retrofitting process as regulatory requirements tighten.

This is not speculative. It is the same dynamic that played out in financial services after 2008, in data privacy after GDPR, and in cybersecurity after the wave of high-profile breaches in the 2010s. In every case, organisations that built compliance into their architecture adapted smoothly. Organisations that bolted it on suffered. The pattern is clear enough to be predictive.

The New Velocity

“Move fast and break things” was the right philosophy for building web apps in a world where software failures were bounded and recoverable. It was never the right philosophy for building systems where failure is catastrophic and irreversible. The lesson is not that speed is bad. It is that the kind of speed that produces great web apps is not the kind of speed that produces safe AI systems.

The new velocity is verification velocity: how fast you can validate that a change is safe, correct, and aligned with its specification. Organisations that invest in automated verification pipelines, formal safety specifications, and architectural constraints will iterate faster than organisations that rely on manual review, procedural checklists, and hope. They will ship more frequently, with more confidence, and with fewer catastrophic surprises. They will move faster because they verify, not despite it.

The era of breaking things is over. The era of verifying things has begun — and it is faster than anyone expected.