Scaling Laws Have a Ceiling

In January 2020, a team at OpenAI published a paper that would reshape the economics and engineering of artificial intelligence for the next half-decade. Kaplan et al. demonstrated that the performance of neural language models follows remarkably clean power-law relationships with three variables: the number of model parameters, the size of the training dataset, and the amount of compute expended during training.^[1] Double the compute, and loss drops by a predictable amount. Add an order of magnitude of parameters, and capabilities expand on a smooth, extrapolable curve. The result was not a vague trend but a precise mathematical relationship — one that held across seven orders of magnitude.

The implications were immediate and profound. If performance is a predictable function of scale, then building better AI becomes primarily an investment problem: spend more on compute, gather more data, train larger models, and improvements will follow as surely as compound interest. This framing catalysed the largest capital allocation to a single technology category in the history of computing. Billions of dollars flowed into GPU clusters, training infrastructure, and data acquisition — all predicated on the assumption that the scaling curve would continue to deliver.

And deliver it did. GPT-3 at 175 billion parameters showed emergent capabilities that GPT-2 at 1.5 billion simply did not possess. GPT-4, trained with substantially more compute, passed professional exams that its predecessors could not approach. Each generation confirmed the trend: more scale, more capability, on schedule and on budget. The empirical evidence was not merely suggestive. It was overwhelming.

So the scaling hypothesis — the proposition that intelligence is fundamentally a function of scale, and that sufficient scale will produce artificial general intelligence — is not a fantasy. It is grounded in real observations. The question is whether those observations support the conclusion that is usually drawn from them. And here, the mathematics of power laws has something important to say.

The Mathematics of Diminishing Returns

A power law of the form L = aC^-α, where L is loss, C is compute, and α is a small positive exponent, has a property that is easy to overlook when plotting on log-log axes: diminishing returns are structural, not accidental. On a log-log plot, the relationship looks linear — steady, reliable, infinite. On a linear plot, it tells a different story: each successive halving of loss requires exponentially more resources than the last. The curve flattens. It approaches an asymptote. This is not a bug in the empirical finding. It is the finding.^[1]

Hoffmann et al.’s Chinchilla paper in 2022 delivered the first major corrective to the naive scaling narrative.^[2] By carefully disentangling the contributions of model size and training data, they showed that the industry had been systematically misallocating resources: training models that were too large on datasets that were too small. Chinchilla, with 70 billion parameters trained on 1.4 trillion tokens, outperformed the 280-billion-parameter Gopher despite using less total compute. The scaling laws were real, but the industry had been following the wrong curve along the wrong axis.

The Chinchilla result was important not because it invalidated scaling laws but because it revealed their complexity. There is not one scaling curve but a family of them, parameterised by the allocation of resources across model size, data, and compute. Optimising along the wrong dimension produces waste, not intelligence. And once the Chinchilla-optimal frontier was identified, a new problem became visible: the data wall.

Villalobos et al.’s analysis estimated that at current rates of data consumption, the supply of high-quality text data for training language models will be effectively exhausted between 2026 and 2032.^[3] This is not a theoretical concern — it is an engineering constraint that is already shaping decisions at frontier labs. Synthetic data generation offers a partial solution, but it introduces its own problems: models trained on the outputs of other models can exhibit mode collapse, where the diversity of the training distribution narrows with each generation. Shumailov et al. described this as “model collapse,” demonstrating that recursive training on synthetic data causes progressive degeneration of the learned distribution.^[4]

The Reasoning Gap

Even if the data wall could be overcome — through synthetic generation, multimodal data, or other means — a more fundamental question remains: are there capabilities that next-token prediction simply cannot produce, regardless of scale? The evidence increasingly suggests yes.

Marcus has argued persistently and with considerable technical detail that certain cognitive capabilities — systematic compositionality, causal reasoning, robust abstraction — are not the kind of patterns that emerge from statistical regularities in text, no matter how much text you train on.^[5] This is not an argument against neural networks per se, but against the specific assumption that autoregressive language modelling is a sufficient computational primitive for general intelligence. A system trained to predict the next token in a sequence may learn impressive approximations of reasoning, but approximations break down at the boundaries — precisely where reliable reasoning matters most.

Dziri et al.’s work on compositional reasoning provides empirical support for this concern.^[6] They demonstrated that language models struggle with tasks requiring multi-step compositional reasoning — problems where the answer depends on combining multiple independent facts in a structured way. Critically, they showed that this failure mode does not improve smoothly with scale. Larger models are better at simple compositions but plateau on complex ones, suggesting a representational limitation rather than a capacity limitation.

Planning presents a similar challenge. Valmeekam et al. systematically evaluated LLMs on classical planning benchmarks and found that even the most capable models fail at tasks that require explicit search over a space of possible actions.^[7] The models can describe plans fluently — they know what a plan should look like — but they cannot reliably generate correct plans for novel problem instances. This is the difference between having seen many examples of a pattern and being able to execute the underlying algorithm. Next-token prediction captures the former; it is unclear whether it can capture the latter at any scale.

Verification may be the most telling case. A correct mathematical proof requires that every step follows logically from the previous ones — a single invalid step invalidates the entire chain. Language models can produce proofs that look correct, with proper notation and plausible-sounding reasoning, but systematic evaluation reveals frequent logical gaps that the model cannot detect because it has no mechanism for checking its own reasoning against formal rules.^[8] This is not a matter of insufficient training data. It reflects the absence of a verification primitive — a computational mechanism for checking whether a conclusion actually follows from its premises.

The Ceiling Is a Signal, Not a Wall

The scaling ceiling should not be interpreted as a dead end. It is more accurately understood as a signal — an indication that the next generation of capability gains will come from a different source than the last. The pattern is not unprecedented in the history of computing. Moore’s Law drove exponential improvements in transistor density for decades, but the gains in usable performance came increasingly from architectural innovations — caching, pipelining, out-of-order execution, specialised accelerators — rather than from raw transistor count alone. The same transition is now occurring in AI.^[9]

The foundation models are not going to get worse. They will continue to improve with scale, and that improvement will continue to be useful. But the rate of improvement per unit of investment is decelerating along the pure-scale axis, and the capabilities that matter most for reliable, autonomous AI operation — multi-step reasoning, self-verification, planning under uncertainty, persistent learning — appear to require computational mechanisms that next-token prediction does not naturally provide.

This analysis points toward a specific research programme. Rather than investing exclusively in pre-training scale, the field needs to develop what might be called post-training cognitive infrastructure — the architectural layers that sit on top of foundation models and supply the computational primitives that pre-training alone cannot produce.^[10]

The components of this infrastructure are becoming clearer. Structured reasoning systems that can decompose complex problems into verifiable steps, check intermediate conclusions against formal constraints, and backtrack when errors are detected. Memory architectures that persist knowledge across interactions, consolidate patterns from experience, and retrieve relevant information by content rather than by position in a token sequence. Planning mechanisms that can search over action spaces, evaluate consequences before committing, and adapt strategies based on feedback. And self-monitoring capabilities that allow the system to assess its own confidence, recognise when it is operating outside its competence, and request help rather than confabulating an answer.

None of these capabilities require abandoning foundation models. They require complementing them. The language model provides the computational substrate — the ability to process natural language, reason about relationships, and generate coherent responses. The cognitive infrastructure provides the scaffolding that makes those capabilities reliable — that transforms impressive demonstrations into trustworthy tools.

The Efficiency Argument

There is a practical dimension to this argument that is often underappreciated. Training frontier models is extraordinarily expensive — current estimates place the cost of training a GPT-4-class model in the range of $100 million, with next-generation models likely to cost substantially more.^[11] If the scaling curve is flattening, then each additional increment of pre-training capability comes at a higher marginal cost. Meanwhile, architectural improvements at the inference layer — better reasoning frameworks, memory systems, verification mechanisms — can extract more useful intelligence from existing models at a fraction of the cost.

This is not a speculative claim. The explosion of post-training techniques — chain-of-thought prompting, retrieval-augmented generation, tool use, multi-agent architectures — has already demonstrated that substantial capability gains can be achieved without touching model weights at all. Anthropic’s research on constitutional AI showed that careful post-training alignment can dramatically improve model behaviour without increasing model size.^[12] These are architectural gains, achieved through engineering rather than scale.

The implication is that the most efficient path to more capable AI systems may not run through the data centre but through the systems engineering layer. A well-designed cognitive architecture can make a 70-billion-parameter model outperform a 700-billion-parameter model on tasks that require structured reasoning, persistent memory, or self-verification — not because the smaller model is inherently smarter, but because the architecture supplies computational mechanisms that raw scale does not. In the T0–T3 framework, scaling alone produces T0 systems (zero adaptive traverses). Post-training cognitive infrastructure is what enables T1 (harness optimisation), T2 (continuous self-training), and T3 (structured theory building) — each level extracting more capability from the same base model.

Beyond the Curve

The scaling era has given us foundation models of remarkable capability — systems that process language with a fluency and breadth that would have seemed impossible a decade ago. That achievement is real, and it provides the substrate on which everything that follows will be built. But the scaling curve is bending. The power law that delivered the last five years of progress is approaching the diminishing-returns regime where each doubling of investment yields smaller incremental gains.

This is not cause for pessimism. It is cause for a shift in engineering focus. The transition from “make it bigger” to “make it smarter” is not a retreat from ambition but an escalation of it. Building cognitive infrastructure — the reasoning, memory, planning, and verification systems that transform raw language modelling capability into reliable intelligence — is a harder problem than training a larger model. It requires insights from cognitive science, formal methods, systems engineering, and software architecture, integrated into coherent systems that work reliably under real-world conditions.

The ceiling on the scaling curve is not a wall. It is an invitation — to look beyond the curve, to recognise that intelligence is not a quantity to be maximised but a system to be engineered, and to build the infrastructure that the foundation models have been waiting for.