What Epistemology Teaches Us About AI Knowledge

Large language models have been trained on a substantial fraction of everything humanity has written down. The scale is difficult to internalise. Wikipedia, the entire indexed web, millions of books, decades of academic journals, every publicly available codebase, legal filings, patent databases, medical literature, philosophical treatises — all of it compressed into billions of parameters that can be queried in natural language. When you ask a modern LLM about Kantian ethics, Bayesian inference, the Krebs cycle, or the tax implications of a cross-border merger, it responds with the kind of breadth and depth that no single human expert could match across all these domains simultaneously.

This is not mere retrieval. The models synthesise. They draw connections across domains that would take a human researcher weeks to notice. They reframe problems in productive ways. They generate novel arguments by recombining existing ones. A physicist asking about protein folding gets not just a summary of the relevant biology but analogies to energy minimisation problems from statistical mechanics. A lawyer exploring contract ambiguity gets references to linguistic pragmatics and game theory. The cross-pollination is genuine, and it is productive.

So in the most intuitive sense of the word, these systems know things. They can answer questions correctly. They can explain complex topics. They can identify errors in reasoning and suggest corrections. If knowledge is measured by the ability to produce accurate, relevant, contextually appropriate information on demand — the operational definition most people implicitly use — then LLMs are the most knowledgeable entities that have ever existed.^[1]

But epistemology — the branch of philosophy that has spent twenty-five centuries analysing what it means to know something — has a sharper definition. And by that definition, something fundamental is missing.

The Gettier Problem, Scaled Up

Since Plato, the standard analysis of knowledge has been justified true belief. To know that P, three conditions must hold: you must believe that P, P must be true, and your belief must be justified — arrived at through a reliable process that connects your belief to the truth in the right way.^[2] In 1963, Edmund Gettier published a three-page paper that changed epistemology permanently by showing that justified true belief is not sufficient for knowledge.^[3] You can have a justified, true belief that is still not knowledge, because the justification and the truth are connected only by accident.

Gettier’s cases are deceptively simple. Smith has strong evidence that Jones will get the job, and that Jones has ten coins in his pocket. Smith forms the belief: “The person who will get the job has ten coins in his pocket.” As it happens, Smith himself gets the job, and Smith himself — unbeknownst to himself — also has ten coins in his pocket. His belief is true. It is justified (by his evidence about Jones). But it is not knowledge, because the connection between his justification and the truth is accidental.

Large language models are Gettier machines at industrial scale. They produce outputs that are frequently true, and that are “justified” in the sense that they are derived from a vast evidential base (the training corpus). But the connection between the training data and any particular output is statistical rather than evidential. The model does not trace a chain of reasoning from premises to conclusion. It does not evaluate the reliability of its sources. It does not distinguish between well-attested claims and claims that happen to appear frequently because they are popular misconceptions.^[4] When the output is true, it is true in the way that Gettier’s examples are true — the justification and the truth coincide, but the coincidence is not the kind of robust, non-accidental connection that epistemology requires for genuine knowledge.

Goldman’s reliabilism — the epistemological theory that justification consists in being produced by a reliable belief-forming process — provides a useful lens here.^[5] On Goldman’s account, a belief is justified if and only if it is produced by a cognitive process with a high truth ratio. This seems initially favourable to LLMs: their outputs have a reasonably high truth ratio across many domains. But Goldman’s framework also requires that the process be reliable in the relevant reference class. An LLM may be reliable on average across all medical questions, but catastrophically unreliable on rare drug interactions or atypical presentations — precisely the cases where the stakes are highest. And the system itself has no way to identify which reference class a given query belongs to, because it has no access to its own reliability profile.

The Problem of Provenance

Consider what happens when a human expert makes a knowledge claim. “The standard treatment for community-acquired pneumonia in adults is amoxicillin.” Behind this claim lies a recoverable chain of justification: clinical trials published in specific journals, meta-analyses conducted by identifiable research groups, guidelines issued by professional bodies with stated methodologies, and the expert’s own training and experience applying these guidelines. If challenged, the expert can trace the provenance of the claim — not perfectly, not always accurately, but the infrastructure for doing so exists within their cognitive architecture.

When an LLM makes the same claim, none of this provenance is available. The claim emerges from a compressed statistical representation of the training corpus. The model cannot say: “I believe this because of the 2019 NICE guidelines, which I weight heavily due to their systematic review methodology, cross-referenced with the BTS/SIGN guidance and my training data from UpToDate.” It cannot say this because the information is not stored or accessible in that form. The training process destroys provenance. What remains is a probability distribution over tokens that happens to assign high likelihood to the correct answer — but the reasons for that assignment are opaque, distributed across billions of parameters, and unrecoverable.^[6]

This is not merely a transparency problem. It is an epistemological problem. Floridi, in his philosophy of information, argues that knowledge in the information age requires not just access to true propositions but the ability to account for their informational provenance — to trace the causal and logical chains that connect an assertion to its evidential basis.^[7] Without provenance, you cannot evaluate reliability. Without reliability evaluation, you cannot distinguish knowledge from lucky guessing. And without that distinction, every output of the system occupies the same epistemic status — formally unjustified, regardless of whether it happens to be true.

Kahneman’s distinction between System 1 and System 2 thinking provides another angle on the same problem.^[8] System 1 is fast, automatic, and associative — it produces answers without deliberate reasoning. System 2 is slow, effortful, and analytical — it evaluates claims, checks evidence, and traces logical dependencies. LLMs are, in Kahneman’s terms, pure System 1. They produce responses through rapid pattern matching without the deliberate, evidence-evaluating processes of System 2. The problem is not that System 1 is always wrong — it is frequently right, and impressively so. The problem is that without System 2 oversight, there is no mechanism for detecting when System 1 is wrong, and no process for correcting it.

What Would Change Your Mind?

There is a question that epistemologists consider diagnostic of genuine knowledge: “What would change your mind?” A person who genuinely knows something can typically articulate what evidence or argument would lead them to revise their belief. A physician who diagnoses pneumonia can specify what lab results, imaging findings, or clinical developments would cause them to reconsider. A physicist who accepts general relativity can describe what experimental outcomes would constitute falsification. This capacity — what philosophers call sensitivity to counter-evidence — is a hallmark of justified belief.^[9]

LLMs cannot answer this question in any meaningful sense. If you ask a model “What evidence would change your assessment?” it will generate a plausible-sounding answer — because the pattern of “what would change my mind” responses exists in its training data. But the answer does not reflect an actual counterfactual sensitivity within the model’s reasoning process. The model does not maintain an internal representation of the evidential basis for its claims, so it cannot genuinely compute what changes to that basis would alter the claims. It is performing epistemic theatre — mimicking the form of defeasible reasoning without possessing the substance.

Sosa’s virtue epistemology offers a framework for understanding why this matters.^[10] Sosa distinguishes between “animal knowledge” (true belief produced by a reliable faculty) and “reflective knowledge” (true belief that the knower can defend and explain). A bird that reliably identifies predators has animal knowledge. A zoologist who can explain why certain visual patterns signal predation has reflective knowledge. LLMs, at best, have something analogous to animal knowledge — reliable outputs without reflective access to the basis of that reliability. And for high-stakes applications, animal knowledge is not enough. We need systems that can explain their epistemic basis, defend their conclusions under challenge, and revise their positions when presented with counter-evidence — systems with reflective epistemic competence.

Toward Epistemological Infrastructure

If the analysis is correct — if LLMs lack genuine knowledge in the epistemological sense despite their impressive performance — then the solution is not to abandon LLMs but to build the epistemic infrastructure they lack. The statistical substrate is powerful. What is missing is the justificatory superstructure.

Concretely, epistemological infrastructure for AI would involve three interlocking systems. The first is provenance tracking — the ability to associate every claim the system makes with its evidential basis. This does not require reconstructing the training process. It requires building a layer that explicitly records which sources, retrieved passages, or reasoning steps contributed to a given output, and making that record available for inspection. When the system says “Amoxicillin is first-line for community-acquired pneumonia,” the provenance layer should be able to produce: “This is supported by NICE guideline CG191 (2014, updated 2019), BTS/SIGN guideline (2015), and appears in 94% of clinical reference sources in the training corpus with consistent recommendations.”

The second component is confidence calibration that goes beyond aggregate statistics. This means not just knowing that the system is 85% accurate on medical questions in general, but maintaining a dynamic, context-specific estimate of reliability for each individual output. How much of this response is drawn from well-attested sources versus interpolated from sparse data? How consistent is the training corpus on this particular claim? Are there authoritative sources that disagree? These are the questions that Goldman’s reliabilism demands answers to, and they require infrastructure that current systems do not possess.^[5]

The third component is defeasibility awareness — the system’s explicit representation of what would change its conclusions. For every substantive claim, the system should maintain a record of its defeating conditions: what evidence, if encountered, would lower confidence in this output? What assumptions is the conclusion sensitive to? What alternative interpretations of the evidence were considered and rejected, and what would make them more plausible? This transforms the system from an oracle that pronounces to a reasoner that holds its beliefs provisionally and can articulate its epistemic commitments.

Epistemic Hashing: A Practical Approach

One concrete mechanism for implementing epistemological infrastructure is what we might call epistemic hashing — a technique borrowed, by analogy, from cryptographic hashing in computer science. In cryptography, a hash function maps data to a fixed-size fingerprint that allows verification without requiring access to the original data. Epistemic hashing would work similarly: each knowledge claim produced by the system would carry a compact fingerprint encoding its evidential basis — the sources consulted, the confidence level, the known defeating conditions, and the date of last verification.

This is not retrieval-augmented generation, although it builds on similar mechanisms. RAG addresses the access problem — giving the model relevant documents to condition its outputs on. Epistemic hashing addresses the justification problem — maintaining a persistent record of the evidential chain that connects each output to its sources, and making that chain available for audit, challenge, and revision.^[11] The distinction is between having evidence and knowing that you have evidence — between using sources and being accountable to them.

Williamson’s knowledge-first epistemology argues that knowledge is not analysable into more basic components (belief, truth, justification) but is itself a fundamental mental state that grounds the others.^[12] Whether or not this is correct as philosophy, it suggests an important engineering principle: knowledge infrastructure should not be bolted onto AI systems after the fact but should be a foundational architectural commitment. Systems designed from the ground up to track their epistemic state will be fundamentally different from systems that generate outputs and then try to assess them post hoc.

The Intellectual Honesty Problem

There is a deeper issue lurking beneath the technical questions, and it is fundamentally an ethical one. When a system produces an output that has the form of a knowledge claim — assertive, confident, specific — without possessing the epistemic infrastructure to ground that claim, it is engaged in what the philosopher Harry Frankfurt would call bullshit: speech produced without regard for its truth or falsity.^[13] Frankfurt distinguishes bullshit from lying. A liar knows the truth and deliberately contradicts it. A bullshitter does not care about the truth one way or another — truth is simply not part of the production process. By this definition, current LLMs are the most sophisticated bullshit generators ever constructed. Not because their outputs are false (they are frequently true), but because truth plays no role in the generation process. The model produces statistically likely continuations, and whether those continuations are true is, from the model’s perspective, a coincidence.

This is not a condemnation of the technology. It is a diagnosis of what needs to change. A system with epistemological infrastructure — provenance tracking, calibrated confidence, defeasibility awareness — would no longer be generating claims without regard for their truth. It would be producing claims that are explicitly connected to their evidential basis, accompanied by honest assessments of their reliability, and held provisionally pending possible revision. This is the difference between bullshit and scholarship — not that scholarship is always right, but that scholarship cares about being right and has the infrastructure to pursue that aim.

The philosophical tradition from Plato through Gettier to Goldman and Floridi converges on a single insight: knowledge is not just true output but true output that is connected to truth in the right way. Building that connection — making AI outputs not just accidentally correct but justifiably correct — is the central epistemological challenge of artificial intelligence. It is a challenge that no amount of scaling will solve, because it is not about the quantity of information but about the quality of the system’s relationship to that information.

The models gave us access to humanity’s collective output. What comes next is the harder, more consequential work of giving AI something epistemology has always insisted upon — not just answers, but reasons. Not just beliefs, but justified beliefs. Not just information, but knowledge.