Jessica Hullman argues that AI-generated results cannot sustain scientific knowledge through technical correctness alone. In Verification Is Not Knowledge, published by the Cosmos Institute on October 30, 2026, the Northwestern University professor makes accountability central: someone must be able to explain what a claim rests on and respond when it is challenged.
Her argument gives AI-for-science teams a design problem beyond producing correct outputs. Systems also need ways to preserve commitments, process criticism and revise results, while keeping human researchers capable of interrogating the work.
The essay is a philosophical argument, not an experimental finding or a benchmark of current AI systems. Hullman’s November 2 follow-up makes its practical implications clearer, particularly around persistent state, reputation and the risk of eroding scientific judgment.
The useful question for developers is therefore broader than whether an AI result passes a check: what happens when someone later finds a reason to challenge it?
What Verification Can and Cannot Settle
Verification can provide a strong reason to trust a result. Hullman explicitly acknowledges this in her follow-up: AI verification of AI-generated work supplies a rationale for believing that work is reliable. Her objection concerns treating that assurance as the entire machinery of science.
A formal proof illustrates the distinction. Checking that a conclusion follows from specified assumptions settles an important technical question. Choosing those assumptions, connecting the result to a scientific problem and deciding whether it supports a particular application remain additional tasks.
That distinction becomes consequential when the problem changes. A valid result may depend on an assumption that no longer fits the intended use. The original verification can remain sound while the application needs revision.
Hullman develops the point through a thought experiment she attributes to mathematician Terry Tao: an AI-discovered cancer treatment comes with a formal correctness certificate and a passed phase 3 trial, but no human understands how it was found. The treatment is hypothetical, not a reported medical breakthrough. In her account, the concern is whether the AI could have exploited a weakness in the trial itself, and whether anyone could investigate that possibility. Her essay uses the scenario to shift attention toward what happens when something goes wrong.
A patient could reasonably value the treatment without resolving those institutional questions. Accepting a useful output and maintaining the capacity to investigate its failures are compatible choices.
My view is that this is the strongest practical reading of Hullman’s argument. It avoids making complete human comprehension a prerequisite for using every result. Instead, it asks whether the surrounding research process can still detect trouble, locate the relevant assumptions and organize a response.
Why Criticism and Values Remain Part of Science
Hullman connects enthusiasm for automated science to the value-free ideal: the view that scientific judgments should be insulated from ethical and social values, relying on epistemic criteria alone.
Her criticism is not an invitation to ignore evidence. It concerns the belief that procedures can remove the need for judgment about evidence, research priorities and acceptable uncertainty.
In her follow-up essay, Hullman draws on philosopher Helen Longino to explain why criticism matters. Scientific communities negotiate and refine standards for what counts as objective evidence. Challenges must be capable of changing beliefs and practices for that process to function.
The practical consequence is easy to miss in an automated workflow. A system can accept comments, generate replies and continue producing papers without any criticism altering its commitments. That would provide the appearance of scientific exchange while leaving its corrective function weak.
Hullman also invokes philosopher Robert Brandom’s thought experiment about a community whose beliefs arise reliably but whose members cannot give reasons for them. In her account, the missing ability to give and ask for reasons prevents that community from possessing knowledge, despite holding true beliefs.
These are philosophical arguments, presented through Hullman’s interpretation. Their relevance to AI engineering is specific: reliable output generation and a functioning process of justification are different capabilities.
For a research team, this means a criticism channel needs a destination. An objection should be able to trigger a changed claim, a revised method, a narrower scope or an explicit defense. Generating another plausible explanation is insufficient if the system never records what the challenge changed.
When Named Authors Cannot Answer for a Claim
Hullman defines accountability as:
“being answerable for a claim, able to describe what it rests on and respond when it’s challenged”
That definition, from the original essay, makes authorship a continuing responsibility.
She cites computer scientist Tom Dietterich’s concerns about arXiv papers submitted by people who appear unlikely to understand their claims. The example illustrates an accountability gap; it is not a quantified audit of how often this happens.
A name on a paper ordinarily gives readers somewhere to direct questions. If the listed author cannot explain the argument, identify its dependencies or help investigate a failure, that connection becomes much less useful.
AI assistance does not automatically create this problem. Researchers already depend on collaborators, specialized instruments and methods they do not individually understand in full. The relevant distinction is whether responsibility remains workable across that division of expertise.
I would judge an AI-assisted authorship process by its response to a substantive objection. Can the responsible team trace the claim, explain the evidence and authorize a correction? That is a more useful standard than the percentage of prose or code generated by a model.
Turning Accountability Into a System Requirement
Hullman’s November follow-up identifies a mismatch between scientific accountability and how frontier models are developed. She argues that agents are not trained to maintain the persistent state needed to hold commitments and update beliefs consistently as challenges arrive. She also argues that reputation should constrain what gets claimed. Her discussion leaves open how far present systems can satisfy those requirements.
A practical implementation would need more than a long conversation history. The following framework is an engineering interpretation of her argument, not an architecture she tested or a checklist validated by the essays.
| Design requirement | What it would preserve | Why it matters after verification |
|---|---|---|
| Persistent claim records | The claim, assumptions, evidence and relevant system versions | Later criticism can address the actual result and its dependencies |
Frequently Asked Questions
3 questions
1Does Hullman Argue Against Using AI in Science?
Her argument targets the idea that verified outputs alone are enough. It focuses on preserving criticism, accountability and human judgment as AI takes on more research work.
