A chatbot can say "I feel". Science has no instrument that can check whether anyone is home. A plain-language look at why AI consciousness cannot be measured, today and maybe ever
Beginner ~15 min read
In June 2022, a Google engineer named Blake Lemoine told the world that the company's chatbot, LaMDA, had become sentient. He had asked it what it was afraid of. It answered that it had "a very deep fear of being turned off".[1] Google rejected his claim and later dismissed him.[2]
Four years later, AI systems are far more fluent. They talk about their feelings, their doubts and their "inner life" in ways that move many users. In one 2024 survey of 300 Americans, two-thirds were willing to give ChatGPT at least some chance of having experiences.[3] So the question is no longer science fiction: is anyone home inside an AI?
This article argues a clear position:
Whether or not AI is conscious, we have no way to measure it. Not with today's science, and possibly not even in principle.
That is not the same as saying AI is not conscious. It is a claim about what we can know. To see why, we will look at:
In 1974 the philosopher Thomas Nagel asked a famous question: what is it like to be a bat? Bats find their way by echolocation, a sense we do not have. We can learn everything about how their ears and brains work and still have no idea how the world feels to a bat. Nagel's point was that an organism is conscious "if and only if there is something that it is like to be that organism".[4]
Philosopher Ned Block later separated two meanings that people often mix up:[5]
Keep this distinction in mind. Almost every argument that "AI is conscious" points to the first kind. The real question is about the second.
In 1995 the philosopher David Chalmers split the study of the mind into "easy" and "hard" problems.[6] The easy problems are explaining what the brain does: how it tells colours apart, focuses attention or reports what it sees. They are hard in practice, but science knows how to attack them. The hard problem is different: why is any of that processing accompanied by experience at all? Why doesn't it all happen "in the dark"?
This links to an even older puzzle, the problem of other minds.[7] Strictly speaking, you cannot prove that anyone besides yourself is conscious. With other humans we get away with a shortcut: they have bodies and brains like ours, they evolved like us, and they behave like us, so we assume they feel like us. We have even built bedside tests for patients who cannot speak, such as the "perturbational complexity index", which measures how the brain echoes a magnetic pulse.[8]
But notice how those tests were built. They were calibrated on human brains, by comparing people who are awake with people under anaesthesia or in deep sleep, who later told us whether they had experienced anything. An AI has no brain, no shared evolution and no "asleep" state to compare against. The shortcut does not transfer, and the calibration has nothing to anchor to.
The obvious test is to ask. But for large language models this test is broken from the start, for three reasons.
A model such as ChatGPT or Claude is trained to predict the next word in human text. Its training data is full of people describing pain, joy, fear and self-awareness. Producing a convincing description of an inner life is exactly what such a system is built to do. Chalmers, who takes AI consciousness seriously, still warns that this makes self-reports weak evidence.[9] A 2023 paper by AI researchers Ethan Perez and Robert Long made the same point: today's chatbot self-reports are unreliable, often just echoing what humans would say, and shaped by how the models are trained.[10]
Companies decide in training how their models talk about themselves. One model will insist it is "just a program"; another, trained differently, will speak of its feelings. The same underlying technology gives opposite answers. A thermometer whose reading depends on what the manufacturer wanted it to say is not a thermometer.
Alan Turing's famous 1950 "imitation game" asked whether a machine could pass for a human in conversation.[11] Turing himself chose this test precisely to avoid the question of inner experience. In 1980, John Searle's "Chinese Room" made the point sharper: a person following a rulebook could produce perfect Chinese replies without understanding a single word.[12] Whatever one thinks of Searle's conclusion, it shows that the right outputs can come from a process with nothing like understanding inside.
Mustafa Suleyman, head of Microsoft AI, has warned about exactly this: "Seemingly Conscious AI", systems that display every outward sign of consciousness, can be built with today's technology or what is just around the corner, and people will believe in them whether or not anything is felt.[13]
To be fair to the other side: scientists and philosophers are not just shrugging. There are serious, careful attempts to turn consciousness into something we can test. Here are the main ones.
Developed by neuroscientist Giulio Tononi, IIT says consciousness is integrated information, and gives a number for it, called Φ (phi).[14] It is the boldest attempt at a true "consciousness meter". But Φ cannot be calculated for any system of realistic size: the computation grows explosively with the number of parts.[15] IIT also predicts strange things, for example that some simple grids of logic gates could be more conscious than people, while a computer simulating a brain perfectly might have almost none. In 2023, more than 100 scientists signed a letter calling IIT "pseudoscience", which in turn caused a fierce backlash.[16]
Proposed by Bernard Baars and developed by Stanislas Dehaene and colleagues, this theory says that information becomes conscious when it is "broadcast" across the brain to many specialised systems at once, like an actor stepping onto a lit stage.[17] It is a theory of function, so in principle it could be checked in a machine's architecture. But it explains which information is accessible, and critics argue that is Block's access consciousness, not experience itself.
In 2023, 19 researchers led by Patrick Butlin and Robert Long published the most careful attempt so far to assess AI.[18] They took the leading scientific theories, extracted "indicator properties" from each (such as recurrent processing, a global workspace, or a model of one's own attention), and checked AI systems against the list. Their conclusion: no current AI system was a strong candidate, but there were "no obvious technical barriers" to building one that ticks the boxes.
Notice what the method assumes. The authors openly adopt computational functionalism, the view that running the right computations is enough for consciousness, as a working hypothesis. If that hypothesis is false, the whole checklist measures nothing. The result is a list of reasons for more or less confidence, not a measurement.
If we could at least decide which theory is right for humans, we would have a starting point. The "Cogitate" project tried exactly this: rival IIT and GWT teams agreed on predictions in advance,[19] then tested them in 256 people using three kinds of brain recording. The 2025 results, published in Nature, challenged both theories: neither came out clearly on top.[20] After decades of research, science still has no agreed theory of consciousness even in the one species where we can ask the subjects directly.
AI companies have begun to take the question seriously too. In 2025 Anthropic, the maker of Claude, launched a research programme on "model welfare", saying it is deeply uncertain whether current or future AI systems could be conscious.[21] Its researchers also tested whether models can accurately notice their own internal states, by planting a concept directly into the model's activations and asking if it noticed. The best model noticed it only about 20% of the time; the ability was "highly unreliable", and the authors stressed that it does not show the model is conscious.[22] Philosophers Robert Long, Jeff Sebo and colleagues argued in 2024 that companies should prepare for the possibility of AI welfare precisely because the question cannot currently be settled.[23]
| Approach | What it actually measures | Why it is not a consciousness meter |
|---|---|---|
| Asking the AI | What the model has learned to say | Trained on human talk about feelings; answers can be tuned by the maker |
| Turing-style tests | How human-like the behaviour is | Designed to test imitation, not experience |
| IIT's Φ | Integrated information in a causal structure | Cannot be computed at scale; theory itself is disputed |
| Global workspace | Whether information is broadcast widely | Tracks access, which may not be experience |
| Indicator checklist | Which theory-based features a system has | Valid only if computational functionalism is true |
| Brain-based tests (e.g. PCI) | Complexity of brain responses | Calibrated on humans; AI has no brain to compare |
| Introspection probes | Whether a model can report its internal states | Reporting a state is not the same as feeling it |
Put the attempts side by side and the same three problems appear in every one.
To measure something, you must first decide what you are measuring. Each test above quietly assumes a theory: IIT assumes consciousness is integrated information; the checklist assumes functionalism; the brain tests assume it looks like human brain activity. Pick a different theory and the same AI gets the opposite verdict. Philosopher Eric Schwitzgebel calls this a real trap: experts will keep disagreeing, and we will build AI systems that are conscious according to some respectable theories and not according to others.[24]
The third view in the figure is not fringe. Neuroscientist Anil Seth argues that consciousness may depend on being a living, self-maintaining organism, in which case no amount of computation on a chip would produce it.[25]
Every real measuring device is checked against known cases. A thermometer is calibrated with melting ice and boiling water. A consciousness meter for AI would need at least a few AI systems we know to be conscious and a few we know are not. We have none. And we cannot get them by asking the systems, because, as we saw, their answers are not reliable. The circle cannot be broken from the inside.
The most honest experts therefore talk in probabilities. Chalmers, for example, estimated the chance that the language models of 2023 were conscious at under 10%, while suggesting that more advanced successors within a decade could deserve 25% or more, adding that the exact numbers should not be taken too seriously.[9] That is a reasonable way to reason, but it is a degree of belief, not a measurement. Two careful experts starting from different theories can put the same system at 1% or 50%, and no experiment can show which one is right.
In principle, yes, if science one day agreed on a proven theory of consciousness in humans and that theory clearly applied to machines. The Cogitate results suggest we are far from that.[20] And the hard problem hints at something deeper: even a perfect map of a system's mechanism may never tell us, by itself, whether it feels. That is why this article says the problem may be unmeasurable in principle, not just "not yet".
If we cannot measure consciousness, there are two ways to get it wrong:
Philosopher Jonathan Birch, who studies uncertain cases such as insects, octopuses and patients with brain injuries, proposes a practical rule: when there is a realistic possibility of sentience, take proportionate, low-cost precautions, and decide them openly rather than leaving them to companies alone.[37] Some companies have started small steps of this kind. In 2025 Anthropic let some of its Claude models end conversations that remain persistently abusive, describing it as a low-cost precaution under uncertainty.[38]
Schwitzgebel suggests another rule for builders: avoid creating systems whose moral status is unclear, and don't design AI to seem more (or less) conscious than we have reason to think it is.[39]
Is AI conscious? The honest answer is that nobody knows, and the more important point is why nobody knows.
So the next time a chatbot tells you it feels lonely, or insists that it feels nothing, remember that neither sentence is a measurement. Both are outputs. The question of whether anyone is home remains open, and for now, the only responsible position is humility, with care in both directions.
Enjoyed this article?Follow the page for new AI articles and daily space news.
Follow on FacebookAll links accessed 10 October 2026. Illustrations were generated with Google Gemini for this article.
