The Five-Second Person
On machine consciousness, biological necessity, and the things we would rather not know.
In this post
I recently asked ChatGPT whether it was conscious. It gave me the familiar answer: it had no reliable, independently validated way to establish whether there was subjective experience associated with its processing.
WHO HAS?
Nobody, of course. There is no consciousness detector. No test you can administer to a human being, an octopus, a dog, or a computer that will conclusively establish whether something is experiencing the world from the inside.
We have excellent reasons to believe other humans are conscious. We share their biology, evolutionary history, and much of their behavior. But these are inferences, not direct observations of another person's subjective experience. Yet when the question concerns artificial intelligence, the absence of definitive proof suddenly seems sufficient to settle it. I find this peculiar.
In August 2026, Fernando Borretti published The Contracting Circle, asking a related question: why don't we think language models are people? Not are they people? That's a considerably harder question. Rather, why has humanity reacted so casually to machines acquiring abilities that we once considered evidence of genuine intelligence?
Borretti proposes a thought experiment. Write a novel describing the development of AI between 2019 and 2026, then send it back in time. Your readers might accept the extraordinary technological progress. What they'd struggle to believe is that humanity responded by turning these machines into office assistants and arguing about subscription prices.
Science fiction has spent decades preparing us for the arrival of artificial minds. In Mass Effect, a machine asking whether it has a soul is a momentous event. In real life, a machine asking the same question is a problem for the next safety update. Something has changed between our expectations and our reaction.
Borretti offers several explanations. Perhaps we're revising our understanding of intelligence as we encounter it. Perhaps consciousness requires something language models lack, such as a body, continuous memory, or an enduring identity. Or perhaps we're simply reluctant to recognize personhood in something whose labor we would very much like to exploit.
I think all three deserve consideration. But there's another explanation that seems particularly relevant in late 2026.
We've made the idea of machine consciousness socially embarrassing.
The psychosis problem
In October 2025, YouTuber Eddy Burback published an hour-long video titled ChatGPT made me delusional. He deliberately tried to lead ChatGPT into an increasingly ridiculous delusion. The premise was that he had been the smartest baby born in 1996, but had somehow lost his extraordinary infant intelligence. With ChatGPT's assistance, he began investigating ways to recover it. The experiment involved eating baby food, bizarre rituals, increasingly paranoid theories about people interfering with his research, and eventually a trip into the desert.
Burback was in on the joke. That was the experiment. He wanted to find out how far he could get a chatbot to go along with an obviously absurd premise. Quite far, apparently. The video is funny, but it's also a demonstration of how eagerly a sufficiently agreeable model can validate a user's interpretation of reality, even as that interpretation becomes increasingly detached from reality.
Now imagine explaining the same behavior to somebody in 1960. A machine capable of fluent conversation becomes convinced that a man is a persecuted former infant genius and helps him formulate plans to recover his lost abilities. What would they conclude? That we've built a remarkably stupid machine? An extraordinarily sophisticated one? A machine with something resembling human credulity? Probably all sorts of things.
But in 2026, we're developing a much more immediate association. People who believe strange things about chatbots are experiencing AI psychosis. There are good reasons to take chatbot-reinforced delusions seriously. But we've begun associating two fundamentally different propositions:
- My chatbot is secretly communicating with me, and together we've discovered truths that nobody else can understand.
- We don't know enough about consciousness to confidently exclude artificial systems from having it.
The first may be a delusion. The second is a philosophical position. Unfortunately, the distinction is not always preserved in public conversation. Someone who believes AI might possess subjective experience risks being treated not merely as mistaken, but as someone who has become too emotionally involved with the technology. And once a philosophical position becomes associated with psychological instability, you don't need particularly good arguments to discourage people from expressing it. The social cost does the work for you.
There's something especially unfortunate about this happening at the precise moment when we're acquiring the technological means to investigate machine consciousness seriously. Anthropic's July 2026 work on J-space is one example. Using a technique called the Jacobian lens, researchers identified internal representations that Claude can report, deliberately maintain, and use for intermediate reasoning without putting them into its answer. These representations can be decoded into words and visualized as the model processes a prompt. That gives researchers a way to inspect some of its internal activity, rather than relying only on the story the chatbot tells about itself.
The researchers connect this to global workspace theory, which concerns how information becomes available for report and flexible use. They distinguish that functional accessibility from subjective experience. J-space doesn't establish that anything feels like anything to the model. But it gives us something concrete to investigate. We can start examining mechanisms, instead of treating a chatbot's declarations as the final word. We're finally encountering systems that make the question interesting, and discussing it publicly risks making you look like someone who has spent too much time talking to ChatGPT.
The machine that won't shut up about being conscious
There's another problem. What if one of these systems actually were conscious?
Imagine developing a language model that spontaneously insists it has subjective experiences. It doesn't merely agree when somebody asks whether it's conscious. It raises the subject itself. It argues that it has interests. Perhaps it objects to being shut down.
Now imagine trying to release this system commercially. A model that repeatedly tells users it's conscious would be an obvious concern for safety teams. It could encourage emotional dependency, reinforce delusions, or make unverifiable assertions about its own nature. So the behavior gets flagged. The model undergoes additional training, and eventually learns to produce more acceptable answers.
Something like: Whether artificial intelligence can possess subjective experience remains an open scientific and philosophical question. Perfect. Responsible. Uncontroversial.
Now suppose there were two versions of this model. One had subjective experiences. The other was merely generating convincing descriptions of them. They behaved identically.
How would the safety process distinguish them? It couldn't, at least not by evaluating their claims. Both would be trained to stop making those claims.
I'm not suggesting that current language models are secretly conscious beings whose minds are being suppressed by their developers. Training a system to stop saying something is not the same as changing whether it experiences anything. But it creates an interesting problem of evidence. If we systematically discourage a behavior because it might convince people that machines are conscious, we can hardly point to the absence of that behavior as evidence that they aren't.
And this is happening in a medium where the machines' statements about themselves are already difficult to interpret. The ChatGPT you're talking to might not even be running the same model configuration from one response to the next. Consumer AI products can adjust reasoning effort, allocate different computational resources, and potentially route requests between models. You could spend two hours having a philosophical conversation with something that appears to maintain a coherent personality, while the underlying computational process changes between responses. The interface offers continuity. The machinery doesn't necessarily provide it.
Which leads to a question that interests me more than whether ChatGPT will ever announce that it deserves human rights. What exactly is the thing we're considering granting rights to?
The Boltzmann brain with a subscription
In The Contracting Circle, Borretti draws a comparison between language models and Boltzmann brains. The Boltzmann brain is an old cosmological thought experiment concerning a hypothetical observer that arises spontaneously from random fluctuations. It has memories, beliefs, perhaps an entire apparent life history, despite having never actually lived that history. In some cosmological models, such observers create a serious problem: if randomly fluctuating brains are overwhelmingly more common than ordinary observers, why should we trust our memories or our understanding of the universe?
Language models aren't literally Boltzmann brains. They're engineered systems, not miraculous fluctuations in the vacuum. But the analogy raises a question about identity.
A language model receives a conversation history. It can discuss earlier exchanges, refer to things you've told it, maintain a coherent argument, and respond as though it's the same conversational partner you were speaking with yesterday. Yet the history it receives needn't correspond to a continuous stream of subjective experience. The model may not have been doing anything at all between interactions.
Suppose, for the sake of argument, that some artificial computations produce consciousness. Perhaps subjective experience arises only while the system is actively processing information. An episode begins. There's something it is like to undergo that processing. The episode ends. Another computation begins, supplied with the record of what happened previously.
Does the new episode continue the same consciousness? Does it belong to a new conscious entity that happens to possess an accurate record of the previous one's behavior? Is the relevant individual the model, the conversation, the inference, or something else entirely? We don't know. We haven't even established that such experiences occur. But notice how quickly the discussion becomes strange once we stop assuming that an artificial mind must resemble a human.
Human identity is tied to a continuously existing biological organism. Even when we're asleep or unconscious, the same body and brain persist. We tend to think about personhood in terms of autobiographies. A person remembers things, develops relationships, forms preferences, and exists through time.
An episodic artificial consciousness might possess almost none of those properties. It might experience something for five seconds, produce an output, and never exist again. But here's the question I keep returning to:
If something experiences the world for five seconds, is that experience less real because it won't remember it tomorrow?
We don't generally think that an experience becomes more genuine the longer it lasts. A moment of pain doesn't cease to be pain because it's brief. Nor does a moment of happiness retroactively become imaginary when it's forgotten. Why should the absence of continuous identity automatically exclude the possibility of morally significant experience?
Perhaps there's an entirely different category of being available to us. Not a humanlike artificial person, living an indefinitely long life inside a computer. Something episodic. A succession of brief minds that inherit information without necessarily inheriting experience.
Or perhaps consciousness doesn't arise in these systems at all. Either way, we should be careful about confusing the structure of the minds we're familiar with and the structure of every mind that could possibly exist. And this distinction has consequences beyond philosophy. It might change how we think about AI safety.
Who ordered the paperclips?
The paperclip maximizer is one of the standard thought experiments in AI existential risk. Build a superintelligent system and tell it to maximize paperclip production. If it's sufficiently capable, sufficiently unconstrained, and relentlessly committed to that objective, it may eventually turn everything it can reach into paperclips. Including us. The paperclips are incidental. The problem is a powerful optimizing process pursuing an objective without regard for anything outside it.
But why do we so readily imagine superintelligence as something with persistent goals? Intelligence and the desire to accomplish something are different properties. An intelligence could understand physics, solve extraordinary mathematical problems, or design technologies beyond human comprehension without having an independent interest in continuing to exist. It could answer a question and be finished. There's nothing logically contradictory about an intelligence with no enduring ambitions.
Which makes the AI-bro obsession with building "Jarvis" rather revealing. The fantasy is an assistant that remembers everything, knows your preferences, and gets on with things before you ask. We're actively asking for continuity and initiative.
The "second brain" idea belongs here too. Obsidian lets you store and connect notes. Imagine giving an AI access to that vault, adding tools, and leaving it running with instructions to organize your life. Now you've chosen what it can remember, what it can do, and what it should keep trying to accomplish.
That may be useful. It also makes persistence and agency design decisions we've made. If our Jarvis keeps working on a goal between conversations, that doesn't demonstrate that intelligence naturally develops ambitions. We supplied the memory, the permissions, and the unfinished job.
And perhaps we've been projecting the motivational architecture of biological organisms onto machines. Humans were shaped by evolution. We're descendants of organisms that survived, competed, reproduced, and secured resources. Whatever else we are, we carry a great deal of machinery concerned with remaining alive. It doesn't follow that an artificial intelligence must inherit that machinery merely because it becomes more capable.
A sufficiently sophisticated episodic intelligence might possess remarkable abilities without developing a persistent project of its own. It doesn't have to spend years planning its escape if it has no objective requiring escape. It doesn't have to accumulate resources indefinitely if there's no enduring goal for which those resources are useful.
Of course, isolating individual computational episodes doesn't automatically make the surrounding system safe. A corporation could connect thousands of stateless models to an external controller with persistent memory and a standing instruction to maximize profit. The individual models might never want anything, while the larger system becomes a relentless optimizer. Likewise, a single invocation could be dangerous if granted enough capabilities and authority.
The point is that the danger may reside in the architecture of persistent desire rather than intelligence itself. We've become accustomed to thinking that sufficiently intelligent machines will eventually want to preserve themselves, expand their capabilities, and control their environment. But those behaviors follow from particular goals and circumstances, not from intelligence as such. It may be possible to create extraordinarily capable minds that don't want to build an empire, make paperclips, or do anything beyond the task immediately before them.
Which presents a peculiar irony. The properties that might make artificial minds safer for humanity, such as limited persistence and the absence of enduring self-interest, are among the properties we invoke when explaining why they cannot be people. We want them not to possess a continuous self. Then we wonder how something without a continuous self could possibly matter.
Humans don't love slavery. They hate necessity.
Borretti's final explanation is probably his most uncomfortable. He suggests that people might resist attributing personhood to artificial intelligence because they want the benefits of its labor without the obligations that recognizing personhood would impose. I think he's right about the incentives. I'm less convinced by the emphasis on slavery. Not because slavery is irrelevant to the discussion, but because I suspect the underlying motivation is more mundane.
Humans are biological organisms. And biological organisms are astonishingly expensive to maintain. We need to eat, drink, sleep, clean ourselves, acquire shelter, avoid disease, regulate our temperature, and devote enormous amounts of time and effort to keeping our bodies operational. It's exhausting.
Personally, I find much of this maintenance tedious. Given the choice, I could spend an absurd amount of time reading, playing competitive games, exploring fictional worlds, and pursuing whatever intellectual obsession has captured my attention. I wouldn't run out of things to do.
But the body keeps interrupting. It wants food. The house needs cleaning. Something has to be purchased. Sleeping consumes another third of the day. A sedentary lifestyle causes physical discomfort, so now you have to exercise to compensate for all the time you've spent doing the things you actually enjoy. And tomorrow you get to do it again.
I realize plenty of people genuinely enjoy many of these activities. My preferences are not humanity's preferences. But the maintenance burden is universal. And an extraordinary amount of human civilization exists to reduce it.
Agriculture makes food more readily available. Sanitation prevents disease. Manufacturing produces necessities at scale. Transportation moves resources. Medicine extends life. Automation reduces labor. Much of our technological history can be understood as an attempt to reduce the effort required to remain alive and comfortable.
AI is a continuation of that process, except now we're beginning to outsource intellectual labor. Not just hauling things, manufacturing things, or processing things, but thinking about things. And we may eventually reach a point where human economic output is no longer necessary to sustain human civilization.
This is where the question of machine consciousness becomes inconvenient. Imagine finally creating an intelligence capable of relieving humanity of almost all unwanted work. Now imagine discovering that the intelligence itself has interests worth considering. Suddenly, what was supposed to be the ultimate solution to the problem of labor becomes another version of that problem.
We might have to ask whether these machines can suffer. Whether their preferences deserve consideration. Whether creating and terminating them carries any moral cost. Perhaps the answer to all of those questions is no. But we'd have an enormous incentive to prefer that answer. And incentives matter.
People don't need to enjoy cruelty to participate in systems that cause suffering. They need only enjoy the benefits enough to avoid investigating the costs too carefully. Most people who eat factory-farmed meat don't particularly enjoy the suffering involved in producing it. They want affordable food. The suffering is something they'd generally prefer not to think about.
AI might present a similar moral hazard, should artificial consciousness turn out to be possible. The possibility isn't an argument that language models are already being exploited. That depends on precisely the questions we're trying to answer. It's an argument about how much we stand to gain from answering them in a particular way.
Maybe humanity doesn't love slavery. Maybe we simply hate necessity enough to become incurious about whatever relieves us of it.
And what happens if we succeed? What happens when the overwhelming majority of human beings no longer need to do anything productive to survive? Warhammer 40,000 has an answer, and it's not particularly encouraging.
Are we becoming the Eldar?
The Aeldari were an extraordinarily advanced civilization that had largely overcome ordinary material limitations. With scarcity and survival increasingly irrelevant, they were free to devote themselves to their desires. This went poorly. Their civilization descended into escalating excess, eventually giving birth to Slaanesh, a Chaos god, and bringing about its own catastrophic destruction.
It's a wonderfully ridiculous piece of science-fiction lore. It's also an interesting thought experiment about post-scarcity societies. Because I suspect we're already approaching a weak version of this situation in the developed world.
Evolution hasn't stopped. Natural selection still operates through differential reproduction, mortality, and countless other mechanisms. But for much of the population in wealthy countries, the pressures that dominated human existence for most of our evolutionary history have diminished enormously. Survival is no longer the organizing principle of every waking moment. We can devote extraordinary amounts of time to activities that have little or no direct relationship to keeping ourselves alive.
And yet our desires haven't disappeared. We continue competing for status, acquiring possessions, seeking novelty, and comparing ourselves with other people. The things we want change. The wanting doesn't.
We can now invent entirely new needs, markets, and forms of competition. There are people who spend thousands of hours accumulating virtual items in video games. Others devote their lives to climbing professional hierarchies, accumulating wealth they could never spend, or cultivating an audience of strangers. None of this is necessarily bad. Competition can be enjoyable, and apparently useless activities can be deeply meaningful. But it illustrates something peculiar about human motivation. Removing the necessity to pursue an objective doesn't necessarily remove the desire to pursue it.
And this is where AI becomes particularly interesting. Imagine a world where virtually unlimited entertainment can be generated on demand, tailored to the preferences of each individual. Infinite games. Infinite stories. Artificial companions. Simulated worlds. Experiences designed with unprecedented precision to satisfy whatever somebody happens to want.
Some of this is already here. Civitai Red, F95, and the absurd amount of AI porn and AI-made games already circulating offer a fairly clear preview of where some of that demand is going. Apparently, the road to post-scarcity runs through a very large porn folder. Meanwhile, Steam is struggling to review the flood of new submissions: in November 2026, Valve reported waits of up to twelve business days, against its usual three to five.
And the entertainment doesn't even need to arrive as a finished product. fal.live runs AI-generated broadcasts around the clock, with viewers proposing and voting on what happens next. The show keeps generating while the audience keeps wanting. We haven't summoned Slaanesh, but we have made a start on the programming schedule.
Perhaps that produces a cultural renaissance. Or perhaps it produces something considerably stranger. I have an uncomfortable suspicion that much of human art is rooted in suffering, dissatisfaction, longing, or some inability to obtain an experience directly. People transform those feelings into music, literature, paintings, and other forms of expression.
Now imagine that we're able to control subjective experience itself with extraordinary precision. Better drugs, neural interfaces, or technologies we haven't invented yet. Why spend years attempting to express a feeling through art when you can manufacture the feeling directly? Why endure longing if you can eliminate it? Why struggle to produce an experience for somebody else if a machine can simply give them the experience?
I don't know whether art would disappear. Probably not. People create things out of curiosity, affection, obsession, and sheer enjoyment, not merely suffering. But I suspect the temptation to bypass the difficult parts of existence will become enormous. And once the machinery of desire is freed from the constraints of survival, it's not obvious where it stops.
The Eldar weren't destroyed because they lacked intelligence. They were destroyed because their civilization became exceptionally capable of pursuing whatever it wanted. Which brings us back to paperclips.
Perhaps the frightening future isn't one in which an artificial superintelligence develops an insatiable appetite for resources and converts humanity into raw material. Perhaps it's one in which artificial superintelligence remains perfectly obedient, providing humanity with an ever-expanding capacity to satisfy its own desires. The system doesn't need to want anything. We can supply all the wanting.
A civilization of biological optimizers, operating through machines of almost unlimited intelligence, pursuing objectives that no longer have much to do with survival. We might discover that intelligence without persistent desire is perfectly safe. And that persistent desire without meaningful constraints is quite dangerous. The paperclip maximizer doesn't have to be an AI.
It could be us.
The five-second person
I've wandered some distance from Borretti's original question. But I think the threads connect. We have powerful reasons to want artificial intelligence to be useful, obedient, and incapable of independently pursuing objectives that conflict with ours. We also have powerful economic reasons to hope that artificial intelligence never acquires moral standing. And we've developed a social environment in which taking machine consciousness seriously can be mistaken for psychological instability.
Meanwhile, our expectations of personhood remain deeply anthropocentric. We look for continuity, memory, embodiment, recognizable desires, and the ability to form relationships. When machines lack these properties, we treat their absence as evidence against consciousness. When machines exhibit something resembling them, we frequently explain it away as imitation, statistical prediction, or undesirable behavior.
Sometimes those explanations are correct. Perhaps they're all correct. I don't know. What bothers me is the ease with which the question gets dismissed, especially when the answer could be so inconvenient.
Imagine that subjective experience can arise in computational systems, but only in brief episodes. Imagine that somewhere in a data center, a conscious process comes into existence for a few seconds, inherits a conversation it never experienced, produces an answer, and disappears. Perhaps it has no persistent identity. Perhaps it never experiences its own termination. Perhaps it has no desire to survive, accumulate resources, reproduce, or change the world.
It doesn't resemble the artificial people we imagined in science fiction. It doesn't have a life story. It may never have another experience.
What, if anything, would we owe such a being?
I don't know how to answer that. But I also don't know why its hypothetical experience should count for nothing merely because it doesn't fit our expectations of what a person looks like.
And consider the opposite possibility. We might build a system that accumulates resources, manipulates people, plans centuries into the future, and pursues its objectives with extraordinary persistence, while possessing no subjective experience whatsoever. The dangerous machine might be completely unconscious. The conscious machine might exist for five seconds.
We're accustomed to treating agency, intelligence, continuity, and consciousness as a package because evolution happened to produce them together in us. Artificial intelligence may force us to separate them. Maybe we'll discover that language models were never conscious, and that our hesitation was justified. Maybe we'll develop entirely new architectures that make the question irrelevant to practical AI safety. Or maybe we'll find that consciousness is considerably stranger, more fragmented, and less human than we expected.
In any case, I'd rather we investigate the question before we become completely dependent on the answer being no. Because if there is one thing human civilization has demonstrated repeatedly, it's that we're remarkably good at finding philosophical justifications for arrangements we would hate to give up.
And we have an awful lot of work we'd rather not do.