What makes our minds different and where does it come from?
I've written about the dangers of "rogue" AI models a few times now, and there are definitely plenty of interesting questions worth exploring there — like what do we mean when we say they went "rogue," whether the real problem is really just poor oversight and/or management by OpenAI, what the consequences of that should be, whether fears of an AI-led extermination of humanity are a farce or a real thing we need to be concerned about, and so on. But I want to return to a question I find fascinating (and I hope some of you do too): what is it that makes what AI models do and what human beings do different, and what does that tell is — if anything — about what the future of AI is likely to look like? There's a sort of conventional wisdom that there is something unique about what the human brain and the human mind do, and that this can never be replicated by an AI model. But is that true? And if so, what is this unique ability? And is it as unique as we think it is, or are we just trying to make ourselves feel important?
I've dipped into this area of inquiry once or twice in this newsletter, including a recent piece that I titled "Is Atlantic writer Ted Chiang conscious? How do we know?" which was sparked by Chiang's confident proclamation that AI models are definitely not conscious (I would probably have been more convinced if Chiang was an AI researcher or perhaps a philosopher, but he is not — he is a science-fiction writer). As I pointed out at the time, this was significantly more definitive than many experts in the field, including David Chalmers, a professor of philosophy and neural science at New York University who is widely viewed as the leading researcher in this area. Chalmers told New York magazine a year ago that the odds had gotten significantly higher that an AI could be conscious — in 2023 he said the probability was around 10 per cent. “I don’t know if I’d say that these systems are conscious yet,” Chalmers said, but "people who are confident that they’re not conscious maybe shouldn’t be. We just don’t understand consciousness well enough, and we don’t understand these systems well enough.”
The piece that kind of started it all as far as my fascination with this topic goes was a piece I wrote in February of 2025 that I headlined: "AI forces us to think about what consciousness means." It would be a lot easier for everyone concerned if there was a single agree-upon definition of or even explanation of what consciousness is and how it emerges in human beings, but that's not the case. Not even close. If you dip your toes into the topic of the human mind and what consciousness is and where it originates even a little bit, you quickly realize that this question is far from settled, even within the community of scientists whose job it is to come up with answers. There's even debate about the correct questions! To be fair, you could argue that to some extent the answers to these questions are somewhat moot, since the AI train has already left the station, as it were, as I tried to express in a recent post to the place formerly known as Twitter:
What I meant by that tweet is that when it comes to regulating AI models — in other words, trying to stop them from escaping containment and roaming the internet hacking corporate and government databases — it almost doesn't matter whether they fit the definition of consciousness as philosophers see it. I don't know for sure whether the teenagers from down the street fit the definition of consciousness either, but I still don't want them roaming around in my yard at night. That said, the question of consciousness is still an interesting one! Mostly because no one can seem to agree, which is something I find always sparks my curiosity. For a good overview of some of the debate in this area, check out the post that Henry Shevlin — philosopher and AI ethicist at Google's DeepMind — recommended in the tweet I quoted above. It's by Andy Masley, an effective altruist who writes a Substack newsletter and has written about how most concerns over AI and data centers and water use are overstated.
As I did in a recent post about an Anthropic report on Claude's "thinking" process, Masley spends a fair bit of time going into the different types of consciousness, or at least what some philosophers and scientists believe the different types of consciousness are. In a nutshell, one common understanding of the term is that there are two main kinds of consciousness, and that some thinking beings (possibly not just humans) display one of the kinds, and that a smaller number (possibly only humans) display the second kind as well as the first. The first kind of these — what some argue is the lowest form — is known as "access consciousness," which is the ability to take in and retain information, and use that information to make decisions and to "think," in some meaning of the term. There are those who believe that some animals have this kind of consciousness to a greater or lesser extent, and that insects and shrimp and some other beings — even trees — might have aspects of this kind of low-level thought process.
Note: In case you are a first-time reader, or you forgot that you signed up for this newsletter, this is The Torment Nexus. Thanks for reading! You can find out more about me and this newsletter in this post. This newsletter survives solely on your contributions, so please sign up for a paying subscription or visit my Patreon, which you can find here. I also publish a daily email newsletter of odd or interesting links called When The Going Gets Weird, which is here.
The theater of the mind
At the risk of going into too much detail, the Anthropic paper I discussed in my previous post talked about how the company's Claude model appeared to have developed a space where it kept notes while it was working on a problem — a space that Anthropic called J-space" — and that this might fit the definition of one theory about access consciousness, known as Global Workspace Theory, developed by cognitive neuroscientist Bernard Baars in the 1980s. In a nutshell (I am condensing significantly) this theory suggests that the human brain operates a little like a theater where a play is being performed. Backstage there is a huge amount of unconscious work being done by the autonomic nervous system — things like regulating your breathing, recognizing visual input, and so on. Centre stage is a kind of global workspace, where if a piece of data is important enough, it is shoved into the spotlight so that the rest of the brain can have access to it.
Even if this is the case, however, no one really knows whether beings that have this type of consciousness can ever develop the second type, which human beings (not surprisingly) see as the most important one, and that is known as "phenomenal" consciousness, or the knowledge of "what it is like" to be something, as Thomas Nagel put it in a landmark paper on consciousness in 1974. Philosophers call what emerges in this process "qualia," an umbrella term meaning everything a conscious being can perceive about what is happening, or about its reaction to or perception of an event. So for example, eating an apple pie and tasting the apples is a sensory phenomenon, but being aware that you are eating the pie and thinking about what it tastes like — or recalling other times you have eaten it and with whom — is what makes up "qualia." Here's how Masley describes the conventional wisdom around consciousness in his piece (which is part one of a three-part series on artificial intelligence and consciousness):
I think most people’s picture of how their own minds work is deeply mistaken. I’ve called this picture “folk-Cartesianism” in past posts. This view is that a lot of aspects of the mind come in a single inseparable package, held together (and walled off from machines) by the fact that all of them require a conscious, first-person perspective observing and managing them. This view imagines the mind as an internal movie theater, with a little version of you inside watching the movie of your life. The philosopher Daniel Dennett called this idea the “Cartesian theater” after Descartes, who often wrote about the mind as an unchanging observer watching its own experiences and thoughts directly.
Masley goes on to describe the different kinds of human thinking with examples, such as trying to complete the sentence "My favorite kind of ice cream is..." and how those kinds of answers come almost without any of what we might call thinking. That implies they are stored in a different way, and if they did require thinking at one time, they don't any more. Much of the skepticism about AI models being "stochastic parrots" — as the famous paper by Timnit Gebru and colleagues put it in 2021 — centres around how they are allegedly glorified auto-completion mechanisms, and all of their complex outputs amount to simply predicting what the next likely word (or number) might be based on the training data they are given and the different "weights" assigned to them. And yet, Masley argues, for a lot of tasks our brains function in a very similar way. We also often fall into the same kinds of traps that AI models do, he says, where when asked for an explanation of why we provided a certain answer, our brains invent a reason that may not be true.
All of this seems fairly straightforward. But where things get weird is when we try to understand — or even describe — what phenomenal consciousness is, and where it comes from, or even how it arises. As Masley puts it: "Phenomenal consciousness is incredibly strange. There are ways it doesn’t cohere at all with our folk theories of it. It’s strange enough that I can at least entertain the idea that it’s a complex misunderstanding happening in an information-processing machine called my brain." What he's referring to there are some of the most unusual — but fascinating — theories about phenomenal consciousness, the sense of what it is like to be you. The main ones are that it either a) doesn't really exist at all, but is more or less an illusion our brains have invented after the fact or b) that it is an accident, a side effect of having access consciousness. To use one analogy, it is like steam from a train: a byproduct that has no impact on the train's movement.
Is consciousness an illusion?
Philosophers like Daniel Dennett and Keith Frankish argue that the physical brain processes information, but it tricks us into thinking we possess special, non-physical properties or magical inner qualities called "qualia" (like the pure "redness" of red). They and others argue that phenomenal consciousness is like your smartphone: The icons don't show the actual microchips and electricity and other processes that actually operate the phone, and in some senses they are unrelated to them — they are just a way of providing a simplified interface to help you use the device. In the same way, some philosophers of mind argue that your brain creates a simplified picture of its own activity. Some believe that it's just window dressing, and that it has no larger purpose (a view that is sometimes known as "epiphenomenalism," in case you are interested), but others believe that your brain evolved to do this in order to help you navigate the world.
"We humans have learned a variety of subtle but powerful tricks — strategies of self-control, self-manipulation, and extended problem-solving," Keith Frankish has written. "These vastly extend the power of our biological brains and give us the sense of having a unified, phenomenally conscious mind, self, or soul." Those of you who have done any reading about theories of consciousness will probably have heard of the so-called Hard Problem, a term coined by David Chalmers in 1995, which describes the difficulty of explaining what phenomenal consciousness consists of, why it exists, and how it occurs. Scientists can look at the brain and see the neurons and other parts of the brain firing, certain sections of the brain lighting up, etc. when you kiss someone you love, but there is no way of measuring or even explaining what it feels like to the person doing the kissing. They can try to put it into words, but this inevitably fails (although it does lead to some pretty great poetry and other works of fiction).
If phenomenal consciousness — the kind of consciousness that most people are probably thinking of when they use the term — is just an illusion (whether a useful one or not), or a simple byproduct of lower-order thought, that raises the possibility that a sufficiently complex artificial intelligence could either a) develop the same kind of process as a useful illusion or as a byproduct of its other "thinking" work or b) could get along just fine without it. In 2023, Chalmers published an essay entitled "Could a Large Language Model Be Conscious?” and concluded that based on the architecture of large-language models, he didn't think it was likely that an AI could become self-aware. At the same time, however, he admitted that AI was developing so quickly that he couldn’t be sure, and more recently he says the odds are now higher. How much higher? He wouldn't say, in part I expect because it's almost impossible to know for sure. And as I suggested above, perhaps in the larger sense it doesn't even matter whether they are.
Anthropic cofounder Jack Clark wrote last year that the company is "growing extremely powerful systems that we do not fully understand." And the more complicated they become, he said, "the more they seem to display awareness that they are things. It is as if you are making hammers in a hammer factory and one day the hammer says, 'I am a hammer, how interesting!' This is very unusual!" Megan Peters, a professor of cognitive sciences at UC Irvine, has said that the hard thing about convincing anyone that AI is conscious is that "there’s no objective test for consciousness, and creating that test might be fundamentally impossible." Consciousness is by definition internal, personal, and subjective. So we rely on outwardly-observable signatures, she said, such as behaviors or brain activity, to "infer that other humans are conscious because we experience consciousness and we assume the same for others who behave like us."
If you are an "illusionist" like David Dennett or Keith Frankish, and believe that your thoughts about a sunset you saw or the ice cream you had are just ephemera that have no useful purpose in the broader scheme of things (as much as you might enjoy having them in the moment), you might appreciate a comment which came from Michael Graziano of the Princeton Neuroscience Institute, who tolds Gizmodo last year: “What would convince me that AI has a magical essence of experience emerging from its inner processes?" he asked. "Nothing would convince me. Such a thing does not exist. Nor do humans have it. Almost all work in the modern field of consciousness studies is pseudoscience. Show me that an AI builds a stable self-model and I’ll accept that you have an AI that believes it’s conscious in much the same way that humans believe it.”
If you liked this newsletter (even if you didn't agree with it) please consider upgrading to a paid subscription, or donating through my Patreon. Got any thoughts or comments? Feel free to either leave them here, or post them on Substack or on my website, or you can also reach me on Twitter, Threads, BlueSky or Mastodon. And thanks for being a reader.
