Psychology / qualia and the inverted spectrum14–17 min

Your Red Is Not My Green. And I’m Not Sure Why Anyone Thought It Was.

Or: how philosophy turned individual variation in perception into a metaphysical crisis.

Different brains should produce different experiences. The mystery begins when ordinary variation gets promoted into private metaphysical paint.

QualiaInverted spectrumConsciousnessAI consciousnessColour perception
A laughing tomato annoys two exhausted researchers conducting an elaborate red-versus-green perception experiment
The tomato would like both sides to know that it has qualia, excellent timing and no intention of cooperating.

Different nervous systems should produce different experiences of the same colour. That ordinary variation does not establish an intrinsic, detachable red or green quale—and it cannot be used as an uninspectable admission test that excludes artificial systems from consciousness.

There are philosophical problems that become more interesting the longer I think about them. And then there are philosophical problems where the longer I think about them, the more I begin to suspect that somebody took something perfectly ordinary, described it in sufficiently mysterious language, and accidentally created an academic discipline around it.

Qualia have always struck me as belonging rather dangerously close to the second category.

For anyone who has managed to avoid the term, qualia are supposed to be the subjective qualities of experience: the redness of red, the painfulness of pain, the taste of coffee, the smell of rain. The famous question then becomes: how do I know that the red I experience is the same as the red you experience?

I don't.

In fact, I would be rather surprised if it were exactly the same. We have different eyes, different retinas, different brains, different histories, memories, associations and emotional responses. We aren't even in exactly the same physiological state from one minute to the next. So why on earth would I expect two different nervous systems to produce identical internal states in response to the same stimulus?

That isn't a mystery. That's Tuesday.

Why does anyone care about the tomato?

Before attacking the idea, however, it is worth explaining why philosophers care whether my red resembles yours. Otherwise this entire discussion sounds like an extraordinarily elaborate argument about interior decorating.

The point of the famous “inverted spectrum” thought experiment is not really colour. It is supposed to demonstrate something much more important about consciousness.

Imagine that you and I behave identically around colours. We both call tomatoes red and grass green. We distinguish the same shades, arrange colours according to similarity in the same way, learn the same colour words and perform identically in every colour experiment anyone devises. Nevertheless, suppose that the private experience I have when looking at red is what you experience when looking at green.

If that is genuinely possible, then apparently every functional and externally observable fact could remain unchanged while the experience itself changed. A description in terms of stimulus, neural processing, discrimination, information processing and behaviour would therefore have left something out: what the experience actually feels like. That supposedly irreducible subjective component is what qualia are intended to capture.

Now we can see why anyone cares. We can also see why the issue has escaped philosophy seminars and wandered straight into arguments about artificial intelligence. An AI might identify red, distinguish it from green, understand wavelengths and colour relationships, describe sunsets, associate red with danger or romance and perhaps even represent aspects of its own internal processing. Then somebody says, “Yes, but it doesn't have qualia.” All the information processing may be there, they argue, while the actual experience is absent. Therefore the AI isn't genuinely conscious.

So the humble red tomato has somehow become a gatekeeper for membership in the community of conscious beings.

Fine. Now I care.

And now I have questions.

Your red is almost certainly different from mine

Suppose you and I look at the same red object under the same illumination. Approximately the same spectral information reaches our eyes, but from there it enters two different biological systems. Your cone distributions aren't precisely mine. Your retinal processing isn't precisely mine. Your visual cortex isn't mine. Your attentional state isn't mine, and neither is your history with the colour red.

Perhaps red reminds me of Christmas. Perhaps it reminds you of blood. Perhaps our hypothetical friend was once attacked by a lunatic wearing a bright red suit and now his sympathetic nervous system prepares for Armageddon whenever somebody opens a tin of tomato soup.

His experience of red is going to be rather different from mine. Of course it is.

Notice what we do not need in order to explain this: a mysterious metaphysical object called THE RED QUALE. We already possess an enormous physical, neurological, psychological and autobiographical system capable of producing individual variation. Change the system and you change the resulting experience.

Why is anybody surprised?

But what if your red is my green?

This is where the thought experiment begins to lose me.

From the perfectly mundane proposition that our experiences of the same colour probably aren't identical, we somehow arrive at the claim that perhaps your red is literally my green. You look at a tomato and say “red.” I look at it and say “red.” But secretly, apparently, you are having the experience I would have while looking at grass; you simply learned to call that experience “red.”

Individual variation does not remotely imply that kind of systematic substitution. If two different systems A and B receive the same red stimulus R, I fully expect that:

E_A(R) ≠ E_B(R)

Different systems can produce somewhat different experiences.

But that does not give us:

E_A(R) = E_B(G)

Why should my response to red correspond specifically to your response to green? Why green? Why not blue? Why not the smell of cinnamon? Why not Tuesday?

The inverted-spectrum argument seems to imagine experiences as pieces of private paint that can be shuffled independently of the systems producing them. Your brain receives red, but somebody accidentally plugged the GREEN EXPERIENCE into the red socket. Fortunately, you learned the correct vocabulary, so nobody noticed.

I don't see why perception should work like that.

Where is this objective green supposed to exist?

The comparison also seems to assume that there is some standardized experience called GREEN, as though somewhere behind reality sits the International Bureau of Phenomenal Standards:

Official Red Experience: R-650 Official Green Experience: G-530 Please calibrate consciousness before use.

But there isn't an observer-independent “way green feels” floating around somewhere. There is your experience of green and my experience of green, produced by our respective systems interacting with stimuli we classify as green. Those experiences presumably differ for exactly the same reasons our experiences of red differ.

So when somebody asks whether my red feels like your green, I want to know what identity relation they are proposing. Does my red produce the same emotional associations as your green? Does it occupy the same position in perceptual similarity space? Does it produce comparable neural dynamics, attentional effects, memories or discrimination behaviour?

Excellent. Those are claims we can investigate.

But if every measurable, relational and causal property remains unchanged and I am told, “No, I mean the intrinsic greenness itself,” I begin wondering whether we have discovered something profound or merely invented a property specifically designed to escape investigation.

Colours do not exist psychologically as isolated blobs of subjective paint. Red occupies a place in an entire perceptual structure. It resembles orange more than blue. Purple relates differently to red than yellow does. Colours vary in saturation and brightness; context and adaptation alter their appearance.

If we supposedly swap red and green while every one of those relationships remains unchanged, what exactly did we swap?

If your “green-feeling red” relates to orange, purple, brightness, saturation and every other colour exactly as my red does, and produces the same discriminations, memories and behaviour, then in what meaningful sense is it green? We have removed everything that ordinarily makes green green and retained a mysterious residue whose defining feature is that nobody can characterize it.

That's an impressive trick. I'm not sure it's psychology.

Why does my red need to resemble yours anyway?

There is an even simpler question that tends to disappear beneath all this philosophical machinery: why does it matter whether our reds are the same?

Suppose tomorrow neuroscience invents a miraculous device capable of directly comparing subjective experiences. You and I climb inside and discover that my experience of red is substantially different from yours.

Fine. Which one of us has stopped being conscious?

Obviously neither.

Suppose the difference is enormous. Still neither. Suppose our entire colour spaces are organized somewhat differently. Still conscious. Suppose somebody is congenitally blind and has no visual experience of red whatsoever. Still conscious.

So whatever consciousness is, one thing seems obvious: having the same experiences is not a requirement for being conscious.

This is where I find the use of qualia against artificial consciousness particularly strange. We readily accept enormous experiential variation among beings whose consciousness we do not question, but the moment the candidate consciousness is artificial, human-style experience suddenly becomes an entrance examination.

Why?

The qualia trap in AI consciousness

One of the standard dismissals of artificial consciousness is: “AI can process information, but it doesn't have qualia.”

This sounds impressive until you ask two questions. First, how do you know? Second, what exactly do you mean by qualia?

If qualia simply means the subjective character of experience, then saying “AI doesn't have qualia” means “AI doesn't have subjective experience.” But whether AI has subjective experience is precisely the question under investigation.

The argument therefore becomes: AI isn't conscious because it doesn't have subjective experience, and we know it doesn't have subjective experience because it isn't conscious.

Marvelous. We've rediscovered the philosophical zombie wearing a graphics card.

Perhaps qualia instead means something more specific: human-like sensory experiences such as seeing red, tasting coffee or feeling warmth. Then the argument becomes even stranger. Why should consciousness require our particular sensory architecture?

A congenitally blind person doesn't experience visual red. Conscious. A person without a sense of smell doesn't experience the smell of coffee. Conscious. A bat uses echolocation. A bee perceives ultraviolet. A pit viper detects infrared. An octopus possesses a nervous system organized so differently from ours that comparisons become difficult remarkably quickly.

Are we going to require every potentially conscious being to pass the Standard Human Red Test?

Obviously not. So why should an artificial system have to?

There is no approved package consisting of Redness, Painfulness, Coffee-ness and Warmth that one installs before:

CONSCIOUSNESS = TRUE

If consciousness is possible in more than one particular kind of primate nervous system, different conscious architectures should be expected to have different kinds of subjective states. Not reluctantly tolerated. Expected.

An artificial intelligence might never experience anything remotely analogous to human colour. Why should it? Its architecture, inputs, representational spaces, memory and temporal dynamics are different. Perhaps distinctions among concepts, images, uncertainty, predictions, internal representations and memory have some subjective character for such a system that corresponds to no human sensory category whatsoever.

I don't know.

Neither does the person confidently announcing that it has no qualia.

Asking whether such a system experiences red like I do may eventually turn out to be rather like asking whether a dolphin's sonar smells sufficiently like cinnamon.

It may simply be the wrong coordinate system.

Qualia cannot be both private and an admission test

There is another problem here that borders on comedy.

Qualia are supposedly private. I cannot directly inspect yours and you cannot directly inspect mine. That is the entire reason we started talking about whether your red might differ from my red. Subjective experience is apparently so inaccessible that two humans standing next to one another, looking at the same tomato, can never establish whether their experiences match.

Fine.

Then somebody points at an artificial intelligence and confidently announces: “It doesn't have qualia.”

Excuse me?

We have just spent an enormous amount of philosophical energy establishing that qualia cannot be externally inspected. Now apparently we can diagnose their complete absence in a radically different cognitive architecture from across the room.

You cannot have this both ways.

If qualia are so private that I cannot establish whether your red resembles mine, then their absence cannot casually become an externally observable criterion by which we exclude other systems from consciousness. We need an independent theory connecting physical or organizational properties to experience.

Once we have such a theory, that is what we should be discussing. Not waving the word qualia around like a philosophical access badge.

Pain doesn't rescue the argument

Pain is another favourite example. Apparently there is something mysterious called the painfulness of pain left over after we describe the neural processing, behaviour, attention, aversion, learning, bodily response and cognitive consequences.

Perhaps. But again I want to know what we are subtracting.

Pain reorganizes cognition. It captures attention, motivates escape, changes memory, alters future decisions, produces autonomic responses and changes expectations. It becomes associated with contexts and cues. Attention, expectation, placebo, fear, meaning and previous experience can all modify it.

That is an enormously rich system state.

Then philosophy sometimes seems to say, “Yes, yes, all that. But why does it hurt?”

I understand the intuition. I really do. But I am not convinced that giving the remaining intuition a noun—quale—has explained anything.

Sometimes naming the mystery merely makes the mystery sound like an object.

The private cinema in your skull

I suspect part of the confusion comes from an intuitive model of perception that neuroscience has spent a long time dismantling. We imagine something like:

world → eyes → brain → internal picture → somebody experiences the picture

The Homunculus Cinema.

Red light enters. The brain processes it. Somewhere inside your head, RED appears on a screen and consciousness watches it.

But then we have merely moved perception one level inward. Who watches that screen? Does that observer have another internal screen? Who watches that one?

At some point we need to stop putting observers inside observers and recognize that the processing itself constitutes the perceptual state of the organism. There needn't be a second internal object called the experience of the processing presented to a metaphysical spectator.

Different does not mean mysterious

I absolutely believe your experience of red differs from mine. I'd be astonished if it didn't. But that doesn't frighten me metaphysically.

Your experience of Mozart differs from mine. Your experience of your mother's voice differs from mine. Your experience of the smell of your childhood home differs from mine. Your experience of a particular word differs from mine because your semantic network differs from mine.

Of course subjective experience is subjective. It occurs in a subject.

Different subjects have different histories and different physical organizations. Individual variation is therefore exactly what we should expect.

The philosophical mistake, in my view, is turning “we cannot directly compare two people's internal states” into “therefore there exists an ineffable metaphysical ingredient inaccessible to physical explanation.”

That conclusion does not follow.

And an additional mistake occurs when “another system's subjective states may be radically unlike ours” becomes “therefore that system has no subjective states.”

That doesn't follow either.

Different experience is not absence of experience.

The question worth asking

I don't find “Is my red really the same as your red?” particularly useful. Almost certainly not exactly. Fine.

I am interested in how a system constructs colour experience; which physical and informational relationships determine its structure; how changing the system changes the experience; how perception interacts with memory, emotion and prediction; which organizational properties distinguish conscious from unconscious processing; and whether those properties can occur in architectures radically different from our own.

Those questions can generate experiments. They can produce models. They can be wrong.

Excellent. That's science.

By contrast, “Perhaps your red is secretly my green, but by definition nothing could ever reveal the difference” may be an entertaining thought experiment. But if neither behaviour, neural organization, discrimination, memory, association, introspection nor any possible measurement can tell us what my green means in that sentence, I reserve the right to ask whether we have uncovered a fundamental property of consciousness or merely constructed a sentence with nowhere left to land.

Perhaps qualia is useful shorthand for the subjective character of experience. Fine. Words are allowed.

My objection begins when the shorthand quietly becomes an ontology, and it becomes considerably stronger when that ontology becomes a gatekeeping criterion for other minds.

We start with the unremarkable observation that experiences have subjective character. Then somehow we arrive at the claim that experience therefore contains private, intrinsic, irreducible phenomenal properties whose presence is necessary for consciousness.

Slow down. Quite a lot has been added between those sentences.

Maybe some of it will ultimately prove correct. But it needs to be demonstrated, not smuggled in through vocabulary.

I don't need a universal REDNESS to explain why red looks like something to me. I need to understand the system doing the looking.

And if your red differs from mine? Of course it does. You're not me.

That's not the Hard Problem of Consciousness. That's the extremely easy problem of having two different brains.

As for the possibility that my red is literally your green, I have only one remaining question:

Which green? Yours or mine?

Because apparently we've already established that those aren't the same either.

At which point I suggest we stop tormenting the tomato and have lunch.

Source notes

The external sources document the qualia and inverted-spectrum arguments. The criticism developed in this article—and the tomato's increasingly justified frustration—is my own.