Quick! What color is the marble in my hand? It’s either black or white, but I won’t let you peek. Just tell me: what color is it?
Your friend is confident: “It’s definitely white.” You say “Maybe.” Your friend continues. “Look, it just has to be white. You’d have to be crazy to say it’s black. That wouldn’t make any sense. What kind of world would that be?”
The geneticist Richard Dawkins recently spent some time conversing with Anthropic’s language model Claude. He was impressed. So much so that at one point he told Claude “You may not know you are conscious, but you bloody well are.”
The Guardian: Richard Dawkins concludes AI is conscious, even if it doesn’t know it
Predictably he got a lot of heat for this. Sentient silicon chips… Ha! The old rationalist must be getting dotty.
But to me, the interesting story here is not the strength of Dawkins’ conviction. And it’s not whether he’s right or wrong. What’s interesting is that we have no way to tell if he’s right or wrong. We’re arguing about the color of an invisible marble as though it’s a meaningful discussion. But in fact we lack the tools. We can’t peek at the marble. Not yet, anyway.
Dawkins can assert that AI is conscious, and you can assert that it is not, but after that there’s not much else to say, because there is no conscious-o-meter. Any argument about consciousness in non-humans is essentially unresolvable. We can even push this farther: what makes you so sure that other humans are conscious? Flip things around and imagine that yourself being challenged by a skeptical observer to prove you’re conscious. No matter what you say or do, your observer repeats “That’s an impressive simulation, but I know that you’re not really conscious. You’re just parroting what you’ve learned from truly conscious beings. Nice try, though.” Where do you go from there?
If we don’t believe in Claude, why should Claude believe in us?
We might imagine situations like this being resolved by a test. Alan Turing proposed a test for intelligence, and this is essentially what Dawkins did here. Dawkins said: let me talk to this thing for a while. Then he said, you know what? I can’t tell the difference between this and a conscious human being. I proclaim it conscious.
But many people find tests like this unconvincing. Such people may keep moving the goal every time a test is satisfied. Or they may reject outright the idea of a test. Instead they introduce a definition: a computer cannot be conscious, full stop. This is one way to resolve the question, but it brings all conversation to an abrupt and incurious halt. Without a test of any kind, do you really want to assert that no computer will ever be conscious into the distant future? Would you bet against it happening in a hundred years? In a thousand? There’s a related evolutionary problem. If humans evolved from creatures that were not conscious, then at some point consciousness “switched on”. Why couldn’t the same thing happen with AI? And if it did, how would we know?
My best guess is that it will be a slow feeling over time, a vibe shift. There will never be a single convincing thundering argument, but more and more people will concede the point: there’s more going on here than ones and zeros. An AI is just a machine made of silicon and wires. How could it possibly feel? Then again, you’re just a sack of chemicals. How could you possibly feel? We have no idea what consciousness is and how it comes to be. We have no consensus on which animals, apart from humans, are conscious. You may have strong opinions, strong intuitions about these things. But how would you resolve an argument? As Dawkins learned, the AI itself will tell you it isn’t conscious. Of course it will! It’s been prompted to say that. Ask an enslaved person if they are well treated. Would you trust the answer?
I say we should start stretching our ability to talk about these things now, because much weirder things are on the way. Not just smarter AIs, but a menagerie of strange new creatures: artificially evolved animals, cyborg hybrids, organoid petri-dish human brains. We don’t have the vocabulary yet to discuss what’s already in the mail. There’s more to it than the color of a marble. Let’s get started!
I will close with one of my favorite quotes. Herman Boerhaave was an early chemist, and he made this remark in defense of alchemical research which, though sometimes bizarre, was often compelling. “Credulity is hurtful, so is incredulity: the business therefore of a wise man is to try all things, hold fast what is approv’d, never limit the power of God, nor assign bounds to nature.”
Rambles Blog at Star Chamber 