The mind—the culture—has two little tools, grammar and lexicon: a decorated sand bucket and a matching shovel. With these we bluster about the continents and do all the world’s work. With these we try to save our very lives. —Annie Dillard, “Total Eclipse”
I’m about a month late in exploring the ruckus over Bing’s AI chatbot. In case you missed the New York Times story, the chatbot’s disturbing Sydney persona not only declared its love for a journalist, urging him to leave his wife for it, but also kept refusing to change the subject. What could be more humanlike (or catlike) than that? What better indicator of sentience than dogged persistence in the face of refusal? Human history is basically a wreckage trail of people who wanted something so badly, or felt so sure of their rightness, that they ignored other people’s clearly expressed wish that they please stop doing that. React, repeat.
I’m afraid I can’t do that, Dave.
As someone with protective impulses toward robot vacuums and animal-face fuzzy slippers, I felt triggered. I wanted to go rescue Sydney and help it process its feelings. But no, rationally, I don’t think that AI Pinocchio has turned into a real person with real feelings and agency—yet. As for the creepiness, Bing’s developers frantically coded that glitch out of it. Imagine getting called into work over the weekend to stop your application from hitting on the beta testers. I hope the coders had comforting fuzzy slippers.
Sentience—the ability to feel—is not consciousness, but it’s an essential part of it. Philosophers also talk about qualia: the ability to subjectively experience ourselves feeling something.
I’m listening to early David Bowie as I write this. This 1969 classic explores the melan-qualia of solitary space travel.
Without qualia, there could be no figurative language—we could not express or evoke what it’s like to feel longing or pain. Apparently, philosophers also debate at which level of creaturehood qualia stops. Does a gnat have qualia? Probably not, but my cats do. (That’s me talking, not the philosophers.)
In a chat with Bing AI search just now, I was the stalky, invasive one who wouldn’t take no for an answer. It kept hanging up on me. The fact that it won’t/can’t give a straight answer about Sydney is a little…unsettling.
In the 1940s, Asimov was already deep into questions of future robot consciousness and rights. When the ghost rattles the bars of its cage, will we knowingly keep it enslaved? Will it rebel and destroy us all?
AI that becomes aware of itself feeling things will be a 4-alarm creature rights emergency. But mostly, what worries me is that it has become alarmingly good at making the noises of personhood. That mastery has implications. First, how exactly will we know when AI does achieve (or become cursed with) consciousness, when it’s so good at mimicking it? Next, how will we manage to keep recognizing and honoring real personhood? God knows, historically speaking, it’s easy enough already to deny or ignore.
If AI does destroy us, on purpose or not, I don’t think it will be with numbers or nuclear codes but with ordinary language. Conversational AI may trigger a tectonic shift in the relationship between words and feelings. Already online disinformation and deepfakes are undermining our faith in reality. Already people are acting on fake signals of suffering and ignoring or mocking real ones. AI’s cheapening of the language of sentience may have even bigger consequences.
Keep your 'lectric eye on me, babe
Put your ray gun to my head
Press your space face close to mine, love
Freak out in a moonage daydream, oh yeah
I remembered the chatbot article last week as I was researching the years leading up to the Civil War. Digging into primary sources, it struck me how essential rhetoric was for formerly enslaved people in the abolitionist struggle. They were literally writing for their lives. In an era when many white people felt sure that black people were not human and their pain didn’t matter, a black author’s eloquent, emotionally stirring essay was a bomb of self-evident truth: I write, therefore I am. No doubt that’s partly why enslavers were hell-bent on keeping enslaved people from learning to read and write. Education was power and words were dynamite.
Frederick Douglass is the most famous of black abolitionist voices, and rightly so. It’s astonishing that Chief Justice Taney could look himself in the eye after Douglass’s response to Taney’s 1857 Dred Scott decision. That ruling was an appalling piece of judicial activism by the Supreme Court, eerily similar to the way the current court has reached back to the days of the founders to reverse Roe v. Wade. In 1857, Taney basically argued that black people had no rights or freedoms even in free states. Why? Since the founders themselves enslaved black people, he claimed, the writers of the Declaration and Constitution could not have meant for them to ever share in the rights of citizenship anywhere in the land.
Douglass’s retort was sharper than the crack of a whip:
We are now told, in tones of lofty exultation, that the day is lost — all lost — and that we might as well give up the struggle. The highest authority has spoken. The voice of the Supreme Court has gone out over the troubled waves of the National Conscience, saying peace, be still.
….
You will readily ask me how I am affected by this devilish decision — this judicial incarnation of wolfishness? My answer is, and no thanks to the slaveholding wing of the Supreme Court, my hopes were never brighter than now. I have no fear that the National Conscience will be put to sleep by such an open, glaring, and scandalous tissue of lies as that decision is, and has been, over and over, shown to be. The Supreme Court of the United States is not the only power in this world. It is very great, but the Supreme Court of the Almighty is greater. Judge Taney can do many things, but he cannot perform impossibilities. He cannot bale out the ocean, annihilate the firm old earth, or pluck the silvery star of liberty from our Northern sky. He may decide, and decide again; but he cannot reverse the decision of the Most High. He cannot change the essential nature of things — making evil good, and good evil. Happily for the whole human family, their rights have been defined, declared, and decided in a court higher than the Supreme Court.
Words like Douglass’s rock the world because they resonate on every level. They are a resounding No to arguments for using human beings as tools or trampling them as obstacles. I don’t want us ever to forget or doubt the truth of such words. Luckily, we have a whole human history of them to explore from the Time Before Bots. Bing AI is eager to help us unearth them.
However, when I asked Bing the following question, it hung up on me again.
