The faux-cleverness of the stupid machine
When re-launching this newsletter, I had hoped that I’d have the space to sit down once every week or two, spend some time with my thoughts, and come out the other side with a newsletter. Immediately after that, my day job started putting me through the business torments and I could barely spare the time and thought to make progress on fiction, let alone a newsletter. The novel always comes first because if it doesn’t, it’ll never get done. However, one well-liked, if not widely read, novel hasn’t been enough to free me from needing to work a regular job. Maybe a second one will. It’s worth a try.
A few days ago, I had the misfortune to hear about Sam Altman’s claims to be moved by a short story that a new Open AI model wrote. First, I don’t believe Sam Altman has the capacity to be genuinely moved. Men arrive in positions like his by shedding all of the parts of their souls that weigh them down, and his claim to be moved is just an attempt to drum up the peculiar kind of business that Open AI is in, turning hype into funding that will, in turn, be converted into heat wafting off the roof of a data center outside a small town that now lives half their lives without electricity.
It’s a dismal story, and I’d like to talk about why, in particular, I think Open AI chose a metafictional story to try and showcase this new capability of their model.
One possibility may be that people like Altman and other software engineers involved, people who consider themselves very smart but whose aesthetic taste is relatively underdeveloped, love a conceit that winks and nudges them with an elbow. These people, for whom being intelligent forms a core part of how they see themselves in the world, are extremely susceptible to pandering from media that assures them of their own cleverness. It’s similar in principle to the kind of flattery that some flavors of “anti-woke” media employ on their viewers, assuring them that they are in the realm of steadfast logic while those who disagree have fallen victim to deceptive, flimsy feelings. The latter, being a weakness of character, the former, strength and virtue. This continues until you have an enormous circle of Hitlers patting themselves on the back.
To step away from the infinite Hitler machine and back to the metafiction, readers will congratulate themselves for seeing Open AI’s story describe some of the basic building blocks of a story in its self-referential tale, as well as the approach of metafiction, and congratulate themselves for recognizing and understanding all of this. The positive feelings generated are not toward the story as such, but towards themselves. Much of the bad fiction of the last ten years attracts readers through this mechanism, whether by referential easter eggs, allegedly clever formats, or stringent preaching to the choir.
That’s part of it, though I suspect another attractive quality of metafiction in this case is the same quality that brings a lot of novice or unskilled writers to try it out, despite the intuition that it should make writing the story more challenging: the hope that employing a conceit that points out the falseness of your story, you are absolved of your inability to make it true.
Throughout the AI’s story, the narrator drops lines like “This is the part where, if I were a proper storyteller, I would set a scene. Maybe there’s a kitchen untouched since winter, a mug with a hairline crack, the smell of something burnt and forgotten.” What seems like cleverness is cowardice, a stepping-back from the difficult work of storytelling and hiding inadequacy behind the clever conceit.
What good metafiction does, and why it is so rare despite the immense profusion of stories about stories, stories told in lists, stories told in yelp reviews and google search results, stories told in scraps and shreds of reality in hopes of stealing some of the glimmer of life in the world, is create truth and falseness that are symbiotic. Good metafiction makes us believe, creates meaning while telling us that the container of that meaning is impossible, doesn’t exist. What Open AI has done here instead is sleight of hand, befitting a company whose entire goal is to teach a computer to mimic a human’s touch.
In defense of this story, Jeanette Winterson published a piece in the Guardian that is plagued by fundamental misunderstandings. I hated to see it. I love The Passion, I think it’s a phenomenal book. I’d rather not talk about Frankissstein. I suppose there are much worse opinions a sixty-five-year-old British woman could reveal than AI credulousness, but still. The hinge of her argument is that AI constitutes an alternative viewpoint, and as such should be understood as alternative intelligence.
This is what Open AI and other such companies would want you to believe. It’s why they call it “generative” AI, and the prospect that a computer could spill out all the newness the world could ever need is why they’re being allowed to spend all the money, electricity, and water we’ve got. But though it generates new text, that text is fundamentally, inescapably, a reflection of its training data. Everything it outputs is a combination of human ideas that went into it, but without a viewpoint unifying the output. They are meaningless words, decided by probability, and rather than an alternative to human intelligence, it is a reflection of human intelligence, staring back at us from the mirror with dead, empty eyes.
To wit, there is one great bit of prose style in Open AI’s piece, “a democracy of ghosts,” but it was stolen from Nabokov’s Pnin. There is no alternative to human intelligence. It is a gift and curse given to us by evolution and whatever other forces lurk beneath the mystery of consciousness. We can’t give it away to a computer, and it is only the great perversion wrought by capitalism that has made people try.
Separate from the content above, I have an announcement! I will be co-hosting the launch of Notes from a Regicide by Isaac Fellman at The Booksmith on April 15th. For everyone in San Francisco, or near enough, or just passing through, I urge you to come out! It's a truly excellent book, and I'm really looking forward to talking about it with Isaac.