Our first essay was a cautionary tale—an injection of seriousness into the discussion of technology “changing everything.” A reminder that the change generated by new technology does not automatically go well. In the long run, the benefits of the new technology could be enormous. In the short run the results can be terrifying.
The last essay reminded us, additionally, that there can be huge lead times in mustering the necessary response to technological change. Because social change has enormous momentum, if one wants to direct society towards a desired goal, one has to plan in advance. Sometimes long in advance. The moment when our metaphorical railroad switch must be set correctly may be years or even decades in advance of the eventual destination; but if the train of society whooshes past the switch, past the plastic moment, disaster may become inevitable.
A Better Clara
In such a situation an engineer might decide that they can better serve society by abandoning engineering and turning to philosophy, or even politics. When we read about the argument between Fritz Haber and his wife Clara Immerwahr, we might have wished that Clara had been willing to choose leadership rather than suicide.
I started this project with the notion that I wanted “a better Clara”—one with more purpose, clearer, more articulate, and more self-assured. However, at the time I started, I didn’t understand exactly what that would mean. What more would we have expected of Clara that we could use as a guide to understand what is expected of us now?
Technology Goes Boom
Having perhaps learned from Haber’s grim example, in more recent times engineers have occasionally organized as activists. For example, Einstein, urged on by Leó Szilárd, famously signed a letter to Truman about the possibility of building an atomic bomb. After the bomb was built, physicists organized to advocate for its responsible use. Most activism by engineers has been either warning about, or advocating proper management of, things that dangerously “go boom.”
We are, as of recently, seeing the beginning of an engineer’s activism surrounding the management of new artificial intelligence technologies. It is falling into the well-worn tradition of warning about things that might “go boom” (in a generalized sense).
However, the most dangerous thing about artificial intelligence and digital technology right now might not be its physical risks. Unlike ammonia, it does not literally explode. Our fears are fuzzier. Perhaps we are most at risk from our own naïve enthusiasms. Like the men who joyously enlisted in World War I’s armies dreaming of the “sweetness and honor,” we might be most at risk not from the technology itself, but from the perverse ways we will throw ourselves into its use. Later generations will shake their heads in wonder at our suicidal stupidity. But they won’t realize how their easy judgments are only made possible by bitterly hard-won experience.
This book will create a framework to discuss what a “1914-like” disaster (in a generalized sense) would be for information technology. While that was a motivating question for me, and important for understanding the medium-future that sets the present in context, it is not actually the most immediately pressing question for our own time. Why not? As a foretaste, the disaster combined a perverse passion for the “sweetness and honor” of dying for one’s country with new weapons making such death all too easy.
What we have in our society today is only the first half of that cocktail. We have reincarnated a perverse passion. The great danger doesn’t come from our new technology itself—it comes from our dangerously misplaced enthusiasm for it. Our technology is not sufficiently advanced to be seriously dangerous. Yet. We don’t need to fix the technology, not now. Right now what we urgently need to fix is our heads.
Engineer Activism
The form of engineers’ activism is driven by their perennial discomfort with anything that seems political or partisan. (This discomfort persists despite the persistent recurrence of the need for activism!) When engineers do overcome their reluctance, they want that activism to be as apolitical and purely informative as possible. For example, when Einstein signed and sent the famous letter, it didn’t matter whether the president he sent it to was a Democrat or Republican. It happened Truman was a Democrat. But if Eisenhower had been president instead, would it have made any difference? The content of the letter did not hold a partisan view, only provided information. It explained a scientific fact, the possibility of building a bomb, and the danger the Nazis might do it first. It was information Truman needed to make proper decisions. The activism consisted only of bringing it to his attention.
More recently, when engineers have again been called to political activism, they have tried to stay within this mold. The message is: “Companies like Microsoft or Comcast need to be regulated to preserve competition on the Internet.” If one lives in Massachusetts, one could direct such a plea at either a Democrat or a Republican without looking at their affiliation. (Of course, to make this work, we have to be talking about a Massachusetts Republican!) Such a politician will automatically place such a plea into the long (Massachusetts!) tradition of busy but milquetoast commercial regulation. All the diagrams of networks or operating system architectures that perennially accompany such pleas are meant to provide the necessary information.
But do they? It was a wake-up call to me when I read Paul Krugman’s deconstruction of the argument in the Microsoft case. (I tell more of this story elsewhere. It provides the details of his argument.) I had been a massive fan of Krugman’s ever since my father had given me the “Myth of the Asian Miracle,” Krugman’s breakout work. In particular, I loved his silly stories: the babysitter co-op, the eternal triangle and others[TODO: references]. They were simple little fables that captured the heart of a difficult issue. Yet, they were also beautifully accessible. Sure enough, his analysis of the Microsoft case had just such a silly story—simultaneously virtuoso, incisive and accessible.
Yet I was bitterly disappointed. For all its virtuosity, the silly story was strangely vacuous. It wasn’t a model of the thinking of the engineers. Instead, it was a model of the politicians who heard the plea and pattern-matched on its form. Politicians, when deciding what to do, are naturally going to choose political responses from the buffet of options that economists have already articulated and blessed. So for an economist to describe the form of a political response isn’t introducing any new ideas. It is just recycling their own thinking.
If the economist’s own thinking was enough, though, then engineers wouldn’t be accompanying their pleas with a vast amount of technical description of operating system middleware, network architecture, or all the rest of that. They were struggling to describe technology that “wants.” They were trying to get at the source of its desires. Maybe they weren’t doing a fantastic job. But they were trying. Engineers don’t usually do that sort of thing. When they do, it is notable—and a cause for alarm.
Yet, Krugman didn’t see it. It wasn’t even that he didn’t understand what the engineers were trying to say. He didn’t even notice there might be something there to understand. He didn’t recognize the struggle to communicate. As a result, he didn’t even start listening.
Perhaps there is a general problem in a world run by experts who possess high-powered, hard-won hammers of honed expertise. Everything comes to look like their favorite nail. Seasoned Massachusetts regulators see regulatory challenges. Seasoned inventors of theoretical applications of increasing returns see a theoretical application of increasing returns. Seasoned antitrust fighters see an antitrust fight. Once they are certain they see the nail they are so wonderfully equipped to hammer, they can’t see anything else.
Nobody was equipped to hammer this particular nail. So no one could see it.1
The Psychohistorian
My disappointment with Krugman’s silly story led me to another thought: what would I have wanted instead? Implicitly, I was imagining a super-Krugman who had already realized there was something to hear, already waded through the morass of technical jargon the engineers using to explain themselves, already trimmed off all the excess detail to render it as one of his simplified models, already cast that into mathematical form, already written the textbook, already taught the course for a zillion terms, then already distilled that mathematics into the correct silly story that would capture the central issue, and then already condensed that into a 750-word New York Times column. Perhaps such an expectation was a bit… excessive?
It falls to me, then, to start filling in the holes. One could see this book as an intellectual treatise peppered with asides about various colorful characters. But another view is that it is all characters. The parts that seem like intellectual exposition are actually a profile of the main character: the Psychohistorian. Krugman has been quoted:2 “there are certain novels that can shape a teenage boy’s life… I grew up wanting to be Hari Seldon, using my understanding of the mathematics of human behavior to save civilization.” This book could be understood as the vision of such a psychohistorian: broadly how he would model the situation, what disintegration he would fear, and what heroic response he would devise to “save civilization” at the end of an age.
https://www.theguardian.com/books/2012/dec/04/paul-krugman-asimov-economics At least one other economist was similarly motivated: https://marginalrevolution.com/marginalrevolution/2009/05/hal-varian-at-google.html
Critically, my task is merely to profile the character, not embody him. I won’t even try to generate all the “mathematics of civilization”. We do start the first chapter with a stern discussion of the interpretation of a certain equation. But it is a simple equation!3 From then on, I leave all the fancy math to my character to fill in. I’m merely trying to make it believable that it could be filled in. The goal is more to create a believable character rather than a believable argument. Of course, the character isn’t believable if the argument isn’t. My character would never stand for nonsense! But my rendition of the argument is left much, much sketchier than my character would ever find acceptable. That is not a mistake. It is by design.
I am deliberately leaving this aspect sketchy. We don’t have space to work out all the details with the care to satisfy professionals.[^satisfy] Only by moving fast can we go far enough to motivate an army of graduate students in economics—to say nothing of political science and sociology—to fill everything in.
My goal is to imagine what my character would do. Suppose engineers had managed to get through to intellectuals who could interpret and translate their concerns. What course of action would such representatives champion? What problems would they identify? What remedy would they propose?
Two Problems
Such a psychohistorian might feel empowered to move beyond the engineers’ preoccupation with things that might go “boom”. Technology is generating two far more immediate and burning concerns than mere dramatic conflagration. One is that poor understanding of digital technology is being exploited to tilt the notion of appropriate social contracts. They are becoming dangerously unbalanced. The second is the risk that, as a result, no-one is in charge. The danger of technology in this case is not that it creates an excess of force, but an absence of it. The danger is implosion rather than explosion.
As we retold in the last essay, poor understanding of the ultimate social consequences of new explosives like TNT was used to sell violently one-sided social contracts. These absolved the upper classes of their responsibilities to the former code of chivalry. But they left in full force the obligation of the peasants to die in the resulting wars.
Similarly, confusion about digital technology is being exploited to justify violently one-sided deals. Somehow, the notion has taken hold that robots are going to take everyone’s jobs.4 As a result, the story goes, it has become justified to take away workers’ safety net: their health insurance, their unemployment benefits, their right to union negotiations, and so on. More intangibly, this justifies an assault on their pride, dignity, and hope. But the capitalist’s prerogative to own the output of factories where people will no longer work goes unquestioned.
Concerning Engineers
As an engineer, this development is acutely upsetting. If I was not occupied writing this book, I might well be working on robotics. Or, more likely in my case, the more foundational technologies that make robotics possible. It is what everyone close to me is doing. So I want to believe this effort will give ordinary people greater freedom, wealth, and reason for optimism. I want it to be a laudable effort.
But the example of history teaches me that when technology is used as an excuse to tilt the social contract, no such rosy result is guaranteed. Maybe people will look back on our breakthroughs with the same attitude with which we look back on Haber’s. They might think it better if the advances were delayed. Delayed, perhaps not indefinitely, but at least a little while. Perhaps it would have been better, they might say, if society had had a chance to grow into the challenges the new technology generated.
Possibly, in the long run, our advances will help humankind the way Haber’s did: hugely, almost immeasurably. Since technology seems to be developing faster, that immeasurably better-off era might be closer than we think. Even so, it still lies after we are all dead. So that might be little consolation when the justification for one’s effort is not some positive impact somewhere out in the hazy indefinite future. We want it things to get better now. One has the sinking feeling that if the only benefit to humanity falls in the distant future, then it might be better brought about by someone else.
Thus, I am dismayed, deeply dismayed, that philosophers and politicians aren’t fighting harder against this increasingly unbalanced story. Even if one doesn’t know what is true, a study of history would show that stories like these must be false. It all reeks of Grand Illusion. It can’t be true.
- Later (TODO: reference) we will talk more explicitly about the “Somebody Else’s Problem Field” at work. If I’m talking up how easy it is to fall into this trap, one question might be: how can I claim I escaped it? One short answer: I didn’t. I was lost in the dark for a long, long time. Not just lost, but stuck on an answer that was wrong (or at least only partially right.) Another longer answer, however, is my motive. I wasn’t motivated by ambition to show off my nail-hammering skills. If I have nail-hammerings skills I’d be thrilled to display, they belong to another area entirely. (Want to talk programming languages? Automatic Differentiation? Linearity? No? What a pity.) In this field, I couldn’t care less if anyone thinks I’m good with a hammer. Instead, I was motivated by fear. I was afraid of the things we can’t see, of the evil that has no name. I was afraid of a political vacuum—and of the leaders who would inevitably fill it. I am a part-Jewish child of war refugees. I was raised to be afraid. Being afraid, posturing pounding nails wasn’t good enough. I had to see. ↩︎
- ↩︎
- Then again, sometimes the simple equations are the important ones. F=ma is simple, after all. This one will be no more mathematically sophisticated. But perhaps it is equal in importance. ↩︎
- Which assertion I don’t believe. Even if it is correct, it is correct in only the narrowest and most misleading sense. There will be much more to say later[TODO: reference] on this subject. ↩︎
