Our storytelling has moved backward from effect to cause. As in the movie Memento, we simulate the confusion of life without short term memory. Why? Because the conversation between engineers and intellectuals had that quality. It had no memory. Engineers never explained their logic: they merely invoked it. Images like The Tar Pits or The Borg referenced a shared worldview. The internal logic was never elucidated.
That absence of logical exposition had a particular painful consequence. The initial force that drove the argument was never mentioned. Never, at all, anywhere. There was a force so pervasive and powerful inside the community which debated its consequences that nobody thought to mention it. Why explain what everyone knows? If a force completely dominates every minute of everyone’s life, everywhere and all the time, nobody will explain it. If it dominates so completely that nobody questions the shared goal, then there is no need for argument. If nobody argues, there is no need to articulate. So it goes unspoken. This community allowed the history of its shared purpose to be forgotten. It was relatively recent history. But even so, it was lost to memory. The key was lost in the dark.
Stating Assumptions
This section will unearth that missing memory. Our reversed story will run backwards to its beginning. As Krugman said, a rule of his research was “Listen to the Gentiles”. He added the caveat: “I have no sympathy for those people who… imagine that they achieve greater sophistication by avoiding stating their assumptions clearly.” Certainly, Brooks’ crowd were great sinners in assumption statement avoidance. However, this propensity was not driven by hope for sophistication. Mostly they didn’t understand the reason for the requirement.
A fisherman who sees and smells and feels the approach of rains doesn’t realize he is required to describe the approaching clouds. Especially since the clouds in question are hidden from his sight, he may not be able to describe them. He can’t see them either, after all! All he sees are the signs. He didn’t see the potential for influence he could win if only he could address the Doppler-radar wielding set in terms they would understand.
Krugman almost certainly didn’t understand the difficulty in grasping this opportunity. Because he was so good at his art, he didn’t understand that others would find it impossibly daunting. He was imperious and curmudgeonly because he assumed the opportunity was understood. He thought they knew he had valuable influence to offer to someone who could persuade him. For sure, he was bitter about being “brushed off.” But that bitterness was accompanied by a potential for reasonableness for those who engaged with him. He thought it was understood. It wasn’t.
There is an aspect of Krugman’s imperiousness about which I am violently ambivalent. Also in his rules for research, he says: “it is also crucial to express your ideas in a way that other people, who have not spent the last few years wrestling with your problems and are not eager to spend the next few years wrestling with your answers, can understand without too much effort.” In that regard, Brooks was a huge offender. He was a project manager writing for other project managers. Entirely without apology, he expected his audience to have “spent the last few years wrestling with” his problems. After all, that endeavor was their profession. In that context, it was an entirely reasonable expectation.
So was it also a reasonable expectation of intellectuals observing Brooks’ spreading influence? Or did they have a right to refuse to “wrestle with [his] answers?” Did they have a basis to demand to “understand without too much effort?”
About That Dynamo.
Alternately, was such a demand professorial imperiousness run amok? Though it is a small thing, I’m stunned by the name-check of Henry Adam’s famous essay on the Dynamo. Why? Because that essay isn’t a FRED-style analysis the productive potential of the Dynamo. Not even close. Instead, it is a lament over the difficulty of narrating cause and effect:
Historians undertake to arrange sequences...assuming in silence a relation of cause and effect. These assumptions, hidden in the depths of dusty libraries, have been astounding, but commonly unconscious and childlike; so much so, that if any captious critic were to drag them to light, historians would probably reply, with one voice, that they had never supposed themselves required to know what they were talking about. Adams, for one, had toiled in vain...
Adams echoes Krugman’s warning about “sleepwalking.” But in his telling, overcoming the tendency to “sleepwalk” is difficult, bordering on impossible. Adams goes on to say that he’s written a dozen volumes of American history, and is still utterly dissatisfied with the result. To hear him tell it, narrating a changing world from the midst of that change is much harder than Krugman is making it out.
Is Krugman overdue for a dressing-down? Maybe he badly needed to face that the whole world must not be treated like his personal graduate seminar. The discussion had no boundaries, no agreement about expectations. That problem was as big or bigger than the lack of shared worldview. Even now, we can’t ignore it.
Both sides...
Both sides faced the frustrations of dealing with the other side. Neither could see the opportunity. Nor did either realize that the implicit opportunity which motivated their imperiousness wasn’t apparent to the other party.
This was true in both directions. Brooks’s imperiousness was a fueled by an implicit opportunity he assumed was understood… but wasn’t. He was a universally acclaimed expert project manager. He had thought hard and honestly about the determinants of success of software projects. His community assumed the potential value he offered had been understood and rejected.
To them, this potential was more valuable than any ideas they were championing. After all, they had no illusions that their ideas, as ideas, were strange, disorganized, and poorly expressed. Neal Stephenson’s discursive essay on metaphors and models admitted as much. Then he further admitted that his attempt to fix the problem wasn’t that great either. Nonetheless, Fred Brooks had an imperious sense of his own value. He offered unmatched experience grappling with “jobs where things get done.” He expected that was understood.
Missed the Value
It probably wasn’t. Another offhand comment by Krugman indicated that he had entirely missed this point:
My favorite example of an innovation that had a huge impact on the economy’s productivity, but didn’t get very much press, was freight containerization. Where have all the longshoremen gone? If you go to a major port, like Oakland’s, what you see is this vast expanse that looks like a scene from a science fiction movie in which the human race has been wiped out. There are no people around, yet hundreds of thousands of tons of freight are being moved through there. That was an enormous change, but it wasn’t something that caused a buzz. There may have been a magazine devoted to containerization, but if there were, it wouldn’t have been as much fun as Wired magazine.
Why did he assume that the “less fun” magazines or books devoted to containerization didn’t exist? Had he tried to find them? He was at MIT at that time. And MIT houses rooms and rooms filled with computer science’s equivalent of “journals of containerization.” It also houses plenty of people who do read them. Attentively, even. And find them fun.1
What’s more, squint a bit, and you realize that these “journals of containerization” discuss… containerization.
Offered By…
The Mythical Man-Month could even be characterized as a work “devoted to containerization.” Most of the book is intensely focused on the problem of raising productivity in software development. After twenty years of grappling with the problem, Brooks concludes:
… modules of code should be encapsulated with well-defined interfaces, and… the interior of such a module should be the private property of its programmer, not discernible from outside… [This] is the only way of raising the level of software design.
The watchword here is “encapsulation.” Squint, and you see again “containerization.” The setting is more abstract, for sure. But it is there.
In its own community, the idea caused a tremendous buzz. Computer science at a place like MIT, in some sense, talks about nothing else. If you missed the emotional energy this community invests in this idea and its refinements, you also have missed the whole value it offers. You have also missed its the motivating force.
It is worth noting the context in which Brooks made this observation. He introduces the question in his characteristic vivid style:
Of all the monsters that fill the nightmares of our folklore, none terrify more than werewolves, because they transform unexpectedly from the familiar into horrors. For these, one seeks bullets of silver that can magically lay them to rest. The familiar software project, at least as seen by the nontechnical manager, has something of this character; it is usually innocent and straightforward, but is capable of becoming a monster of missed schedules, blown budgets, and flawed products.
… But, as we look to the horizon of a decade hence, we see no silver bullet. There is no single development, in either technology or in management technique, that by itself promises even one order-of-magnitude improvement in productivity, in reliability, in simplicity….
Skepticism is not pessimism, however. Although we see no startling breakthroughs—and indeed, I believe such to be inconsistent with the nature of software—many encouraging innovations are under way…. There is no royal road, but there is a road. The first step toward the management of disease was replacement of demon theories and humors theories by the germ theory. That very step, the beginning of hope, in itself dashed all hopes of magical solutions. It told workers that progress would be made stepwise, at great effort, and that a persistent, unremitting care would have to be paid to a discipline of cleanliness. So it is with software engineering today.
In the article criticizing Dertouzos, Krugman bemoaned how little emotional energy was directed towards the unglamorous task of “making productive innovations.” One might imagine he would be thrilled by this passage from Brooks.
Yet it was exactly this effort which led to the passive aggressive intellectual rebellion which so incensed him. He failed to see the driving motive behind the rebellion. Therein lies a tragedy.
The Obscure.
Brooks’s ideas have been built upon so extensively that they are almost never mentioned as such anymore. One must find a book as old as the Mythical Man-Month to find them elucidated explicitly. For an outsider seeking to understand, this is a huge hurdle.
For example, one might claim that Wired does talk about Brooks’s ideas. For instance they appear implicitly in this article about WhatsApp notable productivity. This story was chosen fairly randomly. At any given time it isn’t hard to find one like it. Reportedly, WhatsApp managed to serve 900M users with only 50 engineers. “One of the (many) intriguing parts of the WhatsApp story is that it has achieved such enormous scale with such a tiny team.” Intrigued, the article reports how WhatsApp hired carefully and leveraged high-productivity languages like Erlang. It also mentioned similar feats at Facebook enabled by the language Haskell.
Boiling this story down, it transforms into an assertion akin to Brooks’s observation. It claims that companies can achieve impressive per-employee productivity by leveraging certain languages. They hire programmers who are expert and disciplined about using the special capabilities of those tools. What is the magic bullet they are describing? Informed by knowledge of the technology in question, one can figure out the key point: these tools especially enable encapsulation.
However, the article never explains this idea explicitly. Instead it invokes only the more trendy notion of concurrency. The article doesn’t bother to explain the special value of Erlang and Haskell. These tools provide encapsulation that still contains effectively even in the presence of concurrency. By contrast, most other languages’ abstractions tend to leak or shatter under such stresses.2 It requires specialized knowledge to dig out this point.3
A high-level observer seeking to understand the economy shouldn’t need specialized knowledge of particular technologies such as these. The internal logic driving such news stories is buried. The essential point is unstated.
It is strange to be waylaid into prognostication about the future of the economy after being trained in programming languages. And yet one needs such esoteric training to understand the central point of news reports such as these!
Being Explicit Matters…
Why does it matter that Wired is less than fully explicit about the phenomenon it is reporting? This essay has purposely emphasized the phantasmagorical in the presentation of this worldview. That is how the standard presentation is so likely to make it seem.
These phantasmagorical ideas grow out of a culture rooted in the discipline of “containerization.” Observing this, the apparent strangeness of these ideas can be dispelled. Rivers whose fishes swim backwards may seem fantastical. Bogeymen who prey on the hapless fishies seem even more so. However, it all is simpler and more understandable than it may at first seem.
If we want to be sassy about it, we might point out that the Krugman quote about productivity in the introduction had hidden within it exactly this idea. He had heard rumors of exactly this kind of reverse flow—and angrily rejected them.
Yet he had also used the idea himself, unawares. As we might recall, he bemoaned the difficulty of dealing with large issues. In the public sphere, the large issues are jobs, growth and income inequality. He was concerned by economist’s tendency to get waylaid by “worr[ies] about small things.” In particular, his prime example of a “small thing” is “the state of my basement.”
Let us examine this “small thing” more closely! Note something strange: a busy and well-off Nobel Prize winner with a weekly column wouldn’t normally worry about household chores. For instance, he almost certainly hires someone to mow his lawn. So why not hire someone to clean his basement too?
To my great delight I chanced upon reporting quoting Krugman, Summers and a few other economists concerning their shared hatred of home chores. “[Larry Summers] makes decisions about priorities in ways that others might not,” said Paul R. Krugman.… “He’s not going to force himself to do something that doesn’t interest him. He doesn’t do any handiwork around his home.” To my further delight, the reporter tracked down Summers for elucidation: “It’s true… I don’t mow the lawn… I hate it. I hate mowing the lawn. I love my work.” Invoking the classical economic theories of comparative advantage and the division of labor, he said, “I can do something I love doing for an hour that will pay well enough to pay someone who would enjoy mowing the lawn more than I would.”4
“He’s not a Renaissance man,” [another economist in the conversation] observed. “Not many economists are.”5
However, despite that community’s avowed hatred of home chores, nonetheless Krugman specifically mentioned his anxiety over the state of his basement. Why didn’t the theory of comparative advantage apply in the same way to the basement cleaning chore? As we might intuitively understand, that job feels different. It is much harder to outsource completely.
Suppose someone was hired to guess at a basement organization system. His job would be putting things in boxes and labeling the boxes. Even a busy columnist with looming deadlines might still want to check the boxes. He’d want to ensure the categories were useful and the labels appropriate. He might even spend time rearranging to be sure the organization system suited him. He wants to be able to find things again! Far from being fantastical, this observation seems too mundane to be worth mentioning.
Now let us cast our “basement cleaning economy” as a formal model. Then the hired basement cleaner is the “supplier,” the busy columnist is the “consumer.” The scheme of labeled boxes are the “goods” which float down the economic “river” from supplier to consumer. Such world seems stiffly formal. But bear with us! We will learn something.
In this formalization we need to model the professors’ act of checking the boxes. Where does this activity appear in our model? Check it out: it is a reverse flow of economic energy! Like the aforementioned salmon, it runs “backwards” from the “consumer” to the “supplier!”
The “small thing” of worrying about the “state of my basement,” was to Krugman just a throw-away line. But it contains within it the central intellectual substance of the challenge to the Rhine model! Explained this way, it hardly seems phantasmagorical anymore. Instead, it seems almost too mundane and intuitive to be worth mentioning.
Since these ideas seem mundane and intuitive, the community in which they originated didn’t, indeed, bother to mention them. But that gives the rest of the story a Memento quality. The key—the starting point—has been forgotten. The origin of these ideas doesn’t lie in the distant past. But a quirk of this conversation is that it had no memory: not even short-term memory.
But Powerful Ideas…
For all that this idea seems too mundane to mention, omitting it has serious consequences. As we noted before, practitioners like Brooks tend to present their ideas in reverse logical order. And they leave out the crucial initial steps of their story.
Everything fantastical in this worldview suddenly becomes intuitive when sees where it starts. Just imagine checking the labels of boxes in a basement. Containerization is the key first step. But leave that out, and it makes no sense at all. And Brooks and his intellectual descendants do leave it out.
Were Invisible.
In all this, each side was pushing on the other a job they were extremely unlikely to do. As each side told it, the important work was “Somebody Else’s Problem.”
On the one side, Krugman was clearly furious that his Rhine-model story received no rebuttal. But is it reasonable to expect someone like Fred Brooks to formalize his alternate worldview? But on the other side, Brooks and his community wallowed in self-pity about the supposed lack of respect for his profession. Somehow they didn’t even notice they had generated a dramatically new worldview. Implicitly, they expected the theorists to comb out their logic from a mass of phantasmagorical aphorisms. This was a similarly unlikely and unreasonable demand.
Not Actually Invisible
It is as if the real phenomenon of a New Economy had landed in the Hayward Street parking lot. It sat there bristling with armaments, a neon sign blinking “Death Ray Blaster.” A strange ooze of acidic goo leaking out the bottom. Yet it caused no serious stir. Experts at MIT walked by it every day, looked straight at it, and even wrote extensive monographs on how everything had changed since it had landed.
Yet no one ever quite properly reacted. It was effectively enveloped in a Somebody Else’s Problem field. Even as the pool of acid got deeper, and the blinking neon death threats got more lurid, somehow it remained not-invisible and yet incapable of stirring a serious response. People’s “natural disposition not to see anything they don’t want to, weren’t expecting, or can’t explain” kicked in. The problem was effectively invisible.
Breaking A SEP
In Douglas Adams’s tale, the narrators were immune to the effect of the field cloaking the spaceship “simply because they knew it was there.” There is some threshold of unexpectedness and unexplainability which will render a phenomenon effectively invisible. But move it only a little way into the realm of the knowable, and it will blink into existence undeniably.
Observers who “know it is there” will easily see the phenomenon. They will look with amazement at the others who can’t see it “even though it isn’t actually invisible or anything hyper-impossible like that.” The line between invisibility and inescapability is oh-so-thin. Yet it is somehow oh-so-hard to surmount.
One doesn’t have to argue that the phenomenon is fully known to break the power of the Somebody Else’s Problem Field. One only has to argue that it can be explained and should be expected. Also, crucially, that it isn’t nearly as terrible to handle as observers initially might suppose. We want to convince our readers of the empowerment that might come from making this Their Problem.
Tucker Carlson calls our ruling class “disgusting.” As I said earlier, I think he’s wrong. To my eyes, our problem is that we are specialized. In a world of super-specialized experts, each individual is not able to handle a large problem alone. So they push it away. To protect themselves, they insist it belongs to Somebody Else.
This challenge needs lots of talented people who are willing to accept it as Their Problem. And not only that, but work on it effectively together as a team. Creating energy for teamwork is key to breaking a Somebody Else’s Problem field. For this reason, we will spin an extended story of the Heroes of the Old New Economy. We want to portray not just their individual achievements, but their achievement in working together. As we said earlier, we want to whisper “You’ll be a hero. You’ll be amaziiing.” And not just that, but “You’ll make an amaziing team.”
Towards the Knowable
This book might be mischaracterized as an attempt to devise a formal model of the New Economy. It might be further mischaracterized as offering blueprints for the resulting political response. This characterization would be wrong. One might expect such an attempt, if done professionally and properly, to have formal mathematical models backed up by reams of sociological data, proper statistical analysis, scholarly notes and references, and so on. Further it should have position papers, policy analysis, etc. etc. In other words, the whole nine yards. And it would most certainly not begin with silly science fiction imagery lifted from a book with “DON’T PANIC” emblazoned on the cover.
This book has none of that scholarly stuff. Nor is its omission an oversight. I could have generated it if I had wanted to.6 But that wasn’t the point. I’m not trying to establish truth. I’m trying to break a Somebody Else’s Problem field.
The goal is to lift the shroud of darkness from the spot where the key is lost. There are plenty of people ready to pick up the key—as long as it sits lost with a bright streetlamp shining on it. We aim to light that lamp.
- Incidentally, I’d vastly prefer to be reading such a periodical right now instead of writing this sentence. If I could ditch this problem, the next day I’d ask Edward Kmett for a list of the current computer science papers worth reading. (Ed Kmett is the person whose judgment I trust most to make such a selection to match my predilections.) Then I’d snuggle up happily with a big stack papers. And they would all of them—every single one—be about containerization. Of course, not as such. They would be discussing it using the crazy unintelligible Ed Kmett-way of talking about the problem. It sure wouldn’t sound like anything akin to an Oakland port. But it would be there. ↩︎
- [TODO: link to essay about what makes a language powerful?] ↩︎
- The one and only one place in this book where all my hard-won graduate studies come into play. Except they don’t, because my thinking was already far beyond this point by the time I was engaged in such studies. But let’s pretend that special expertise contributed to these insights. It didn’t. Yet our world is sufficiently overawed by obscure expertise that I’ll happily take the opportunity to claim (in possibly misleading manner) some tiny share of it. ↩︎
- A commentator objected to this last comment: about those who “enjoy” mowing the lawn. He called it off-key. Sadly, he has a point. It does scream “tone-deaf shill of the global elites with no empathy for the life of the common man.” To be fair, though, that comment was made in 1988. In 1988, the common man had not yet begun to suffer, nor the global elites to oppress, nor Summers to shill for it all. Perhaps he can be forgiven for lacking the prescience to predict his future vulnerability on that score. And if he now is such a shill, the tone-deaf mistake he made has little to do with these particular plutocratic self-serving illusions. Perhaps the guy sweating behind a lawnmower tending to his front lawn is not enjoying the effort expended. But does such a comment betray a failure of character which led to his other, more consequential mistakes? (For that matter, later we will quote an operator of a much glorified mower. He does profess true enthusiasm about the act of mowing.) ↩︎
- Apparently the New York Times’ definition of “all the news that’s fit to print” is broad enough to include reporting on economists’ feelings about home chores. Not only that, but reporting about their feelings about lawn mowing specifically. I am amazed and delighted. OMG, I love the New York Times. ↩︎
- I did go to MIT. I can deal with technical detail. I can do math. I can slip into the culture that feels most safe and cozy when in full-on technocrat mode. I can sorta, kinda, almost blend in. I can do all that if I choose. I’m choosing not to. ↩︎
