The Efficiency-Destroying Magic of Tidying Up

In his seminal book Seeing Like a State, James Scott describes what he calls “high modernists:” lovers of orders who mistake complexity for chaos, and rush to rearrange it from the ground up in a more centralized, orderly fashion. Scott argues that high modernists end up optimizing for a system’s legibility from their perspective, at the expense of its performance from that of the user.

Indeed, that love of order is above all else about appearances. Streets arranged in grids, people waiting in clean lines, cars running at the same speed… But everything that looks good doesn’t necessarily work well. In fact, those two traits are opposed more often than not: efficiency tends to look messy, and good looks tend to be inefficient.

This is because complex systems — like laws, cities, or corporate processes — are the products of a thousand factors, each pulling in a different direction. And even if each factor is tidy taken separately, things quickly get messy when they all merge together.

The chaotic look of structural orderliness shouldn’t be so surprising. Intellectually, we do understand that appearances are misleading — things don’t have to look as they are, nor be as they look. But intuitively, we all remain hopeless slaves of appearances, no matter how often we were misled by them.

This natural messiness of efficiency is demonstrated by recent advances in industrial design. When a God-level AI takes over in a science fiction book, it often remakes the world in its image: full of straight lines, smooth acceleration rates, and lots of chrome (AIs love that stuff). But as we start using algorithms to design things, we get results that look a lot more chaotic than that, confirming that our intuitive preference for “straight line” designs has nothing to do with performance — it just comes from our limited ability to reason about more complex solutions. Ironically, it’s us humans who think like robots.

This is a messiness similar to that you’d find in nature — which makes sense, since both these algorithms and nature are optimizing for efficiency.

I submit that we should look with suspicion at simple-looking systems. The physical world is like a river in which a thousand streams come rushing — it is supposed to look messy.

Again, this insight applies to any complex system. For example, a city can look as messy as an anthill. But really, it’s a beautiful equilibrium that evolved to satisfy a thousand competing constraints: topology, weather, people’s traditions, skills, wealth, preferences… Planners may make their maps look better when they use zoning to separate the city into business, residential, and commercial neighborhoods, but they also destroy a subtle, efficient balance. They forget that the only activity that goes on in any city is that of people living their lives, which requires all the activities above — preferably in close proximity. Splitting a city into residential, commercial and business zones is like throwing dough, cheese and pepperoni into the different compartments of a bento box and calling it a pizza.

Speaking of pizza. One good example of an attempt to avoid the high modernist fallacy is Amazon’s “two pizza teams,” set up to run like small companies and encouraged to use the tools and processes that make sense at their level. The outcome can be messy — new Amazonians who come from more centralized companies often complain about the chaos there. But I, for one, would choose chaotic success over tidy failure any day.

Interestingly, other managers often pick the latter. They’ll see a dozen teams, each using different project management software, and lament: “what a mess! No one even knows what everybody else is doing!” They’ll then resolve to “put some order in here,” by mandating every team uses a single company-wide tool. But teams had been using different tools for a reason. Maybe they’re in different businesses — for example, hardware and software groups have different needs. Or their members could just have different preferences, which matter too — people are more productive when they use tools they like! It can be okay to sacrifice a company’s productivity so as to increase the coordination between its parts — so long as one makes this compromise with their eyes wide open. Unfortunately, managers all too often look only at the upside of such changes, and ignore their downside.

“Please clean up your room,” asks the mother. “Fool,” retorts the three-year-old with an eerily deep voice. “Can’t you see the beauty in my glorious chaos?”

I’m not suggesting all chaos is good. But when you hear someone suggest we put some order into a system, there should at least be a red light flashing in your head. Before you touch anything, you should make sure that mess is not concealing a deep order, by answering the following questions:

• How much information is contained in the system’s current state? What constraints are expressing themselves through it? For example, when throwing dice, where they land depends on how you threw them. You do lose this information when you take the dice and put them back into a box — and that’s probably fine. But in the case of a city, its structure is the function of all the factors mentioned above — most importantly, its people’s skills, means and preferences. When you use zoning to impose your arbitrary aesthetic upon people, you block them from shaping their environment so it fits their lives.

• How old is the system? How malleable is it? How strong are the forces put on it? The older a system, the more malleable or subject to strong forces, the least likely it is to be truly chaotic — and the more careful you should be when messing with it. That’s because its components had the time, ability and stimuli to order themselves into a stable configuration. Crystals are a good example of this. Take some randomly arranged molecules, keep them warm enough so that they remain mobile, and put them under high pressure for long enough, and they’ll spontaneously self-organize into one of the most orderly structures in the Universe.

• Finally: who is complaining about the chaos? If outsiders complain, but people living inside the system seem happy with it, it probably means that the chaos is serving them right, and that it’s just foreign eyes who are unable to perceive its underlying order.

This is a special case of Chesterton’s Fence, which states you should never take down a fence before knowing why it was put up. Here, I propose Scott’s Law: never put order in a system before you understand the structure underneath its chaos.