Deets
- Seeing Like a State by James C. Scott
- ISBN-13: 978-0300078152
Review
I finished this book on new years day, 2025. I however count this as one of the best books I read in 2024. While the tech industry is never explicitly mentioned, James C. Scott’s detailed discussions of large scale failures to improve the human condition feel incredibly applicable to my industry.
What is it about technology that spurs so many of it’s members to audacious and ill-fated schemes to change the world around them? Why did Tony Hsiesh think he could rebuild Las Vegas downtown? What drove the Bill and Melinda Gates Foundation to think that Common Core was going to improve education? The intentions are always good, but the road to hell is paved with them. Seeing Like a State examines this through the lens of governments, looking at suburban developments, Tanzanian resettlements, the Soviet Union command economy, and other examples.
The famous torment nexus quote. The fact that I am linking this image from another blog post of mine illustrates my hubris.
I admittedly came into my reading thinking that there is nothing wrong with desiring to build something grand and to restructure the world around it. With sufficient will I, a software engineer could build grand things. The much hated panopticon? I am in security, I yearn for the panopticon. I was ready to “Build the Torment Nexus” as the famous tweet goes. However, as I read more and understood the issues faced, I understood that I had fallen into the same trap as the examples in the book. I “regarded myself as far smarter and farseeing than I really am” to paraphrase a line near the ending.
The great takeaway for where things go wrong in these grand schemes comes near the start:
In sum, the legibility of a society provides the capacity for large scale social engineering, high-modernist ideology provides the desire, the authoritarian state provides the determination to act on that desire, and an incapacitated civil society provides the leveled social terrain on which to build.
Legibility is a key concept throughout the book. It refers to the concept of measuring things that the state cares about. The amount of grain harvested from a field, the number of lumber boards taken from a forest, the speed one can get from point A to B in a city, are all examples of legibility. High-modernism desires to improve on those metrics, and an authoritarian state let’s one execute on those desires.
I came across this while taking a break from writing the review and it was a very good example of both legibility and the hubris of those that make use of legibility
The improvements involve grand sweeping changes to discard antiquated, and backwards, things in favor of new modern ones. Where this goes wrong is when the state overrides the civil society and carries out the changes without regards for the expertise of the populace or the world around them. The focus on legible statistics and goals means that the illegible ones, overlooked with intent or by ignorance, are ignored. Disasters result, in the form of starvation, low yields of manufactured goods, and bland, unfulfilling cities.
Where then, is the solution to be found? James C. Scott unfortunately offers less stories of success than of failure, possibly because it is easier to point out problems than find solutions. There are a few repeating patterns of success when he can identify them: metis, acceptance of illegibility, and attempting to achieve harmony with the world around you. In his own words: “The proper test for any practice was whether it worked in the environment concerned, not whether it looked ‘advanced’ or ‘backward.’”
Metis, or the respect of the metis of workers close to the job, is a key, reoccurring item. This is respecting, even empowering, the civil society to achieve goals. “Metis” itself is translated from Greek by James as a “cunning intelligence”, or “experience through hands on work and lived lessons.” The book defines it as “metis represents a wide array of practical skills and acquired intelligence in responding to a constantly changing natural and human environment.” The value of metis comes from the fact that there is a massive amount of knowledge out there that cannot be taught in classrooms nor abstractly, but must be learned through hands on labor and producing outputs. This is true of weaving, programming, evaluating mathematical equations, building cities, and growing crops. Oftentimes the lowest level workers, those doing the work, will have greater metis than the higher level admins and can understand what is needed to do for a project to succeed. That prior fact is why respecting the metis of the lower level workers is so important.
Acceptance of illegibility comes next. High levels of legibility require increasing amounts of effort to track every little detail. As more of the world becomes understood, there are more items to track. As this digs deeper and deeper, one eventually hits some application of the Heisenberg Uncertainty Principle at a decidedly non-atomic scale: one cannot perfectly measure the complete state of a system, and have that system function. The more you focus on one, the less is done with the other. As the system itself functioning, and producing outputs, is what matters the most, measuring the functioning of the system is only useful to the detail required to make sure it is producing the approximate expected outputs.
Finally, harmony with the world around you is important. Because a system must exist and function in reality, and because reality cannot be completely subsumed in some assemblage of mechanical and mathematical “perfection” one must allow for the world around to affect the system being built. Crops are more successful when they account for the environment they are grown in then when trying to perfectly control that environment. Distributed computer systems that allow for latency, and dropped data, are more successful than those which do not. This does not mean that a system accepts sloppiness and disorder for the sake it of it, but that it accepts and works with complexity. As Jane Jacobs was quoted in the book: “The order of a thing is determined by the purpose it serves.”
Personal Lessons
As I read through this I kept on thinking about the tech industry and how this applied. My industry is particularly notorious for both a great desire to have detailed legible systems as well as a failure of those systems to meet their goals. Griping about ticketing systems are evergreen; the desire of PMs and Managers to see detailed updates and metrics, and the annoyance of engineers in attempting to just do their jobs, without filling out reports, are a familiar tale. Jokes about cloud service dashboards that don’t truly tell you if a service is up or not may be a more succinct reference.
Part of the problem may itself be structural. As software engineers we are trained on computers that act exactly how they are instructed to. “Episteme” is a concept that comes up near the end, referring to learned knowledge, where there is a connotation that it must be Scientific and it is Settled Fact from the Experts. In other words, the things you must recite to get Good Grades in University and get your Degree. “Techne”, another greek word, refers to the art (yes, art) of crafting. We are technologists who craft things, where we rely on assumptions that 1 + 1 = 2. Within software we rely on epistemic knowledge to decide how to craft things, and there is a inherent bias to rely on experts rather than defer to experience.
A factory worker, understanding that the written instructions are bunk, may make small adjustments to improve the process. Computers require enormous sums of money and compute to arrive at minor improvements to carry out poorly worded instructions. We call ability to infer “artificial intelligence” and are amazed by it, at least when it works. But the perfection that we believe is a delusion, one which exists at best skin deep. Web browsers offer varying levels of support for well known standards such as HTML and Javascript. Protocols like TLS and IPsec are notorious for having specifications which are ambiguous and too large to reliably implement without significant testing between both themselves and other implementations. Assumptions end up being made and are then baked into the standard because that is what everyone is doing. Attempts to measure progress are notoriously unreliable and large scale projects are notoriously hard to successfully execute on.
What lessons does Seeing Like a State offer then? I propose the following for successful projects:
- The people working on it must understand:
- What is the problem?
- Why have we decided on this solution? The solution itself should be something where there is some consensus among those working on it. Not complete consensus, but enough people must believe in it that they can carry it out.
- Also returning to a quote I enjoyed from Army Leadership and the Profession (ADP 6-22): Direct leaders understand the mission of their higher headquarters two levels up and when applicable the tasks assigned one level down. This provides them with the context in which they perform their duties."
- The project must focus on the solution:
- Is it solving a problem that someone has?
- Can small pieces be rapidly delivered to see if they solve the problem, at least in part?
- Can those small pieces be iterated on to see if improvements can be found?
- And if the pieces don’t solve the problem, or make it worse, it must be possible to reverse course and even remove those pieces. It must be possible to discard a deliverable in favor of a real solution.
- Finally, the project itself may change over time, and that is okay! Both the creators working on it as well as the users will have their own metis and insights that they bring. The project as originally envisioned may not be the right solution in the real world. In the end you have to just execute, deploy, and iterate. Success is determined by what works.