Posted by epb_hn 9/13/2025
Human cultural systems are even worse than non-human living systems: they actively fight you. They are adversarial with regard to predictions made within them. If you're considered a credible source on economics and you say a recession is coming, you change the odds of a recession by causing the system to price in your pronouncement. This is part of why market contrarianism kind of works, but only if the contrarians are actually the minority! If contrarianism becomes popular, it stops being contrarian and stops working.
So... predicting doom and gloom from overpopulation would obviously reduce the future population if people take it seriously.
Tangentially, everything in economics is a paradox. A classic example is the paradox of thrift: if everyone is saving nobody can save because for one to save another must spend. Pricing paradoxes are another example. When you're selling your labor as an employee you want high wages, high benefits, jobs security, etc, but when you go shopping you want low wages, low benefits, and a fluid job market... at least if you shop by comparing on price. If you are both a buyer and a seller of labor you are your own adversary in a two-party pricing game.
I personally hold the view that the arrow of time goes in one direction and the future of non-linear computationally irreducible systems cannot be predicted from their current state (unless you are literally God and have access to the full quantum-level state of the whole system and infinite computational power). I don't mean predicting them is hard, but that it's "impossible like perpetual motion" impossible.
I also wonder if we are being fooled by randomness when we think we see a person or a technique that yields good predictions. Are good prophets just luck plus survivorship bias? Obviously we forget all the bad prophets. All lottery winners are lucky, therefore lucky people should play the lottery. But who is lucky? The only way to find out is to play the lottery. Anyone who wins should have played, and anyone who loses should not have played.
How confident are you of your ability to identify such systems?
Many systems are not such and therefore easy to predict.
I've seen it happen more than a few times that when software needs to get made quickly, a crack team is assembled and Agile ceremonies and other bureaucratic decision processes are bypassed.
Are there general principles for when process is helpful and when it's not?
If you have need for speed, a team that knows the space, and crucially a leader who can be trusted to depart from the usual process when that tradeoff better meets business needs, it can work really well. But also comes with increased risk.
General principle 2: create focus on the critical path. (If each ticket you work on is slightly different from other tickets and no cookie-cutter solutions exist, then there is some chain of slow, annoying, winding steps, and the rest of the dependency graph doesn't really matter, just these big pains in the butt that often are linked in the dependency graph only by the fact that it's going to be one developer working on all of them and they can't work on them all simultaneously. It follows that you can only get interesting speed improvements if multiple developers are working on the same change. Note that daily stand up is an example of a meeting which does not make a decision—it could but in practice nobody uses it that way—but instead its function is to create pressure on the critical path. Often unhealthy pressure, someone was sprinting at 100% and now they are getting a little burned out, and daily stand up forces them to do something that they can report at standup lest they be honest and say that they're suffering.)
General principle 3: process helps create shared reality. (Consider two different ways to protect prod from the developers. In one way, everyone commits to main, some file or database table or configmap contains a list of features, and those features can be toggled in dev, uat, or prod. The process here is, whenever you change anything, you wrap it in a feature toggle, so that your change does not impact prod if that toggle is off. Versus, consider having three different branches, you can usually commit new features to dev, eventually we cut a release from Dev and push it to the UAT branch, cut a release from UAT to push to the prod branch. But these are separate branches because we might need to hotfix UAT or Prod. The process here can go in these two different directions, see, but one of them leads to a shared reality, this is the entirety of the code and all of the different ways that it can be configured in our production environment, and we need to carefully consider how we remove those feature toggles—versus the other one has three independent realities and nobody is considering all of the different ways that it can be configured or what is being removed, and periodically you get bugs because people didn't revert the revert—what, you didn't know that you needed to revert the revert, you always need to revert the revert. So process tends to be more lightweight if it generates one shared reality).
General principle 4: process needs to help you figure out, and keep highlighted, the “Useless Without.” (There are many nice-to-haves, in a given project. There are a lot of them that people will say are must-haves. The product must be secure, the product must be available at this website address, okay fine. But there is one business goal that the project serves, which, if that business goal is not accomplished, the whole project is useless. That is the Useless Without feature. So I worked on a shop floor system of kiosks for like 6 months once before I determined from talking to the stakeholders that the thing was actually Useless Without time tracking, and this is a sensitive issue because unionized pipefitters are understandably skittish around surveillance technology that could be used in dystopian ways. But we're going to address their needs by looking at the project only, trying to figure out how long each of the steps in building the project takes, but we still don't talk about how we're trying to make the shop floor run efficiently. But you understand every meeting I had before we had clarified this, was actively detrimental to my productivity on this task.)
> But, as we now know, the results were also wrong. Adjusting for inflation, world GDP is now about five times higher than it was in 1970 and continues to rise. More than 90 percent of that growth has come from Asia, Europe, and North America, but forest cover across those regions has increased, up 2.6 percent since 1990 to over 2.3 billion hectares in 2020. The death rate from air pollution has almost halved in the same period, from 185 per 100,000 in 1990 to 100 in 2021. According to the model, none of this should have been possible.
Okay, forest cover increasing and death rate from air pollution decreasing contradicts the prediction from 1970, but I feel these trends are a result of richer countries being able to outsource their environmental pollution/destruction to the global south and scavenging it for raw materials. I wonder if the forest cover statistics count the massive expanses of land deforested and replaced with monoculture plantations (palm oil, etc.) that end up having a giant effect on biodiversity (and will no doubt come back to bite us in the ass)? Even if this outsourcing of externalities couldn't be modeled 50 years ago, I feel like that doesn't detract from the spirit of the takeaways from the Club of Rome and The Limits to Growth.
> "The Club of Rome asked an even more intricate question: how would social and economic forces interact in the coming decades? Where were the bottlenecks and feedback mechanisms? Could economic growth continue, or would the world enter a new phase of equilibrium or decline?"
The problem is, as systems grow more complex they often start to demonstrate sensitive dependence on conditions, eg with tiny variations in inputs to one node of the system resulting in wild swings in outputs from that node. Equally problematic, nodes in a complex system can change their connectivity to other nodes if conditions change enough (think of a breakdown in trade between nations due to wars, natural disasters, diseases etc).
The ideal systems to depend on are stable (not hypersensitive to small forcings, with predictable behavior) and have consistent structure. They can still be complicated but should fail gracefully back to simpler structures under stress, eg an emergency power supply for electricity at a hospital that normally relies on the grid.
From this perspective, our electrical grids are well-designed systems - not given to huge power fluctuations - that will nevertheless need major expansions and improvements if electricity demand keeps rising with data centers and eVs etc. However, expanding the grid isn't adding fundamental instabilities, it's just modular addition in the same pattern as the existing system.
In contrast, the USA's current financial-monetary system is not that stable, predictable, or reliable. All kinds of fundamental instabilities exist, and wild swings in behavior under pressure are expected - and since everything else relies on it, eg you can't update the electrical grid without capital input, you risk avalanching catastrophes by relying on such an unstable system.
Having said that, there is no magical thinking in asserting that the government is bad at software. The government is bad at 99% of the things it does, but that 1% is keeping stuff running, despite the government trying really hard to fail at that 1%, too.
Stupid comment, I know.
Why was there a sharp turn to waterfall-like thinking? Why did we lose the learnings?
Does Forrester’s model get updated and re-run?
Of course, software people have not had much success lately fixing physical/bureaucratic systems.