Posted by r4um 18 hours ago
> you lay out a huge specification that would fully work through all of the complexity in advance, then build it.
But I do still think there's a lot of value into coming up with a good plan before jumping in. A lot of software people like to jump in and I see them portray the planning people as trying to figure everything out first. (I wonder if we reinforce the jumping in head first mentality because people figure out you can't plan everything) A good plan helps you prevent changing specs and prepares you for hiccups. It helps by having others but basically all you do is try to think of all the things that could go wrong. Write them down. Triage. If needed, elevate questions to the decision makers. Try a few small scale tests. Then build out. But building out you're always going to find things you didn't see. You can't plan forever because you'll never solve the unknown unknowns until you build, but also good prep makes for smoother processes. It's the reason engineers do the math before they build a bridge. Not because the math is a perfect representation and things won't change (despite common belief, it's not static) but because the plan is cheaper than the build and having a plan allows you to better track changes and helps you determine how off the rails you've gone.
It is also perplexing to me that people think they can just plan everything out and give it to a LLMs. Do you really believe your manager knows everything that needs to be done when they assign jobs to you? Of course not, they couldn't. Half the job is figuring out what the actual requirements are.
> This has never happened and never will. You simply are not omniscient. Even if you're smart enough to figure everything out the requirements will change underneath you.
I am one of those "battle-scarred twenty-year+ vets" mentioned in the article, currently working on a large project for a multinational company that requires everything to be specified up-front, planned on JIRA, estimates provided and Gantt charts setup before they even sign the contract for the next milestone.
I've worked on this project for 18 months, and I can count on zero hands the times a milestone hasn't gone off the rails due to unforeseen problems, last-minute changes and incomplete specifications. It has been an growing headache for the engineers that have to deliver within these rigid structures, and it's now got to the point that management itself has noticed and is trying to convince the big bosses we need a more agile and iterative approach.
Anyone who claims upfront specs are the solution to all the complexity of software either has no real world experience, or is so far removed from actual engineering they just don't know what they're talking about.
Nothing will get you to hit every milestone. However you can make progress if you have years of experience in that project and the company is willing to invest in the needed time to make things better (they rarely are)
My approach, especially for a project with a lot of unknowns, is usually to jump in right away and try to build a prototype. Then iterate a few times. If it's a small enough thing, a few iterations is enough to have a good result.
If it's something bigger, this is the point where it's worth doing some planning, as many of the problems have already been surfaced, and the problem is much better understood.
And things like "race conditions"/lack of scalability due to improper threading architecture aren't especially easy to fix(!)..
Also, there's a certain point where you can't avoid management sabotaging things.
Of course, it requires some discipline to not just yolo the prototype into production when that’s not appropriate.
It's sort of the old General Eisenhower quote: "In preparing for battle I have always found that plans are useless, but planning is indispensable."
I discussed some of this in https://www.ebiester.com/agile/2023/04/22/what-agile-alterna... and it gives a little bit of history of the methods.
We are nearly 70 years into this discussion at this point. I'm sure Grace Hopper and John Mauchly were having discussions about this around UNIVAC programs.
> But I do still think there's a lot of value into coming up with a good plan before jumping in.
Definitely, with emphasis on a _good_ plan. Most "plans" are bad and don't deserve that name.
> be specified up-front, planned on JIRA
Making a plan up-front is a good approach. A specification should be part of that plan. One should be ready to adapt it when needed during execution, but one should also strive to make the spec good enough to avoid changing.
HOWEVER, the "up-front specification" you mentioned was likely written _before_ making a plan, which is a bad approach. It was probably written as part of something that was called "planning" and has nothing to do with actual planning. In that case, the spec is pure fiction.
> estimates provided
Unless this project is exceptional, the estimates are probably fiction too.
> and Gantt charts setup
Gantt charts are a model, not a plan. Modeling is good; it gives you insight into the project. But a model should not be confused with a plan. It is just one tiny fragment you need to build a plan, and Gantt charts are just one of many many many types of models needed to build a plan.
> before they even sign the contract for the next milestone
That's a good thing. Signing a contract is an irreversible decision. The only contract that should be signed before planning is done is the contract that employs the planners.
> Anyone who claims upfront specs are the solution
See bove. A rigid upfront spec is usually not a plan, but pure fiction.
> My approach, especially for a project with a lot of unknowns, is usually to jump in right away and try to build a prototype.
Whether this is called planning or "jumping in" is a difference in terminology, not in the approach. The relevant clue is that you are experimenting with the problem to understand it, but you are NOT making irreversible decisions. By the terminology used in that book, you are _planning_, not _executing_.
> after the 2000 pages specification document was written, and passed down from the architects to the devs
If the 2000 page spec has never been passed to the devs while writing it, it's not part of a plan, it's pure fiction. Trying to develop software against that spec is part of planning.
You need smaller documents - this is the core technology we are using. This is how one subsystem is designed - often this should be on a whiteboard because once you get into the implementation details you need to change the plan, but the planning was useful. This is how to use core parts of the system so new comers can start working quick.
You need disciple to accept that sometimes libfoo is the best way to solve a problem in isolation, but since libbar is used elsewhere and can solve the problem your local problem will use libbar despite making your local problem uglier. Have a small set of core technologies that everyone knows and uses is sometimes more valuable than using the best tool for the job - but only sometimes.
My best project to date was a largely waterfall one - there was somewhere around 50-60 pages of A4 specs, a lot of which I helped the clients engineer. As with all plans, a lot of it changed during implementation, actually I figured out a way of implementing the same functionality, but automating it to a degree where about 15 of those could be cut out.
Furthermore, it was immensely useful because by the time I actually started writing code, most of the questions that needed answers and would alter how it should be developed had already come up and could be resolved, in addition to me already knowing about some edge cases (at least when it came to how the domain translates into technology) and how the overall thing should work and look.
Contrast that to some cases where you're just asked to join a project and help out and you jump into the middle of ongoing development, not going that much about any given system or the various things that the team has been focusing on in the past few weeks or months.
> It’s not hard to see that if they had a few really big systems, then a great number of their problems would disappear. The inconsistencies between data, security, operations, quality, and access were huge across all of those disconnected projects. Some systems were up-to-date, some were ancient. Some worked well, some were barely functional. With way fewer systems, a lot of these self-inflicted problems would just go away.
Also this reminds me of https://calpaterson.com/bank-python.html
In particular, this bit:
> Barbara has multiple "rings", or namespaces, but the default ring is more or less a single, global, object database for the entire bank. From the default ring you can pull out trade data, instrument data (as above), market data and so on. A huge fraction, the majority, of data used day-to-day comes out of Barbara.
> Applications also commonly store their internal state in Barbara - writing dataclasses straight in and out with only very simple locking and transactions (if any). There is no filesystem available to Minerva scripts and the little bits of data that scripts pick up has to be put into Barbara.
I know that we might normally think that fewer systems might mean something along the lines of fewer microservices and more monoliths, but it was so very interesting to read about a case of it being taken to the max - "Oh yeah, this system is our distributed database, file storage, source code manager, CI/CD environment, as well as web server. Oh, and there's also a proprietary IDE."
But no matter the project or system, I think being able to fit all of it in your head (at least on a conceptual level) is immensely helpful, the same way how having a more complete plan ahead of time can be helpful with a wide variety of assumptions vs "we'll decide in the next sprint".
And by doing this sort of exercise, you can avoid wasting time on dead ends, bad design, and directionless implementation. It's okay if requirements change or you discover something later on that requires rethinking. The point is to make your thinking more robust. You can always amend a design document and fill in relevant details later.
Furthermore, a mature design begins with the assumption that requirements (whether actual requirements or knowledge of them) may change. That will inform a design where you don't paint yourself into a corner, that is flexible enough to be adapted (naturally, if requirements change too dramatically, then we're not really talking about adaptation of a product, but a whole new product).
How much upfront design work you should do will depend on the project, of course. So there's a middle way between the caricature of waterfall and the caricature of agile.
There is a related phenomenon in some types of software where the cost of building an operational prototype asymptotically converges on the cost of just writing the production code. (This is always a fun one to explain to management that think building a prototype massively reduces delivery risk.)
Some projects have been forced so far, by diverting resources (either public-funded or not-yet-profitable VC money), but these efforts have not proven to be self-sustaining. Humans will be perpetually stuck where we are as a species if we cannot integrate the currently opposing ideas of up-front planning vs. move fast and break things.
Society is slowly realizing the step-change in difficulty between projects in controlled conditions that can have simplified models to these irreducibly complex systems. Western doctors are facing an interesting parallel, now becoming more aware to treat human beings in the same way--that we emerge as a result of parts which can be simplified and understood, but could never describe the overall system behavior. We are good examples of the intrinsic fault-tolerance required for such systems to remain stable.
If you are doing a CRUD web app for a local small business - there are thousands of examples. If you are writing control software for a space station - you may not have access to code from NASA/Russia/China but you can at least look at generic software that does the things you need and learn some lessons.
A system of services that interact, where many of them are depending on each other in informal ways may be a complex system. Especially if humans are also involved.
Such a system is not something you design. You just happen to find yourself in it. Like the road to hell, the road to a complex system is paved with good intentions.
If the definition of "complex" is instead something more like "a system of services that interact", "prone to multiple, coincidental failures", then I don't think it's impossible to design them. It's just very hard. Manufacturing lines would be examples, they are certainly designed.
The design of the manufacturing lines and the resulting supply chain are not independent of each other -- you can trace features from one to the other -- but you cannot take apart the supply chain and analyze the designs of its constituent manufacturing lines and actually predict the behavior of the larger system.
AFAIK there's not a great definition of a complex system, just a set of traits that tend to indicate you're looking at one. Non-linearity, feedbacks, lack of predictability, resistance to analysis (the "you can't take it apart to reason about the whole" characteristic mentioned above"). All of these traits are also kind of the same things... they tend to come bundled with one another.
(And no, this is not "my" definition, it's how it's defined in the systems-related disciplines.)
The set of system designs that exhibit naturally stable behavior doesn't overlap much with the set of system designs that deliver maximum performance and efficiency. The capability gap between the two can be large but most people choose easy/simple.
There is an enormous amount of low-hanging opportunity here but most people, including engineers, struggle with systems thinking.
The law is maybe a little too simplistic in its formulation, but it's fundamentally true.
Care to exemplify?
Point is though eventually some system runs out of ability. It works different in programming from physical construction, but the concept is the same, eventually you can't make a bad early design work anymore.
See also: "there is nothing more permanent than a temporary solution"
In this sense, web applications haven't changed so much in the last twenty years: client, server, database...
Not sure why you're trying to bring AI development into this.
The first is too ambitious and ends in an unmaintainable pile around a good core idea.
The second tries to "get everything right" and suffers second system syndrome.
The third gets it right but now for a bunch of central business needs. You learned after all. It is good exactly because it does not try to get _everything_ right like the second did.
The fourth patches up some more features to scoop up B and C prios and calls it a day.
Sometimes, often in BigCorp: Creators move on and it will slowly deteriorate from being maintenaned...
That's very simple. The balanced path depends directly on how much of the requirements and assumptions are going to change during the life time of the thing you are building.
Engineering is helpful only to the extent you can forsee the future changes. Anything beyond that requires evolution.
You are able to comment on the complexity of that large company only because you are standing in the future into 50 years from when those things started take shape. If you were designing it 50 years back, you would end up with same complexity.
The nature's answer to it is, consolidate and compact. Everything that falls onto earth gets compacted into a solid rock over time, by a huge pressure of weight. All complexity and features are flattened out. Companies undergo similar dynamics driven by pressures over time, not by big-bang engineering design upfront.
2. I would be more receptive to this argument if they had listed some famous examples of successful, large systems that were built like this. On the other hand, I can easily list many failures: FAA Advanced Automation System (1980s), IRS Tax Systems Modernization (1990s), UK NHS National Programme for IT (2000s).
3. Waterfall vs. agile is a continuum. Nobody plans everything, down to each if-statement, and nobody wings it without some kind of planned architecture (even if just inside one person's head). Where you are on the continuum depends on the nature of the problem (are all requirements known?), the nature of the team (have they done this before?), and the criteria for success (are there lives depending on this?).
4. The analogy to building a building is flawed. At large enough scale, software is like a city, and all successful cities have gradually evolved in complexity. Come back to me when someone builds a 1-million person arcology on some island in the Pacific.
5. Just as some PhDs are sensitive about being called "Doctor", some software engineers are sensitive about being "real engineers". Stop thinking about that. What we do as software engineers is immensely valuable and literally changing the world (usually, but not always, for the better). Let's stop worrying about whether or not what we do is "engineering" and focus on what we do best: building complex systems that have never before existed on earth.
The core point they're trying to make is that agile (or similar) practices are the incorrect way to approach consolidation of smaller systems into bigger ones when the overall system already works and is very large.
I agree with their assertion that being forced to address difficult problems earlier on in the process results in ultimately better outcomes, but I think it ignores the reality that properly planning a re-write of monumentally sized and already in use system is practically impossible.
It takes a long time (years?) to understand and plan all the essential details, but in the interim the systems you're wanting to rewrite are evolving and some parts of the plan you thought you had completed are no longer correct. In essence, the goal posts keep shifting.
In this light, strangler fig pattern is probably the pragmatic approach for many of these re-writes. It's impossible to understand everything up front, so understand what you reasonably can for now, act on that, deliver something that works and adds value, then rinse and repeat. The problem is that for sufficiently large system, this will take decades and few software architects stick around at a single company long enough to see it through.
A final remark I want to make is that, after only a few years of being a full-time software developer, "writing code" is one of the easiest parts of the job. The hard part is knowing what code needs to be written, this requires skills in effective communication with various people, including other software developers and (probably more importantly) non-technical people who understand how the business processes actually need to work. If you want to be a great software developer, learn how to be good at this.
I highly applaud this idea. IMO this is why big upfront design is so risky.
> The most prevalent one, these days, is that you gradually evolve the complexity over time. You start small and keep adding to it.
> The other school is that you lay out a huge specification that would fully work through all of the complexity in advance, then build it.
I think AI will drive an interesting shift in how people build software. We'll see a move toward creating and iterating on specifications rather than implementations themselves.
In a sense, a specification is the most compact definition of your software possible. The knowledge density per "line" is much higher than in any programming language. This makes specifications easier to read, reason about, and iterate on—whether with AI or with peers.
I can imagine open source projects that will revolve entirely around specifications, not implementations. These specs could be discussed, with people contributing thoughts instead of pull requests. The more articulated the idea, the higher its chance of being "merged" into the working specification. For maintainers, reviewing "idea merge requests" and discussing them with AI assistants before updating the spec would be easier than reviewing code.
Specifications could be versioned just like software implementations, with running versions and stable releases. They could include addendums listing platform-specific caveats or library recommendations. With a good spec, developers could build their own tools in any language. One would be able to get a new version of the spec, diff it with the current one and ask AI to implement the difference or discuss what is needed for you personally and what is not. Similarly, It would be easier to "patch" the specification with your own requirements than to modify ready-made software.
Interesting times.
We have yet to see a largely llm driven language implementation, but it is surely possible. I imagine it would be easier to tell the llm to instead translate the Java implementation to whatever language you need. A vibe-coded language could do major damage to a companies data.
[0] https://iceberg.apache.org/spec/ [1] https://lists.apache.org/thread/whbgoc325o99vm4b599f0g1owhgw...
This is a really good observation and I predict you will be correct.
There is a consequence of this for SaaS. You can imagine an example SaaS that one might need to vibecode to save money. The reason its not possible now is not because Claude can't do it, its because getting the right specs (like you suggested) is hard work. A well written spec will not only contain the best practices for that domain of software but also all the legal compliance BS that comes along with it.
With a proper specification that is also modular, I imagine we will be able to see more vibecoded SaaS.
Overall I think your prediction is really strong.
One issue is that a spec without a working reference implementation is essentially the same as a pull request that's never been successfully compiled. Generalization is good but you can't get away from actually doing the thing at the end of the day.
I've run into this issue with C++ templates before. Throw a type at a template that it hasn't previously been tested with and it can fall apart in new and exciting ways.
> The WHATWG was based on several core principles, (..) and that specifications need to be detailed enough that implementations can achieve complete interoperability without reverse-engineering each other.
But in my experience you need more than a spec, because an implementation is not just something that implements a spec, it is also the result of making many architectural choices in how the spec is implemented.
Also even with detailed specs AI still needs additional guidance. For example couple of weeks ago Cursor unleashed thousands of agents with access to web standards and the shared WPT test suite: the result was total nonsense.
So the future might rather be like a Russian doll of specs: start with a high-level system description, and then support it with finer-grained specs of parts of the system. This could go down all the way to the code itself: existing architectural patterns provide a spec for how to code a feature that is just a variation of such a pattern. Then whenever your system needs to do something new, you have to provide the code patterns for it. The AI is then relegated to its strength: applying existing patterns.
TLA+ has a concept of refinement, which is kind of what I described above as Russian dolls but only applied to TLA+ specs.
Here is a quote that describes the idea:
There is no fundamental distinction between specifications and implementations. We simply have specifications, some of which implement other specifications. A Java program can be viewed as a specification of a JVM (Java Virtual Machine) program, which can be viewed as a specification of an assembly language program, which can be viewed as a specification of an execution of the computer's machine instructions, which can be viewed as a specification of an execution of its register-transfer level design, and so on.
Source: https://cseweb.ucsd.edu/classes/sp05/cse128/ (chapter 1, last page)
> the size of the iterations matters, a whole lot. If they are tiny, it is because you are blindly stumbling forward. If you are not blindly stumbling forward, they should be longer, as it is more effective.
You are not blindly stumbling forward, you're moving from (working software + tiny change) to (working software including change). And repeat. If there's a problem, you learn about it immediately. To me that's the opposite of moving blindly.
> you really should stop and take stock after each iteration.
Who is not taking stock after every iteration? This is one of the fundamental principles of agile/lean/devops/XP/scrum. This one sentence drastically lowers my impression of the author's ability to comment on the subject.
> The faster people code, the more cleanup that is required. The longer you avoid cleaning it up, the worse it gets, on basically an exponential scale.
Unsafe tempo is as likely to happen in big-spec design projects as in small iterations. In fact, working in careful small iterations helps us manage a realistic tempo because we know we can't move faster than we can get things into production and evaluate.
The terrible outcomes listed in the same paragraph are linked to unwise practice and have nothing to do with small iteration size.
indeed, i would argue 'big iterations' are the ones where all the problems which the author mentions crop up in the first place!
That’s certainly my experience
In a more typical modern sense systems thinking is more about relationships and wholes, rather than isolating parts, which is the traditional engineering approach.
While much of the base material on systems thinking will be based around cybernetics, it is really a complement to traditional engineering, used in parallel to identity more natural complexity boundaries and to help avoid confusion and accidental complexity.
Gregor Hohpe’s Architect Elevator is probably a good place to start on why this change in perspective is important and why investing in flexibility is crucial when there is uncertainty.
While you may have to accept this article’s definition in some groups, accepting the more modern definition will help you get jobs in places that are nicer to work.
This type of false dichotomy that is presented in the article is a warning that there is soft work to be done.
People mentioning mechanical engineering in this thread are possibly the people who may benefit most from examining the material. I encourage you to see if this is a path forward for your needs.
The 1980s and 90s were full of DOD-497 multi-kilogram documents being analyzed atomically to determine the specification, and they rarely came in close any of the 3 main dimensions of success: time, quality, or cost.
On the other hand, neither has Agile with a capital A, with the ceremony of documents replaced with the ceremony of JIRA tickets and t-shirts.