I'm not sure I agree that systems are products though. The product mindset is toxic. It means that you've got goals which are independent of the user's goals (typically to make money, which sometimes means doing something dastardly to the users on behalf of the shareholders).
All the best software is more in the "tool" category and less in the "product" category. Usually it's made by the users, only bothers with solving problems they have, and has no ulterior motives.
Nuanced, but I’ve heard “don’t let perfection be the enemy of completion”, which is real in the cases of not shipping because one is constantly polishing/perfecting, or, as I heard sitting beside phk[0] at BSDCan commenting on rejecting some environmental sensors into FreeBSD, something to the effect of “We need ‘good’, not ‘good enough’.”, which draws a line at accepting janky “solutions” that are architecturally weak, or otherwise look obviously problematic.
Quality takes longer. Sometimes, significantly longer, for many reasons. That’s why something top-shelf, costs so much more than commodity, when it doesn’t seem to have much more, in the way of features.
Many orgs aren’t interested in spending that much extra time on it (which is their prerogative). The customer base tends to be smaller, and a lot more fickle. It’s really difficult to make a living, doing top-Quality work.
Source: I worked for many years, for a company known for top-shelf stuff. I’m quite familiar with Quality as an everyday feature.
Fixing critical bugs is not perfectionism. Fixing low-impact bugs by making high-risk, well-intentioned changes might be.
In a thirty year career of software engineering, I haven't once come across one of these mythical perfectionists that everyone is constantly retelling cautionary tales of, who endlessly rewrites perfectly serviceable software and never releases it. I have, however, worked at multiple companies that have collapsed under the weight of their unmaintainable spaghetti tech stacks.
So I said: screw performance, we are rewriting this thing into Go, as that's what our existing developers were willing to work with - with the intention of leaving when this would be done.
And the day I finally submitted my resignation letter, a competent C developer was found, and later I heard that the Go rewrite was scrapped.
I don't really feel it collapsing under the weight of unmaintainable spaghetti tech stacks, I'm mostly thinking that writing the missing features of the tech stack would require months of work and hard thinking that in the end might simply get rejected and now I would be stuck maintaining it myself so nothing has changed other than that there are even more things to maintain.
All of this for the sake of clean architecture that seemingly has no impact on users in the end... Like the users complain about completely different things.
Oh, I have. A LOT! In my friend group theres at least 5 of these right now. They work day jobs (that they dislike), they try to build a side hustle but never get it off the ground because they endlessly rewrite perfectly serviceable software and never release, the reason is:
- they believe that running a startup only requires writing a good product
- they avoid releasing because thats "judgement day" and might flop
- they think if they code it "just right" a money Waterfall will open up magically like a lottery ticket win and be instant success
- they believe that because theres the occasional exception to these rules above, it will happen to them too (everyone thinks they are the special exception,) because thats easier than accepting building a successful business takes discipline, hard work, and doing the tasks you dont want to do.
You havent come across them because by definition they never release and they dont share (they are embarrassed it might fail) so theres thousands and thousands of them, grinding away under false assumptions. They enjoy the act of building (which is totally cool!) but think thats all there is to it. Convinced that just one more rewrite is the thing holding them back from success - because they'd rather rewrite than talk to users. Millions of people doing this, millions of dreams that will die, because /just one more rewrite/
Just because you havent seen them, doesnt mean they aren't there
Latin amator, French amateur, means lover.
In this sense, it means somebody who is doing something because he/she loves it, not as his/her profession. It is not a large semantic shift to infer that he/she is not as skilled.
I think you're getting caught up on nomenclature and limited sample sizes.
If you're looking for somebody to design a screwdriver for you, you're better off with the guy who wants to drive a screw.
Like you hint at, though, a tool is useful immediately and without end (at a particular task - windows operating systems remain useful despite whatever current version is being sold), the product is often an attempt to continuously derive value from the tool (attach a time limited license, offer "upgrades", new features, etc.).
A service is more like a product than a tool - selling data is not something that a tool can (by itself) provide (unless you accept virtual or test data).
Attaching a product mindset to a tool then becomes a matter of complication/value reduction, vs a product mindset to a consumable, like a game, or dataset, where the value is at least constant, if not growing over time.
I.e., I'm not sure I agree that the "enshittification cycle" is the variable here - even in the age of AI the value e.g. of a stock image service is constant or growing, even if the LLM generation process dilutes the market.
I like this. The question is really whether you're facing the "business end of the stick" or not (although stick isn't quite right here, because not all software has both ends).
You are throwing the baby out with the bathwater by retreating to "tools". If you want to stop the enshittification, this is not the way. You're just opening yourself up to new scams. i.e. AI tools, political agendas, etc.
Better products are just better. Demand them.
- linux
- nix
- nushell
- helix
Are these products... at all? Do the open me up to new scams? I don't think so.
I'm not in a position to demand anything from the people who make these things. If they were subject to demands, their craft would be tainted by compromises made in acquiescence to those demands, and I'd probably be less enthusiastic about their software (because presumably, my tastes don't align with whichever others are also in a position to be making demands).
Supply and demand are well and good if what you're after is barley. But when you compare what there's demand for with what's being supplied re: software, there appears to be no correlation.
We gotta stop selling picks and shovels and start learning to be miners who have good taste in picks and shovels and the ability to make and remake our tools as needed. The disconnect is creating a hell for our users.
The worst clusterfuck I've ever worked on was a web application that was actually solving a real problem fairly well, but the team was spending time building an absurd Rube-Goldberg contraption of microservices when the entire platform had less MAU than my hobby website. It wasn't the wrong problem, but it certainly was over-engineered!
This is literally the example that I use, the most common case of over-engineering, having more microservices than team members. Microservices are the right solution for certain problems, but those were not problems they had.
Some anecdotal evidence. I worked in many of these places, and the most common tell of over-engineering is that when you ask "what problem were we solving when we decided to have all these many microservices?" the answers you will get is problems they either didn't have (for example, high availability) or they state a problem they actually had but could have been solved in the monolith.
In other words, they "overshoot" and - as I write in the post - end up with "a system that solves multiple problems partially, none of them completely, while introducing a bunch of problems you wouldn't have had otherwise."
Or more microservices than customers
Just one problem, actually: Conway's law.
When your organization needs to operate like the macro economy, where independent groups provide services for each other, then you are going to see the same structure found in the macro economy mirrored in the microcosm of that single organization. Hence the name microservices. Same as services, except not across business lines.
At Google scale you have no choice but to have an internal economy, but trying to build an entire economy inside a small business with less activity than a personal blog is crazy.
Such a lovely analogy. FYI it’s mine now.
While I don't actually agree with OP, I do agree with their sentiment. I've seen people say something is "over engineered" when there's an elegant design. Elegance isn't over engineering, it is solving problems effectively. It's something we should chase! Elegance is solving the right problem, which usually people are having a hard time seeing. (It's not always easy)
If we constantly let people drag quality down then we get into this frustrating world where everything is constantly half broken.
I think these both fall comfortable under the umbrella of "solving the wrong problem"
It's said to head off a specific complaint from a specific kind engineer who will object that the proposed solution won't work because it doesn't cover some obscure edge case which rarely comes up in production.
"We're not trying to build a perfect solution here" is saying "We acknowledge not everything will be covered, we're setting the requirements at the 90th percentile use case".
Additionally, as I get older, I find the sentiment of "we're not trying to build a perfect system here" is less about "let's just go fast vroooom" and more akin to saying "I've been humbled before by thinking I had the perfect mental model of the universe before a single user touched the product."
Then engineers are on the hook because they run into those constraints while building and everyone always blames „those lazy software developers” ;)
At work we agreed that some use cases are very niche. These have guards in place to log an alerted-upon marker + return HTTP/500. They have not tripped for years by customers. So, it's fine to not support some rare cases and to deliberately leave known gaps in some contexts. As long as you don't close these paths forward if you need them.
On the other hand, we have contexts like our PostgreSQL instances. Those have a very well defined scope and rooting out all known problems has been the right choice. Most issues we have ignored in that scope have bitten us in the butt sooner rather than later. Very hard in some cases, I may add.
Realizing this about a domain is very important.
No. It is cutting unused features to make a manageable product.
* "That won't work on GCC 5.5", we don't support GCC 5.5, or any compiler which wasn't shipped this decade.
* "What about FreeBSD?" What about it? We only have Linux servers
* "This only works on systemd." Good, we're a systemd shop.
In online discussions people always blame „lazy developers „ like there would be no product owners, testers, business analysts, scrum masters etc.
In my space this is usually something like, "X won't work on POSIX make", being a reason not to add X to the build system. Well, it works on Ninja, and on GNU make, so it just won't work on POSIX make and the handful of deployments still using POSIX make just won't use X.
I'm ok with this as long as dropped requirements are documented and nobody wakes you up at night when those edge cases show up in production. Also, when the next feature needs to build on top of those dropped edge cases, you are given enough resources to redo the previous solution.
Striving towards perfection too often leads to yes over-engineering but I think more importantly it also: leads to way too much bike shedding due to the need to be perfect and allot of even emotional baggage when we don't make that perfect things.
Even the author admits that their definition of perfect can only arise with stringent requirements and I'll take it a step further, maybe it was perfect for that problem at that time but guess what change happens all the time, and as architects we have to think about future problems as much as current ones. With that in mind I think that having something that is good in many scenarios is better than the thing that is perfect in the scenario you start in.
Most new product launches are an exercise in figuring out what the product requirements should be through trial and error (really: through ongoing dialogue with your users). Even mature products can have requirements change over time as the market changes.
I think internalizing this reality is why most senior engineers who work in domains that touch the messy real world will reflexively push back against perfectionism.
Sure, but these have to be well-defined trials. In other words, yes, you will test several hypotheses, but your hypotheses have to be hypotheses, not hand waving.
Over-complicated is adding too many features, too much mechanism, too many moving parts. Over-engineering is far exceeding the requirements in an unhelpful way. For example: build me a treehouse. Said treehouse could probably be made of wood. If you made it out of concrete and steel, it might be a heck of a lot stronger and last a heck of a lot longer, without being more complicated (just more expensive). Too strong is over-engineered. If you made a treehouse with 13 bedrooms and an elevator, glass windows, solar power, and running water, that's overcomplicated.
Sometimes overbuilding (e.g. using really thick wood on a treehouse) means you can lower complexity (less bracing needed) at the cost of being more wasteful with material. That is often a great tradeoff.
- Antoine de Saint-ExupéryThe “honest” part is the important one. It’s also the hardest. It’s so tempting to project preferences onto a new development, which tends to confuse the distinction between desire and necessity. There’s so many things we want to try. Maybe having lots of personal projects to vent the need for novelty helps one to make good choices when it’s critical that they must be right.
I think this is a bit of a generous reframing of what seeking perfection is. Generally I think perfection seeking is best described as over-focusing on the details and pre-planning instead of laying a general blueprint that leaves room for pivots, future decisions, and iteration along the way. There's a whole breed of engineers that are really great thinkers but get stuck in the mud trying to pre-think the best way to do something instead of being adaptable.