If you spend more time with a product, you’re more likely to choose to do it again, even if it’s because of failure or annoyance. You’d justify it somehow (I have a leg up now or something). This actually applies to looking at things as well; a brightly colored box on the supermarket shelf is simply more likely to be chosen because you look at it first and longer.

Wasting your time and resources is a power signifier, but getting you to waste your own time and your own resources is hegemony.

"Appeal to adult animated show" is lame enough on Reddit but it's particularly odious blogspam here.

See also "normal accidents" [1], which are an even bigger worst case scenario, which we already struggle with, and will only get worse.

"Normal accidents, or system accidents, are... inevitable in extremely complex systems. Given the characteristic of the system involved, multiple failures that interact with each other will occur, despite efforts to avoid them... while operator error is a very common problem, many failures relate to organizations rather than technology, and major accidents almost always have very small beginnings. Such events appear trivial to begin with before unpredictably cascading through the system to create a large event with severe consequences." [1]

[1] https://en.wikipedia.org/wiki/Normal_Accidents

That’s not even an attempt to be charitable to jev. It can make decisions from unstructured data. Nobody uses anything just because it is cheap…

Another of the ways that software "engineering" is drifting even further away from traditional engineering. Failure mode analysis? Root cause analysis? I have no idea, I just talk to this magic box.

That was awfully specific. Apple has managed to normalize inexplicable failures of things that had been working before for over a decade, across all of their apps and the OS. And they didn't even need AI for that. But AI will definitely speed up the rate of blunder everywhere.

This is the problem building "it does everything" machines.

If you are pitching that your service can do potentially "whatever the client wants" you have such a thin basis on which to provide contracts and guarantees as a provider. The narrower the function, the clearer you can be about what's supposed to happen and why things might have gone wrong.

When you're using probabilities as the fundamental approach to computation, all of that goes out the window. Nondeterminism is powerful because it's insanely flexible, but the cost of that flexibility is predictability and expectation. Determinism was humanity's primary choice for formalisms and technology precisely because it reduces complex problems and situations to repeatable mechanics that are easy to understand. Deterministic tools can't do a lot in the grand scheme of things, but it is precisely these limitations that make them work well in concert and keep them comprehensible.

Inexplicable has a range from “I can’t explain that” to “that is not in the realm of possibility”. Windows failures especially have been steadily moving from A (already pretty sad) to B in the past years and I’ve been thinking about how utterly unempowering that is. Tools stop being tools if they have a secret life of their own.

Excellent post!

I've been saying this for years – it's a huge societal problem that most people, including powerful people, view technology in general and software in particular as something slightly magical, something that cannot be truly understood. We're seemingly forever doomed to accept that technical things are a bit shitty and that we'll have to do a rain dance every now and then to appease them. Instead of, you know, throwing out the shitty products and making or buying good ones.

Lovely post, and lovely phrase (normalizing of inexplicable failures).

It frustrates me quite a bit that the best solution to many issues is "Just try again tomorrow", trusting that the problem occurring on Azure is caused by some obscure backend service being down six timezones from our office, and the thing™ will start working again in a few days when Microsoft fixes said backend service.