Posted by canto 1 day ago
I've seen so many projects bending over backwards to avoid k8s and pay large cloud bills to avoid it at all costs. (ECS and app services are hopelessly expensive and bothersome)
K8s is really good, pretty easy to maintain, but a bit hard to understand. Mostly because distributed, zero down time systems are a bit hard to get by nature. But if you have someone that wants to take it on I've managed k8s clusters, solo, without incident, while doing lots of other stuff too (working with larger teams now though). Not to mention there's a lot of competence out there that can take over if I'd move on. Most of the deep complexity comes with more advance use cases, that wont show up for smaller deployments.
That said, no h8 towards going your own way! If your a solo developer (or small team) for a smallish project, don't feel the absolute need. If you get to the point you need it you should be earning enough to start paying someone to help ya get your app to a distributed system like k8s.
I think it's good to invest the time in understanding k8s though as a professional. Even if you won't directly run it it teaches you a lot about how to think about distributed, zero downtime systems. And what requirements that puts on an app.
1. Retrying a non-idempotent request on a failure type that does indicate that no action was taken is not necessarily safe.
2. It’s possible and actually fairly common to design a backend that can do a clean shutdown: it stops accepting new requests, completes old requests, then exits. I sincerely hope that Docker’s tooling is good enough for a service to unregister itself before it actually stops accepting requests, but I’m not actually very familiar with using Docker to manage HTTP routing. (I use a home grown tool that is far simpler.)
https://notnotp.com/notes/what-job-interviews-taught-me-abou...
I still think almost all of my complaints were valid: - not just container orchestration, more like a whole AWS solution (its own routing)
- has single-points of failure that would be avoided if you just used aws
- unnecessary complexity for almost all use cases
- ecosystem easily allows you to 2x-8x that complexity (helm+argocd+istio+carpenter+dozens more=hundreds of new failure modes)
- fundamentally moves many teams to a "I don't understand why my service is crashing, so let's just bring up more nodes every time rather than ever learning to debug it" mentality.
- All the feature teams that are supposed to "own their own kubernetes implementation for their own services" never do.
Of course a lot of that frustration is more at working at a place with a poorly-operated SOA where it takes a half-dozen services to send an email to a client or send a text or something silly. It sure is a waste of runway to obsess over a SOA when you aren't profitable yet.
But at this point it's sort of a sunk cost because it's become the industry standard. And AI can help with 90% of the complexity, which is it's own yellow-flag, but here we are.
The reason my position has changed is because:
1. The tooling has gotten better for setting up and managing K8.
2. In two of the last 3 jobs where we opted for a simplified alternative to k8, we came to regret that decision within a couple of years of that decision being made. If you’re core architecture is changing on a timescale of months (not years) then you picked the wrong foundations to build from.
That all said, I still think there is a pragmatic decision that needs to be made. And if I were in the author of this articles position I probably wouldn’t have picked k8s for this task either, despite what I said above. But, and as I said in my comment dismissing this article, they are dealing with low traffic and none of the problems that lend themselves to the benefits of k8. So my criticism of this article is that it’s misleading because their problem is easy but they’re writing as if they’re having to deal With problems of scale when they’re actually not.
But yeah, pretty cool DNS resolving features in HAProxy, that's nifty
That isn’t a lot. You could easily run that from one host. The reason people reach for Kubernetes (and similar) is because they need to scale past that single host dependency.
It's great.
They’re multi-region, but that doesn’t mean they’re running across multiple hosts in each region.
Docker compose doesn’t support pooling multiple hosts, so if they are running multiple hosts per region then there’s a lot more complexity to their setup than they’re documenting in that blog. Even if that complexity is human toil managing each host as a separate entity.
I have some stuff on single-node k3s. Because it's standard so I don't have to care.
To be fair: using Kubernetes anyways builds the skill just in case you become one of the 0.1% who actually need it down the line.
K3S takes about 5 minutes to setup the first time and you instantly have an entire universe of standardized operational tooling. I wouldn't touch docker compose with a 20 foot pole for production work.
Setting up K8s isn't rocket science, but maintaining it are offputting, to say the least.
Which is exactly what is happening with us, too bad we didn't choose K8S from the get-go and stuck with a "simpler" tool (gaining very little in the process).
This shittake was probably valid 10y ago, I would have agreed with you back then
> The entire infra the vast majority of Kubernetes users have could run on a single bare metal machine
Where are you pulling this out of? A large number of k8s users don't need it, but the alternative you have sounds hyperbolic.
The readme covers connection draining with Traefik which should solve one of the issues the author mentions
These are things I'm trying to figure out at work using Podman. Would love to hear about any experience in these areas.
This is pretty small scale, Kubernetes comes in when you've got a larger workload.
“We don’t know how to scale Traefik so we went with haproxy”
Well doh. Haproxy is designed for this. You can make haproxy serve copious amount of traffic on a single arm core and a little bit of ram. Imagine what you can do with a few replicas on your large clusters.
This has nothing to do with the choice of CI/CD or docker versus kubernetes.
Over the past decade, I'm seeing k8s used everywhere for everything, companies setting up clusters to run literally one simple app with couple of hundred requests per hour.
the site is down for me.