Posted by codesmash 9/5/2025
Podman with pods is a better experience than docker-compose. It's easy to interactively create a pod and add containers to it. The containers ports will behave as if they were on the same machine. Then `podman generate kube` and you have a yaml file that you can run with `podman kube play`.
Rootless networking is very slow unless you install `passt`. With Debian, you probably should install every optional package that podman recommends.
The documentation is lacking. Officially, it's mostly man pages, with a few blog posts announcing features, though the posts are often out of date.
Podman with its docker socket is often compatible with Docker. Even docker-compose can (usually) work with podman. I've had a few failures, though.
Gitlab-runner can use podman instead of docker, but in this case the is no network aliases. So it's useless if the runner needs to orchestrate several images (e.g. code and db).
If the software that you're running inside the container supports it, you can use socket activation [0] to get native performance.
[0]: https://github.com/containers/podman/blob/main/docs/tutorial...
I came across just how slow recently:
- Container -> host: 0.398 Gbps vs. 42.2 Gbps
- host -> container: 20.6 Gbps vs 47.4 Gbps
Source: https://github.com/containerd/nerdctl/blob/main/docs/rootles...
Rootless works great, though there are some (many) images that will need to be tweaked out of the box.
Daemonless works great as well. You can still mount podman.sock like you can with Docker's docker.sock, but systemd handles dynamically generating the UNIX socket on connect() which is a much better solution than having the socket live persistently.
The only thing that I prefer Docker for is Compose. Podman has podman-compose, which works well and is much leaner than the incumbent, but it's kind of a reverse-engineered version of Docker Compose that doesn't support the full spec. (I had issues with service conditions, for example).
I'd guess that's because "the spec" is more .jsonschema than a spec about what behaviors any random version should do. And I say "version" because they say "was introduced in version $foo" but they also now go out of their way to say that the file declaring what version it conforms to is a warning
1. ssh orb (or machine name if you have multiple) 2. sudo apk add docker docker-cli-compose (install docker) 3. sudo addgroup <username> docker (add user to docker group) 4. sudo rc-update add docker default (set docker to start on startup)
Bonus, add lazydocker to manage your docker containers in a console
1. sudo apk add lazydocker
The benefit is that, Alpine has access to all your local and network drives so you can use them. You can sandbox them as well. It's not a big learning curve, just a good VM with access to all drives but isolated to local only.
You might expect that setting User=foo via systemd would enable seamless rootless containers, but it turns out to be a hard problem without a seamless solution.
Instead, there's this discussion thread with 86 comments and counting to wade through to find some solutions that have worked for some people in some cases.
https://github.com/containers/podman/discussions/20573#discu...
User=per-service-user
ExecStart=!podman-wrapper ...
Remove layers, keep things simple.
That being said, it is here to stay. So any alternative tooling that forces Docker to get it's act together is welcome.
> Remove layers, keep things simple.
Due to the first line above, I'm not sure if I'm reading the second line correctly. But I'm going to assume that you're referring to the OCI image layers. I feel your pain. But honestly, I don't think that image layers are such a bad idea. It's just that the best practices for those layers are not well defined and some of the early tooling propagated some sub-optimal uses of those layers.
I'll just start with when you might find layers useful. Flatpak's sandboxing engine is bubblewrap (bwrap). It's also a container runtime that uses namespaces, cgroups and seccomp like OCI runtimes do. The difference is that it has more secure seccomp defaults and it doesn't use layers (though mounts are available). I have a tool that uses bwrap to create isolated build and packaging environments. It has a single root fs image (no layers). There are two annoyances with a single layer like this:
1. If you have separate environments for multiple applications/packages, you may want to share the base OS filesystem. You instead end up replicating the same file system redundantly.
2. If you want to collect the artifacts from each step (like source download, extract and build, 'make install', etc) into a separate directory/archive, you'll find yourself reaching out for layers.
I have implemented this and the solutions look almost identical to what OCI runtimes do with OCI image layers - use either overlayfs or btrfs/zfs subvolume mounts.
So if that's the case, then what's the problem with layers? Here are a few:
1. Some tools like the image builders that use Dockerfile/Containerfile create a separate layer for every operation. Some layers are empty (WORKDIR, CMD, etc). But others may contain the results of a single RUN command. This is very unnecessary and the work-arounds are inelegant. You'll need to use caches to remove temporary artifacts, and chain shell commands into a single RUN command (using semicolons).
2. You can't manage layers like files. The chain of layers are managed by manifests and the entire thing needs a protocol, servers and clients to transfer images around. (There are ways to archive them. But it's so hackish.)
So, here are some solutions/mitigations:
1. There are other build tools like buildah and packer that don't create additional layers unless specified. Buildah, a sister project of Podman, is a very interesting tool. It uses regular (shell) commands to build the image. However, those commands closely resemble the Dockerfile commands, making it easy to learn. Thus you can write a shell script to build an image instead of a Dockerfile. It won't create additional layers unless you specify. It also has some nifty features not found in Dockerfiles.
Newer Dockerfile builders (I think buildkit) have options to avoid creating additional layers. Another option is to use dedicated tools to inspect those layers and split/merge them on demand.
2. While a protocol and client/servers are rather inconvenient for lugging images around, they did make themselves useful in other ways too. Container registries these days don't host just images. They can host any OCI artifact. And you can practically pack any sort of data into such an artifact. They are also used for hosting/transferring a lot of other artifacts like helm charts, OPA policies, kubectl plugins, argo templates, etc.
> So any alternative tooling that forces Docker to get it's act together is welcome
What else do you consider as some bad/sub-optimal design choices of Docker? (including those already solved by podman)
In general we do actually try to provide full context for errors from dockerd. Some things can be cryptic because, frankly, they are cryptic and require digging into what really happened (typical of errors from runc), but we do tend to wrap things so at least you know where the call site was.
There's also tracing data you can hook into, which could definitely be improved (some legacy issues around context propagation that need to be solved).
I've definitely seen, in the past, my fair share of errors that simply say "invalid argument" (typically this is a kernel message) without any context but have worked to inject context everywhere or do better at handling errors that we can.
So definitely interested in anything you've seen that could be improved because no one likes to get an error message that you can't understand.
But Docker is simply a non-starter. It's based on a highly privileged daemon with an enormous, hyper-complicated attack surface. It's a fundamentally bad architecture, and as far as I've been able to tell, it also comes from a project that's always shown an "Aw, shucks" attitude toward security. Nobody should be installing that anywhere, not even if there weren't an alternative.
When Docker Desktop changed licensing I tried to switch to Podman and it was a disaster, Podman was brand new and despite many blog posts that claimed it was the perfect replacement it did not work for me, and I have very simple requirements. So I ended up using Rancher Desktop instead, which was also very unstable but better.
Fast forward 1 year, Rancher was pretty good and Podman still did not work reliably on my mac.
Fast forward another year or so and I switched to colima.
I tried podman last time about one year ago and I still had issues on my old mac. So far colima has been good enough for my needs although at least two times a brew update broke colima and I had to reinstall from scratch.
Podman has a number of caveats that make it not a drop in replacement out of the box, but they are mostly few and far between. Once you've learned to recognize a handful of common issues, it's quite simple to migrate.
This might sound like me trying to explain away some of the shortcomings of podman, but honestly, from my point of view, podman does it better, and the workarounds and fixes I make to our scripts and code for podman are backwards compatible and I view them as improvements.
1. People hear about how great rootless is with Podman but then expect to be able to switch directly from rootful Docker to rootless Podman without changing anything. The only way that could work is if there was no difference between rootful and rootless to begin with, but people don't want to hear that. They combine these two selling points in their head and think they can get both a drop-in replacement for Docker and also rootless by default. The proper choice is to either switch from rootful Docker to rootful Podman *or* put in the work to make your container work in rootless, work you would also have had to do with rootless Docker.
2. Docker Compose started out as an external third-party add-on (v1) which was later rewritten as an internal facility (v2) but `podman compose` calls out to either `docker-compose` (i.e. v1) or to its own clone of the same mechanism, `podman-compose`. The upshot is a lot of impedance mismatch. Combine that with the fact that Podman wants you to use Quadlets anyway, resulting in less incentive to work on these corner cases.
3. Docker has always tried to pretend SELinux doesn't exist, either by hosting on Debian and friends or by banging things into place by using their privileged (rootful) position. Podman comes from Red Hat, and until recently they had Mr SELinux on the team. Thus Podman is SELinux-first, all of which combines to confuse transplants who think they can go on ignoring SELinux.
4. On macOS and Windows, both Podman and Docker need a background Linux VM to provide the kernel, without which they cannot do LXC-type things. These VMs are not set up precisely the same way, which produces migration issues when someone is depending on exact details of the underlying VM. One common case is that they differ in how they handle file sharing with the host.
https://tangentsoft.com/podman/wiki?name=Not%20a%20Drop-In%2...
Thanks!
https://tangentsoft.com/podman/wiki?name=Not%20a%20Drop-In%2...