It's not a vm: It's all namespaces, baby! Unshare one or more of your mnt, pid, net, ipc, uts, user, cgroup, or time namespaces and you have a "container".
Performance is basically irrelevant because you're not really interacting with the running application. You're running as root, so security-critical. You're talking to the internet for things like fetching images. You need rather complicated business logic to deal with all the possible options, protocols, and file formats.
Sure, you could do it in C, but dealing with dependencies is a major PITA and you're permanently one tiny mistake away from getting pwned, while not needing any of C's strengths. Go is safe, easy to write, and ships as a single batteries-included binary: what's not to like?
Go is good for writing web servers. For system software? Not so much. 90% of Unix is written with C. It's not that bad.
Podman has not required root in years, just saying.