Posted by tradertef 13 hours ago
I don't want to diss SQLite because it is awesome and more than adequate for many/most web apps but you can connect to Postgres (or any DB really) on localhost over a Unix domain socket and avoid nearly all of the overhead.
It's not much harder to use than SQLite, you get all of the Postgres features, it's easier to run reports or whatever on the live db from a different box, and much easier if it comes time to setup a read replica, HA, or run the DB on a different box from the app.
I don't think running Postgres on the same box as your app is the same class of optimistic over provisioning as setting up a kubernetes cluster.
What features postgres offers over sqlite in the context of running on a single machine with a monolithic app? Application functions [2] means you can extend it however you need with the same language you use to build your application. It also has a much better backup and replication story thanks to litestream [3].
- [1] https://andersmurphy.com/2025/12/02/100000-tps-over-a-billio...
- [2] https://sqlite.org/appfunc.html
- [3] https://litestream.io/
The main problem with sqlite is the defaults are not great and you should really use it with separate read and write connections where the application manages the write queue rather than letting sqlite handle it.
for inserts only into singe table with no indexes.
Also, I didn't get why sqlite was allowed to do batching and pgsql was not.
Actually, there are no inserts in this example each transaction in 2 updates with a logical transaction that can be rolled back (savepoint). So in raw terms you are talking 200k updates per second and 600k reads per second (as there's a 75%/25% read/write mix in that example). Also worth keeping in mind updates are slower than inserts.
> no indexes.
The tables have an index on the primary key with a billion rows. More indexes would add write amplification which would affect both databases negatively (likely PG more).
> Also, I didn't get why sqlite was allowed to do batching and pgsql was not.
Interactive transactions [1] are very hard to batch over a network. To get the same effect you'd have to limit PG to a single connection (deafeating the point of MVCC).
- [1] An interactive transaction is a transaction where you intermingle database queries and application logic (running on the application).
> - [1] An interactive transaction is a transaction where you intermingle database queries and application logic (running on the application).
could you give specific example why do you think SQlite can do batching and PG not?
The same thing SQL itself buys you: flexibility for unforeseen use cases and growth.
Your SQLite benchmark is based in having just one write connection for SQLite but all eight writable connections for Postgres. Even in the context of a single app, not everyone wants to be tied down that way, particularly when thinking how it might evolve.
If we know our app would not need to evolve we could really maximize performance and use a bespoke database instead of an rdbms.
It seems a little aggressive for you to jump on a comment about how it’s reasonable to run Postgres sometimes with “SQLite smokes it in performance.” That’s true, when you can accept its serious constraints.
As a wise man once said, “Postgres is great and there's nothing wrong with using it!”
SQLite on the same machine is akin to calling fwrite. That's fine. This is also a system constraint as it forces a one-database-per-instance design, with no data shared across nodes. This is fine if you're putting together a site for your neighborhood's mom and pop shop, but once you need to handle a request baseline beyond a few hundreds TPS and you need to serve traffic beyond your local region then you have no alternative other than to have more than one instance of your service running in parallel. You can continue to shoehorn your one-database-per-service pattern onto the design, but you're now compelled to find "clever" strategies to sync state across nodes.
Those who know better to not do "clever" simply slap a Postgres node and call it a day.
Actually 35% faster than fwrite [1].
> This is also a system constraint as it forces a one-database-per-instance design
You can scale incredibly far on a single node and have much better up time than github or anthropic. At this rate maybe even AWS/cloudflare.
> you need to serve traffic beyond your local region
Postgres still has a single node that can write. So most of the time you end up region sharding anyway. Sharding SQLite is straight forward.
> This is fine if you're putting together a site for your neighborhood's mom and pop shop, but once you need to handle a request baseline beyond a few hundreds TPS
It's actually pretty good for running a real time multiplayer app with a billion datapoints on a 5$ VPS [2]. There's nothing clever going on here, all the state is on the server and the backend is fast.
> but you're now compelled to find "clever" strategies to sync state across nodes.
That's the neat part you don't. Because, for most things that are not uplink limited (being a CDN, Netflix, Dropbox) a single node is all you need.
I feel like the advice from people with your experience is worth way way way way more than what you'd hear from big tech. Like what you said yourself, big tech tends to recommend extremely complicated systems that only seem worth maintaining if you have a trillion dollar monopoly behind it.
Deployment, caddy holds open incoming connections whilst your app drains the current request queue and restarts. This is all sub second and imperceptible. You can do fancier things than this with two version of the app running on the same box if that's your thing. In my case I can also hot patch the running app as it's the JVM.
Server hard drive failing etc you have a few options:
1. Spin up a new server/VPS and litestream the backup (the application automatically does this on start).
2. If your data is truly colossal have a warm backup VPS with a snapshot of the data so litestream has to stream less data.
Pretty easy to have 3 to 4 9s of availability this way (which is more than github, anthropic etc).
You seem terribly confused. Backups don't buy you high availability. At best, they buy you disaster recovery. If your node goes down in flames, your users don't continue to get service because you have an external HD with last week's db snapshots.
Streaming replication lets you spin up new nodes quickly with sub second dataloss in the event of anything happening to your server. It makes having a warm standby/failover trivial (if your dataset is large enough to warrant it).
If your backups are a week old snapshots, you have bigger problems to worry about than HA.
- When AWS/GCP goes down, how do most handle HA?
- When a database server goes down, how do most handle HA?
- When Cloudflare goes down, how do most handle HA?
The down time here is the server crashed, routing failed or some other issue with the host. You wait.
One may run pingdom or something to alert you.
This is a disingenuous scenario. SQLite doesn't buy you uptime if you deploy your app to AWS/GCP, and you can just as easily deploy a proper RDBMS such as postgres to a small provider/self-host.
Do you actually have any concrete scenario that supports your belief?
This is...not true of many hyperscaler outages? Frequently, outages will leave individual VMs running but affect only higher-order services typically used in more complex architectures. Folks running an SQLite on a EC2 often will not be affected.
And obviously, don't use us-east-1. This One Simple Trick can improve your HA story.
Nonsense. You can't outrun physics. The latency across the Atlantic is already ~100ms, and from the US to Asia Pacific can be ~300ms. If you are interested in performance and you need to shave off ~200ms in latency, you deploy an instance closer to your users. It makes absolutely no sense to frame the rationale around performance if your systems architecture imposes a massive performance penalty in networking just to shave a couple of ms in roundtrips to a data store. Absurd.
Not everyone needs monopolistic tech to do their work. There's probably less than 10,000 companies on earth that truly need to write 240k rows/second. For everyone else, we can focus on better things.
It's a static blog that renders markdown... there's literally nothing to code, let alone vibe code.
Running 100,000 `SELECT 1` queries:
PostgreSQL (localhost): 2.77 seconds
SQLite (in-memory): 0.07 seconds
Exactly. Back in the real world,anyone who is faced with that sort of usecase will simply add memory cache and not bother with the persistence layer.
Also:
> PostgreSQL (localhost): (. .) SQLite (in-memory):
This is a rather silly example. What do you expect to happen to your data when your node restarts?
Your example makes as much sense as comparing Valkey with Postgres and proceed to proclaim that the performance difference is not insignificant.
Running 100,000 `SELECT 1` queries:
PostgreSQL (localhost): 2.71 seconds
SQLite (in-memory): 0.07 seconds
SQLite (tempfile): 0.07 seconds
I hope you understand that your claim boils down to stating that SQLite is faster at doing nothing at all, which is a silly case to make.
Running 100,000 `SELECT 1` queries:
PostgreSQL (localhost): 2.84 seconds
PostgreSQL (Unix socket): 1.93 seconds
SQLite (in-memory): 0.07 seconds
SQLite (tempfile): 0.06 seconds
Isn't this idea to spend a bit more effort and overhead to get YAGNI features exactly what TFA argues against?
More features is a net negative if you don't need those features. Ideally you want your DB to support exactly what you need and nothing more. Not typically realistic but the closer you can get the better.
The thing is one you learn the technology, everything else seems more work than the "easy way".
What I like about sqlite is that it's simply one file
My experience with sqlite for server-based apps has been that as your app grows, you almost always eventually need something bigger than sqlite and need to migrate anyway. For a server-based app, where minimizing deployment complexity isn't an extremely important concern, and with mixed reads and writes, it's rarely a bad idea to use Postgres or MariaDB from the start. Yes there are niche scenarios where sqlite on the server might be better, but they're niche.
Saying "you can just run things on a cheap VPS" sounds amateurish: people are immediately out with "Yeah but scaling", "Yeah but high availability", "Yeah but backups", "Yeah but now you have to maintain it" arguments, that are basically regurgitated sales pitches for various cloud platforms. It's learned helplessness.
In my case I'm seeing it a lot on the front-end side. My clients end up with single-page apps that install Shadcn, Tailwind, React, React Router, Axios, Zod, React Form and Vite, all to center a some input elements and perform a few in-browser API calls. It's a huge maintenance burden even before they start getting value out of it.
These large setups are often a correct answer, but not the right one for the situation.
It's all of five minutes to write a deployment yaml and ingress and have literally anything on the web for a handful of dollars a month.
I've written rust services doing 5k QPS on DO's cheapest kube setup.
It's not rocket science.
Serverless node buns with vite reacts are more complicated than this.
Ten lines of static, repeatable, versioned yaml config vs a web based click by click deploy installer with JavaScript build pipelines and magical well wishes that the pathing and vendor specific config are correct.
And don't tell me VPS FTP PHP or sshing into a box to special snowflake your own process runner are better than simple vanilla managed kube.
You can be live on the web from zero in 5 minutes with Digital Ocean kube, and that's counting their onboarding.
Neither is "apt install caddy".
- [1] https://litestream.io
I do like this: cron to run the backup and then rsync to https://www.rsync.net, then an after script that check it was run and post to my telegram the analysis.
That is.
You don’t need backups until you have customers.
Worrying about HA when you don't have customers that need it is one thing, but I wouldn't want to be in a place where I have to put a banner on the website asking users to please make a new account because we had an oopsie.
Too many have forgotten what it means to administrate a single system. You can do a lot with very simple tooling.
You get such a large performance malus and increase in complexity right from the start with The Cloud that it’ starts at a serious deficit, and only eventually maybe overcomes that to be overall beneficial with the right workload, people, and processes. Most companies are lacking minimum two of those to justify “the cloud”.
And that’s without even considering the cost.
What I think it actually is, is a way for companies that can’t competently (I mean at an organizational/managerial level) maintain and adequately make-available computing resources, to pay someone else to do it. They’re so bad at that, that they’re willing to pay large costs in money, performance, and maybe uptime to get it.
It seems to really help if you can put a term to it.
Devops engineers did not know 101 of cable management or what even a cage nut is and being amazed to see a small office running 3 used dell servers bought dirt cheap, and shocked when it sounded like a air raid when they booted up, thought hot swapping was just magic.
It is always the case - earlier in the 80s-90s programmers were shaking their heads when people stopped learning assembly and trusted the compilers fully
This is nothing and hardly is shocking? new skills are learnt only if valuable otherwise one layer below seems like magic.
A year or so after I left they ran out of money. They would've lasted longer if the infra guy would've just stayed the backend guy and helped get projects done more quickly instead of shiny k8s setups for projects with a dozen end-users per day. Recently I saw that the CTO has started a new startup - and ironically the only guy who he took with him onto the new team looks to have been the infra guy!
I don't blame infra guy, he genuinely believed he was doing the right thing.
If you get one dedicated server for multiple separate projects, you can still keep the costs down but relax those constraints.
For example, look at the Hetzner server auction: https://www.hetzner.com/sb/
I pay about 40 EUR a month for this:
Disk: 736G / 7.3T (11%)
CPU: Intel Core i7-7700 @ 8x 4.2GHz [42.0°C]
RAM: 18004MiB / 64088MiB
You could have 15 VMs each with 4 GB of RAM and it would still come out to around 2.66 EUR per month per VM. It's just way more cost efficient at any sort of scale (number of projects) when compared to regular VPSes, and as long as you don't put any trash on it, Proxmox itself is fairly stable, being a single point of failure aside.
Of course, with refurbished gear you'd want backups, but you really need those anyways.
Aside from that, Hetzner and Contabo (opinions vary about that one though) are going to be more affordable even when it comes to regular VPS hosting. I think Scaleway also had those small Stardust instances if you want something really cheap, but they go out of stock pretty quickly as well.
I just have a few large VMs, each a different environment with slightly different ways how I treat them - the prod ones get more due diligence and being careful, whereas all of the dev ones (including where I host Gitea, Woodpecker CI, Nextcloud, Kanboard, Uptime Kuma etc.) I mess around with the configuration in and do restarts more often. I personally used to run a Docker Swarm cluster, but now just use Docker Compose with Ansible directly, still multiple stacks per each of those servers, dead simple
So my setup ended up being:
* VPS / VMs - an environment, since don't really need replication/distributed systems at my scale
* container stack (Compose/Swarm) - a project, with all its dependencies, though ingress is a shared web server container per environment
* single container - the applications I build, my own are built on top of a common Ubuntu LTS base more often than not, external ones (like Nextcloud and tbh most DBs) are just run directly
It’s very interesting how people rent large VMs with a hypervisor. I’m wondering if licenses for VPS have any clauses preventing this for commercial scale.
Since I only needed about 3 VMs (though each being a bit beefier, running containers on them, a web server sitting in front of those with vhosts as ingress), I could give each VM its own IPv4 address and it didn’t end up being too expensive for my use case. Would be a bit different for someone who wants many small VMs.
I assign few VMs public IPs and use them as ingress / SSL termination / load balancer for my workloads running on VMs with only internal IPs.
I personally use kvm with libvirt and manage all these with Ansible.
If you have a plan from the start and you know what you'll need and you're pretty confident it won't change, then sure.
If you want a box that you can slice and dice however you want (VMs, containers, etc) then something like Proxmox might be worth it.
One note: you can absolutely use Python or Node just as well as Go. There's Hetzner that offers 4GB RAM, 10TB network (then 1$/TB egress), 2CPUs machines for 5$.
Two disclaimers for VPS:
If you're using a dedicated server instead of a cloud server, just don't forget to backup DB to a Storage box often (3$ /mo for 1TB, use rsync). It's a good practice either way, but cloud instances seem more reliable to hardware faults. Also avoid their object store.
You are responsible for security. I saw good devs skipping basic SSH hardening and get infected by bots in <1hr. My go-to move when I spin up servers is a two-stage Terraform setup: first, I set up SSH with only my IP allowed, set up Tailscale and then shutdown the public SSH IP entrypoint completely.
Take care and have fun!
If you’re backing up to a third party losing your account isn’t a disaster, bring up a VM somewhere else, restore from backups, redirect DNS and you’re up and running again. If the backups are on a disk you can’t access anymore then a minor issue has just escalated to an existential threat to your company.
Personally I use Backblaze B2 for my offsite backups because they’re ridiculously cheap, but other options exist and Restic will write to all of them near identically.
Note that you don't need all of that to keep your SSH server secure. Just having a good password (ideally on a non-root account) is more than enough.
It's easier to add a small config to Terraform to make your config at least key-based.
Once I had Postgresql db with default password on a new vps, and forgetting to disable password based login, on a server with no domain. And it got hacked in a day, and was being used as bot server. And that was 10 years ago.
Recently deployed server, and was getting ssh login attempts within an hour, and it didn't had a domain. Fortunately, I've learned my lesson, and turned of password based login as soon as the server was up and running.
And similar attempts bogged down my desktop to halt.
Having an machine open to the world is now very scary. Thanks God for service like tailscale exists.
And one simple mistake, and we're screwed
Funny you said that. I migrated an old, Django web site to a slightly more modern architecture (docker compose with uvicorn instead of bare metal uWSGI) the other day, and while doing that I noticed that it doesn't need PostgreSQL at all. The old server had it already installed, so it was the lazy choice.
I just dumped all data and loaded it into an SQLite database with WAL and it's much easier to maintain and back up now.
Not everybody says so... So, can anyone explain what's the right way to think about WAL?
Now this is more controversial take and you should always benchmark on your own traffic projections, but:
consider that if you don't have a ton of indexes, the raw throughput of SQLite is so good that on many access patterns you'd already have to shard a Postgres instance anyway to surpass where SQLite single-write limitation would become the bottleneck.
[0] https://www.sqlite.org/src/doc/begin-concurrent/doc/begin_co...
If you were seeing errors due to concurrent writes you must adjust BUSY_TIMEOUT
Curious as to why you say this. I’m using litestream to backup to Hetzner object storage, and it’s been working well so far.
I guess itt’s probably more expensive than just a storage box?
Not sure but I also don’t have to set up cron jobs and the like.
At least with Storage Box you know it's just a dumb storage box. And you can SSH, SFTP, Samba and rsync to it reliably.
[0] https://docs.hetzner.com/storage/object-storage/supported-ac...
He's mainly talking about the tech implementation which is the easy part.
the hard part of creating a business is finding a problem valuable enough to solve and reaching the users who need that problem solved. that's where the real value is.
Thinking about on how to fit everything on a $5 VPS does not help your business.
I recall running LAMP stacks on something like 128MB about 20 years ago and not really having problems with memory. Most current website backends are not really much more complicated than they were back then if you don't haul in bloat.
I don't know what you value your time or opportunity cost as... but the $10/mo doesn't need to save very many minutes of your time deferring dealing with a resource constraint or add too much reliability to pay off.
If resource limitations end up upsetting one end user, that costs more than $10.
And most VPSs allow increasing memory with a click of a button and a reboot.
$100 is peanuts to most businesses, of course. But even so, I'd rather spend it on fixing an actual bottleneck.
For example: Ticketmaster makes a ton of money and their site is complete dogshit.
if the scalability is in the number of "zero cost" projects to start, then 5 vs 15 is a 3x factor.
That's 17 million hits per day in about 3.9 MiB/sec sustained disk IO, before factoring in the parallelism that almost any bargain bucket NVME drive already offers (allowing you to at least 4x these numbers). But already you're talking about quadrupling the infrastructure spend before serving a single request, which is the entire point of the article.
# ioping -R /dev/sda
--- /dev/sda (block device 38.1 GiB) ioping statistics ---
22.7 k requests completed in 2.96 s, 88.8 MiB read, 7.68 k iops, 30.0 MiB/s
generated 22.7 k requests in 3.00 s, 88.8 MiB, 7.58 k iops, 29.6 MiB/s
min/avg/max/mdev = 72.2 us / 130.2 us / 2.53 ms / 75.6 usThere is a good reason: teaching yourself not to over-engineer, over-provision, or overthink, and instead to focus on generating business value to customers and getting more paying customers. I think it’s what many engineers are keen to overlook behind fun technical details.
This is specious reasoning. You don't prevent anything by adding artificial constraints. To put things in perspective, Hetzner's cheapest vCPU plan comes with 4GB of RAM.
It is specious reasoning. Self-imposing arbitrary constraints don't make you write good, performant code. At most it makes your apps run slower because they will needlessly hit your self-impose arbitrary constraints.
If you put any value on performant code you just write performance-oriented code, regardless of your constraints. It's silly to pile on absurd constraints and expect performance to be an outcome. It's like going to the gym and work out with a hand tied behind your back, and expect this silly constraints to somehow improve the outcome of your workout. Complete nonsense.
And to drive the point home, this whole concern is even more perplexing as you are somehow targeting computational resources that fall below free tiers of some cloud providers. Sheer lunacy.
I think your analogy is flawed; a more apt one would be training with deliberately reduced oxygen levels, which trains your body to perform with fewer resources. Once you lift that constraint, you’ll perform better.
You’re correct that you can write performant code without being required to do so, but in practice, that is a rare trait.
The Macbook Neo with 8GB RAM is a showcase of how people underistimated its capabilities due to low amount of RAM before launch, yet after release all the reviewers point to a larger set of capabilities without any issues that people didn't predict pre-launch.
Even their $5 plan gives 4GB.
In my head, I call this the 'doubling algorithm'.
If there's anything that's both relatively cheap and useful, but where "more" (either in quality or quantity) has additional utility, 2x it.
Then 2x it again.
Repeat until either: the price change becomes noticeable or utility stops being gained.
Tl;dr -- saving order-of single dollars is rarely worth the tradeoffs.
> Immediately proposes alternative which is literally 4x the cost.
So yea, stick with what worked for decades if you don't see a reason not to. Also, I remember reading that StackOverflow runs on a bunch of super powerful root servers?
> Here is the trick that you might have missed: somehow, Microsoft is able to charge per request, not per token. And a "request" is simply what I type into the chat box. Even if the agent spends the next 30 minutes chewing through my entire codebase, mapping dependencies, and changing hundreds of files, I still pay roughly $0.04.
> The optimal strategy is simple: write brutally detailed prompts with strict success criteria (which is best practice anyway), tell the agent to "keep going until all errors are fixed," hit enter, and go make a coffee while Satya Nadella subsidizes your compute costs.
Wow. I'll definitely be investigating this!
https://old.reddit.com/r/GithubCopilot/comments/1r0wimi/if_y...
Worth noting however that they are starting to introduce rate limits lately so you might struggle to run multiple concurrent sessions, though this is very inconsistent for me. Some days I can run 3-4 sessions concurrently all day, other times I get rate limited if I run one non-stop..