Posted by roywashere 4/18/2025
The overhead at low volume is pretty high, but in the higher volumes (25M transactions/24h) it's a massive cost saving for us.
Edit:
There were just some initial headaches with needing to increase kafka partitions and add replications to the transaction processors, otherwise we didn't quite leverage the available compute and the backpressure would fill Redis up until OOM.
For those interested in only errors, the self-hosted version recently introduced errors-only mode which should cut down on the containers.
Bugsink's also quite scalable[0], but I wouldn't recommend it a 25M/day.
Well, your homepage disagrees with this statement:
> Bugsink can deal with millions of events per day on dirt cheap hardware
But it's a very fuzzy way of quantifying something, and open to various interpretations.
Wow, that's really cheap. I'm seriously overpaying for my cloud provider and need to try Hetzner. I always assumed Hetzner was only European based.
FWIW, https://lowendbox.com/ is good fun for the former set of things, too
I think this is a repeated question but... are you considering the cost of the people managing the deployment, security oversight, dealing with downtime etc?
Disclosure: I'm a sysadmin.
What I said is true for places where they already have sysadmins for various tasks. For the job I do (it's easy to find), you have to employ system administrations to begin with.
So, at least for my job, working the way I described in my original comment is the modus operandi for the job itself.
If the company you're working in doesn't prefer self-hosting things, and doesn't need system administrators for anything, you might be true, but having a couple of capable sysadmins on board both enables self-hosting and allows this initiative to grow without much extra cost, because it gets cheaper as the sysadmins learn and understand what they're doing, so they can handle more things with the same/less effort.
See, system administrators are lazy people. They'd rather solve problems for once and for all and play PacMan in their spare time.
There's a ticket now open to stop this, but it's still in progress.
Forking has down sides that can't be hand waved away too, especially for a service like this.
Feel free to email - david at sentry
https://github.com/getsentry/sentry-dotnet/issues/3636#event...
Transactions like full user flows start to finish, or 1 transaction = 1 post/get and 1 response?
For most applications we are talking closer to 1 transportation 1 web request. Distributed tracing across microservices is possible, the level of extra effort required depends on your stack. But that's also the out of the box, plug and play stuff. With lower level APIs you define your own transactions, when they start and end, which is needed for tracing applications where there isn't a built in framework integration (e.x not a web application).
As others have said, we've [0] found the only practical way to deploy this for our clients is Kuberentes + Helm chart, and that's on bare-metal servers (mostly Hetzner). It runs well if you can throw hardware and engineering time at it, which thankfully we can. But given the option we would love a simpler solution.
[0]: https://lithus.eu
But we specialise in this so that our clients don't have to. As much as I do actually love Kubernetes, the fact that the _easiest_ way to self-host Sentry is via Kubernetes is not a good sign. And choosing to spin up a Kubernetes cluster just to run Sentry would feel a lot like the lady who swallowed a fly[0].
[0]: https://en.wikipedia.org/wiki/There_Was_an_Old_Lady_Who_Swal...
That said I would honestly prefer if the industry would just settle on K8s as our OS.
I really do not see any benefit that sentry could bring on its own compared to a solid set of Helm charts for k8s.
When I posted this myself on Reddit, I said the following:
I've long held off on actually posting this article to a platform like this one (don't bash your competition and all that), but "isn't Sentry self-hosted?" _is_ one of the most asked questions I get, and multiple people have told me this blog-post actually explains the rationale for Bugsink better than the rest of the site, so there you have it.
Feedback on competition bashing: sometimes they deserve it, they should really just come out and say it: “open sourcing our stuff isn’t working for us, we want to keep making money on the hosting”, and that would be ok
https://blog.sentry.io/building-an-open-source-service/
We enable self-hosting because not everyone can use a cloud service (e.g. government regulation), otherwise we probably wouldn't even spend energy on it. We dont commercialize it at all, and likely never will. I strongly believe people should not run many systems themselves, and something that monitors your reliability is one such system. The lesson you learn building a venture backed company, and one that most folks miss: focus on growth, not cost-cutting. Self-hosting for many is a form of cost-cutting.
We do invest in making it easier, and its 100% a valid complaint that the entire thing is awful today to self-host, and most people dont need a lot of the functionality we ship. Its not intentional by any means, its just really hard to enable a tiny-scale use-case while also enabling someone like Disney Plus.
On your second point, you're right that self-hosting is cost cutting, but I found that when a business is growing, the features that sentry offers are more of a nice to have along the way, not quite critical to its success (No point capturing errors if no-one's using your bloody thing). Its easy to rack up that monthly bill with nothing but nice to have hosted services.
What I'd love to have is dumb versions of tools to self host initially when the requirements (and traffic) is very low. Kibana for example is a pig to self host, at one point it was taking up 25% of our production capacity. I found Loki is much better for simple cases in the beginning.
Good example that strikes a balance is Grafana / Prometheus. Its practically impossible to run a software shop without them, and everywhere I worked went through the same phases: Chuck a set of random containers in prod -> Deploy helm chart -> Migrate to Thanos -> Move to hosted Grafana when user/teams management gets out of hand.
Benefit is that absolutely everyone is familiar (and even likes) your tooling, I hope that's enough of a reason to give away a simpler offering.
In fact I did one last week, but it got only a fraction of today's article's traction... I'll try again in whatever the prescribed interval is :-)
We also found the same problem as OP with self hosting sentry. Each release would unleash more containers and consume more memory until we couldn't run anything on the 32gb server except Sentry.
We looked at both GlitchTip and BugSink and only settled on GlitchTip as it was maintained by a bigger team. Feature wise they were quite similar and both good alternatives.
So far so good with GlitchTip.
And thanks Op for making BugSink, the more alternatives the better.
Although with Bugsink (which is what came out of this origin story of annoyance) I'm aiming for _even more_ simple (1 Docker container at minimum, rather than 4), 30 seconds up and running etc.
I use Sentry with most of my clients, and for effective debugging I need to spin my own Sentry in a Docker container which ends up being quite heavy on my machine especially when combined with Grafana and Prometheus.
I'm really unhappy with virtually all monitoring/telemetry/tracking solutions.
It really feels they are all designed to vendor lock you in their expensive cloud solutions and I really don't feel I'm getting my $s back at all. Worst of all those cloud vendors would rather add new features non-stop rather than honing what they currently have.
Their sales are everywhere, I've seen two different clients getting Datadog sales people join private Slacks to evangelize their products.
Both times I escalated to the CTO, both times I ended up suspecting someone in management had something to gain from pushing teams to adopt those solutions.
Killing flies with hammers and all, but since I really like my hammer I actually do all my local development with my full-blown error tracker too:
I can only commend the hustle on their part, but it does feel a little like a high pressure time share situation.
david at sentry.io
I’m not sure how I feel about the license though (Polyform Shield, basically use-but-don’t-compete). It’s a totally valid choice – I just wish it would convert to FOSS at some point. (I understand the concern as I’ve had to make a similar decision when releasing https://lunni.dev/. I went with AGPL, but I’m still overthinking it :-)
Quite rare to hear this wise line these days. An I guess with AI coding assistant, this is only the beginning of this kind of horror story
If you're running on commodity hardware, sure; if you happen to be a $1T company that solders 3x marked-up RAM then that's definitely not true https://www.apple.com/shop/buy-mac/macbook-pro/16-inch-space... is the entry model, and clicking the 48GB option dials the price up to $3k
But it was a good call sending it to the cloud. Better than "my problem" it is something being "somebody else's problem"
Any insights on why Sentry is so complex and needs so much resources? Is collecting, storing, and organizing errors messages and stack traces at scale difficult? Or it's the other features on top of this?
- they had enough money that they never needed to think seriously about maintenence cost, and the sales process was strokg enough to keep customers arriving anyway (look to Oracle for another example of hopelessly complicated installation process but people keep using it anyway)
- at some point someone realized this was actually a feature: the more complicated it got, the harder it became to self host. And from that perspective it is a win-win for the company: they can claim it is open source without being afraid that most people will choose to self host.
> actually a feature
I would guess that for a few people people (e.g. the ones who made the scary visual of rising costs) this is explicitly so, but for most people it's more implied. i.e. I don't think anyone advanced their career with Sentry by making self-hosting easier.
They have all sorts of caching, autoscaling, distributed systems and other stuff thats complete overkill for all except that largest installation. Plus all sorts of software features only needed by a few customers and extra layers to be multi-customer.
It's the difference between a hoop in your back yard and a NBA Stadium
As in, a huge SaaS company offers their product for self-hosting to individual companies, but it's not practical to self-host because the code is highly specialized for supporting hundreds of companies instead of just one? And it's hard to have an architecture that works well for just one and for hundreds?
For example Sentry requires ClickHouse, Postgres, Kafka, and Redis presumably because they were the right tools for their needs and either they have the resources to operate them all or the money to buy the managed options from vendors.
Also, the main concern people have with hosting Sentry is the sheer number of containers required but most of them are just consumers for different Kafka queues which again is presumably this way because Sentry ops prefers it this way, whether it be for fine tuning the scaling of each one or whatever the reason.
What makes sense for a SaaS company rarely translates to sensible for self-hosting.
For my self-hosted analytics platform [0], I chose to only use the LAMP stack (PHP/MySQL), as it's one of the easiest to set up and maintain. Yes, it won't be able to track 100 million events per day, but most people don't have that amount of traffic anyway. It runs on the cheapest VPS, and can be installed in many different ways (adding more and more, if the platform is self-hosted only, my goal is to make installation as simple as possible, especially with the free trial, so customers can try it anywhere in a few minutes). Sorry if it sounds like a rant, but it's hard competing with huge corporations promising free "self-hosted options", only to use it as a lead-magnet for the cloud service. I saw this trend growing, even with smaller companies, where their self-hosted one-click app installs fail and they provide no support for it or have no intention to fix the installer.