Well, I can tell you this is making me actively reconsider dropping duckdb for ch as we go from prototype to prod

Definitely encroaching.

Our last product (SaaS observability) uses Clickhouse.

Our next product (self-hosted observability) uses DuckDB.

DuckDB is an in-process (now I guess less so with Quack) OLAP database for analytical workloads. PostgreSQL or MariaDB are OLTP row-oriented databases that are great for application/transaction-focused workloads but are less great when you want to query across a giant amount of data.

DuckDB is much like SQLite, but built for OLAP workloads: it's in-process, with a single file format on a disk, and (unlike SQLite) the data is stored in columns for better OLAP performance.

Like SQLite, concurrency options are limited compared to client-server databases like Postgres or MariaDB. DuckDB 2.0 will be adding a client-server mode with the Quack protocol which can allow for greater concurrency.

But you can also use DuckDB as a multitool to connect to and query all kinds of other data sources from one connection. Being able to pull in data from Postgres, CSVs or parquets on a file system or S3, and JSON returned by a web API, and then query across all of it in one place, can be quite handy for ad-hoc data analysis and exploration.

You might know this already, but you can query pandas/polars/arrow tables directly w/ duckdb and use whatever stats packages you feel like alongside it in the same python script. I feel like they do a decent job sticking to the simpler statistical fans that make sense in sql.

which statistical functions do you want? i’m curious because i love duckdb and use it for a variety of projects but always want to learn more about how to use tools better.

I’m currently contemplating that MySQL apparently cannot do an INSTANT change of the collation of an unindexed column, even though, AFAICT, it has no effect whatsoever on the on-disk format or any data structure at all except for the metadata saying what the column type is.

I do not enjoy dealing with text encodings and collations in databases.