Posted by nikolay_sivko 6 hours ago
So what does this mean for ingestion (and indexing)? Iceberg V3? Paimon? Bespoke ingestion through the DB engine to do the indexing?
I assume the native formats will always be faster / more optimized but the need for Trino as a separate executor while running either of these databases seems to be close to gone.
Native format is faster (especially for colocated joins), but it's way more expensive if you have to run a bunch of separate storage nodes vs just using S3, especially your query volume isn't that high.
I liken it to the BigQuery cost model, where storage is effectively free.
With that, I was able to get the interview although I ended up failing it. My lack of experience in C++ was probably one of the main reasons. Your course helped me stand my own during the interview and even though I had zero database experience apart from finishing the course, I felt adequately prepared.
Thank you for making the course open source. As a working professional, it was helpful to be able to do the course on my own time. Suffice to say, I am a big fan of your work and thank you for introducing me (and other fellow learners) to the interesting world of Databases!
Congrats Andy, hope you enjoy the ride =)
Best of luck!