Its a yes if you do not know the domain space, query patterns well enough and also if the cost of optimization or time for optimization may have detrimental impact to business. In this case it most likely means that the crowd in the room did not anticipate much on this in early phases and no one in the room pointed these things out. The irony is that these performance and disk numbers are heavily discussed as a part of system design interviews.

> In fact optimization is by far the easiest part of the process because there are many system programming experts on this HN thread who consider these optimizations to be trivia

This is a misconception when you including roll out as a part of the change too, changing data once its running in production is hard, changing the data structure is even harder and when you talk about making changes in cache which is at the hot path its probably the hardest. Looking at the graph at the end it looks like it took them 4+ months to roll out the changes after optimization.

I do not think Cloudflare was a less-than-peers optimized product when they launched. This is one of their blog posts which describes taking one aspect even further.

I think Cloudflare became big only because they were so much more optimized than others that they offered some services for free that others were not offering. If running costs are high, you only burn (VC) cash and then you exit.

Or optimize a bit earlier and prevent having to scale out to a bazillion systems.

This assumes that you have plenty of cash to burn in the process, which is approximately correct for VC-backed ventures, and for offshoots of large corporations that play a lomg game.

> start generating profit, and then you can start optimizing your costs

Good thing they jumped on that as soon as they were profitable instead of burning cash. Oh wait...

I think a distinction to draw here is that Cloudflare had relatively large capital raises and were almost immediately profitable¹. They had the luxury of throwing away money. Judicious optimisation makes sense for scrappy start-ups, especially when trivial optimisations like these could easily be farmed off to an agent.

¹ https://timeline.www.cloudflare.com/

This reasoning assumes you have access to infinite runway. You don't.

Only if you have loads of capital

> Produce working product first, validate the idea, stabilize the business, start generating profit,

not everybody is so lucky to be able to go in that order? The first part requires upfront capital/investment?

[dead]

For the curious, this is technically possible in Rust using a dynamically sized type [1], but in practice is difficult and doesn't really play nice with the rest of the language. The nomicon entry concludes with "Yes, custom DSTs are a largely half-baked feature for now." [2]

[1] https://doc.rust-lang.org/reference/dynamically-sized-types....

[2] https://doc.rust-lang.org/nomicon/exotic-sizes.html

> putting the record data right after the CacheEntry members

I assumed they couldn't do that because they're using it with some kind of generic HashMap<K, V>. In that situation, can "V" be dynamically sized?

A dynamically sized "V" would mean you can't have an array of them, which might preclude some hash map implementations.

Unfortunately, Rust is not a good choice for this kind of tricks. This is where Zig shines. In Rust, you can’t even use proper arenas, which can help a ton with allocations.

Cloudflare started to pick Zig recently, for projects, that have memory constraints.

System programming always matters. Things are cheap until they aren't one day.

Depends on how the CacheEntry is stored, it's probably stored in a slice of &[CacheEntry] which precludes storing the record data alongside it as the size of each entry must be fixed.

I wish more programming languages implemented record types as seen in databases, where dynamically sized fields are packed into a contiguous area of memory.

The CloudFlare manually implemented a clumsy version of this.

Wouldn’t it be nice for the compiler to manage this for you in the same way that your database engine does when it saves a “row”?

less ergonomic, but still totally doable

Rust does that automatically unless you switch to the C layout.

In langages that don’t there’s a tension between memory use and human readability / consistency of the layout. There are also other domains which can be affected e.g. databases, it’s a concern / issue when using postgres.

Why do I always find interesting new Twitter accounts just as the person is leaving :)

I think it’s more of a time vs code tradeoff, if done properly.

For example in the Vec case, you could theoretically build an alternative which encodes the “three sections” property internally, and ensures correctness at construction time for the pointers. Not as completely safe as a Vec, but you can still get similar benefits for the “business logic”.

But I agree, just having a custom structure that does not provide a safe wrapper around this would be sacrificing standard guarantees.

You can make a wrapper type that abstracts the offset lookup logic with a safe interface. If it's a transparent struct then rust will compile it away into nothing but you still get the abstraction in your code.

> I can't help to think that the approach of joining several distinct list into a single one in some way undercuts Rust's safety guarantees.

Not really. You just need to make the underlying fields private and provide methods to get slices to the data you need.

you could always do a .get into the vector and handle the error, it doesn't necessarily need to panic.

Thank being said in this case it should be impossible to index out of bounds so maybe a panic is warented.

It's the exact thing Rust is made to protect against, on a more local scale. Every memory corruption bug is just an out-of-bounds index that wasn't protected against.

Tools exist to serve us, not the other way around.

I'd love to hear what was taking up that 17GiB if you can share, even if it is commonly optimized things like packing, still fun to hear about