Is anyone even offering single tenant inference as a service?

Is that a new class of tenant isolation vulnerability?

I kind of forgot about the shared tenancy attack model. I think I was assuming/hoping the software was magically isolating my tokens, when in reality that's impossible for shared tenants. There's whole classes of attacks on things like shared cache, before we get to the LLM-specific features and potential bugs. Enterprises with highly sensitive data should really be on non-shared hosts.

[flagged]

[flagged]

[flagged]