Those are the two main interfaces: the inference hardware and the endpoints. Endpoints are pretty much standardized to the point that you can switch them out in seconds. CUDA still has an edge but every release like this chips away at that. Even so, TPUs are unobtanium so while I appreciate Google's effort here I also recognize that it is not something that will be useful to me.

In that sense AMD is still better positioned to give NVidia a run for their money but given the family connection and AMD's underdog status I am not very hopeful that we'll see meaningful competition in inference hardware any time soon.

NVidia is a much harder nut to crack than OpenAI or Anthropic. They are easily a decade ahead of viable competition. Check out the GB300 if you don't believe that.

Even if it's allegedly possible, the devil has always been in the details. It always sucks to find out that there's some quirk of your model architecture that breaks an implicit assumption that underpins the export process.