So in theory you could feed it incomplete text, and then ask it for the probabilities of what the next character could be?

Side note: it took me more time than I would like to admit to realize that Diogo Almeida isn’t a satirical version of the name Dario Amodei

I would love for things like this to be accessible via hubs like open router or AWS bedrock. It's hard to justify adding new model vendors directly with all the heightened concerns about privacy and security, but if bold new capabilities are added to a centralized already-vendor like AWS, technical people can adopt them without going through a whole compliance/purchasing/vendor review process. And an extra middleman tax is well worth it when the cost savings of the model itself can be one-two orders of magnitude.

I'm trying to parse it down to what we had before vs what is new here.

We already had encoder models that skipped text generation for giving us a numerical output that could be computed as a probability. we also got no hallucinations and faster inference for free there. So we already had

1. "unstructured state in, probabilistic decisions out" 2. "orders of magnitude faster and more efficient"

What was hard there was to train the model head without ML expertise, and considerable amount of data.

This seems like this is a democratization of those encoders? The addition over existing encoders seems to be coming from being able to specify the output shape (up to a cardinality of 255). It is unclear to me if this is possible using Jev without additional labels for fine-tuning.

If so, that is still very impressive, but I think the faster inference and 0 hallucinations might come for free, from it not being generative.

This sounds good but so far all claims just sound like marketing terms. I'd love to see real proof. e.g. "RLCD" and "parallel sampling" have nothing to back it up.

also "70-500ms vs 3-329 seconds" are apples-to-oranges unless the LLM baseline is doing comparable work (e.g., long chain-of-thought). If Jev is skipping generation entirely for a narrow structured task, of course it's faster.

Nonetheless i want this to be true, so I'm looking forward to Jev

Edit: I really have to say that I like their manifesto https://typesafe.ai/manifesto

I think what this shows is how important branding and comms are. They've captured imaginations with their demos and nomenclature, despite the arguably non-novel architecture. One forward pass, read the embedding space, train some regressors on predicate structure, [??]

Always exciting to see people working on novel models, rather than the Nth version of the same slightly tweaked LLM.

I'm very curious how much ressources are needed to run such a model. This could be a complete game changer for local applications.

I could see this being fantastic for classification tasks. Last year I shifted from using LLMs for bulk data classification tasks (1M transcripts) to generating embeddings and categorizing based on cosine similarity. It saved a ton of costs and time, but wasn't as accurate as LLMs. This seems like it can give me Terra-level classification ability with the cost/speed I need.

I think this is a great direction -- for some kinds of users. And this makes me wonder if the 'vs' framing is misleading.

Yes, I think it's a mistake that many organizations are cramming LLMs inside of automated pipelines where the extreme generality/flexibility of the model is at odds with the fact that you're using it for a very specific task that gets repeated over and over, and needs a very specific structured output to be successful. But specifying your task carefully (as well as deciding what counts as your input state representation etc) seems like a form of programming. Something (a person or a model working in a relatively unrestricted way) will need to produce a configuration/specification for this system.

So rather than Jev vs Claude I imagine that using Claude/ChatGPT/whatever interactively to define / refine your Jev config which then runs in prod might be the happy combination?

> Input tokens: $0.042 / MTok ($42 per billion tokens).

> Output tokens: FREE (too cheap to meter).

Insane. The video demos are really compelling, in particular the speed.

> Structured outputs slot into ordinary software as fuzzy decision rules: classify, route, score, extract, or branch where hand-written logic is too brittle. The surrounding code constrains their freedom, making them easier to compose into reliable systems.

I buy this vision. A lot of LLM integration I see these days is ultimately exactly this. OpenAI-style structured outputs works decently but this would be a great improvement in cost, latency.