If we could come up with a system to classify the probabilities across a large number of candidate words (or components thereof) then this could actually be good at producing text, one element at a time. We could call these elements 'tokens' and picking the right one could be called something like 'decoding'. Crazy idea but hear me out...

On a more serious note, it will be fascinating to see how this different spin on modelling inference will create new paradigms or slot into existing ones.

Seems like LLM can do everything Jev can do (just structured outputs?) but Jev is highly optimized and purpose built for it and thus way faster and cheaper. Is that a fair description?

This will be insane for tool usage, and probably where the major economics for day-to-day usage will be.

The goal is going to be to use llms to distill operations down to some dsl, and pass it into something like Jev.

While I understand that accelerating development isn't necessarily the target for this, and it's not at all intended to generate code the way many of us are...

I think this could be pretty decent in CI? There's a lot of "flakes" I've mediated that this could have handled much more efficiently. Maybe observability as well, triggering elevated logging and other initial measures?

Hm, would be good to understand the architecture better. Is this answering just from a world model informed prior? How informed is it by the information in the prompt? I can't see this maintaining calibration across all domains and all types of structured output.

Is there anything published on how it maintains calibration? Or when you say "outputs calibrated probabilities" you mean "as calibrated as frontier LLM models, just cheaper" - which is a different claim; as LLM's aren't particularly well calibrated

Hasn't there been a lot talk about Astra's opaque reasoning capabilities (being able to think through complex questions without using a chain of thought)?

Given that, can't you just replicate Jev by telling Astra "here is the question, you must make a multiple choice decision / output a score between 1-10, please answer directly in a single word, no reasoning allowed"?

(Edit: Ok, Jev is much cheaper in input tokens so these two aren't directly comparable at all)

Interesting concept, I can't see a reason to use a generalist classifier over an api rather then just training my own? If it was open weights I would probably mess around with it.

This is amazing. I really could use this.

I like the idea of System one models but all LLMs so far work as system 1 thinking because humans generate speech subconsciously with system 1.

System 2 thinking requires consciousness which AI does not have, so even reasoning models are still system 1 thinking as system 1 in humans has reasoning with heuristics.

Its limited but most people navigate the world with it completely, so it's enough for AI.

This is best thing to use for decision making, evaluation, classification. If I'm not wrong.