Posted by bashbjorn 15 hours ago
Still good. In practice for Jev the devils in the details. As you all know by now, it's easy to write PoC and understand with AIs (or even manually, which is now a prestious practice).
That demo will get you 80% there
Getting to that 100% or even 99% to JEV level will be hard with all the edge cases, infra, API, communications, etc.
Still a good article.
here's 7 lines
import os
import dspy
lm = dspy.LM("openrouter/z-ai/glm-5.3-flash", api_key=os.environ["OPENROUTER_API_KEY"])
jev = dspy.Predict('email:str -> choice:Literal["Legitimate", "Spam", "Phishing"]')
email = "Payroll asks for your password on a non-company sign-in page."
pred = jev(email=email, lm=lm)
print(pred.choice)
it is the latency that makes it significant
the example uses an external api, and i don't think they return probabilities from those anyway.
1. draw a circle
2. import the rest of the owlEnds with referring to a product, and saying "this is a parody post", after pretending to make a serious point.
In real life, a human doesn't do classification tasks with the System One part of their brain, they use System Two. So by definition what Jev does isn't System One thinking.
If anything, regular programming that automatically executes based on logic, without requiring "thinking" would be "System One".
I'd argue that most human classification is pre-conscious / System One. You see a table, you recognize it as a table without asking yourself "is this a table?"
I guess their marketing implies that it moves classification into system one response time.
I think you answered your own question. Executives are going to ask two questions, 1) how is this different/why does it matter and 2) how will i use it to make money?
Leya came to market more than a year before Jev, and failed because nobody understood how to use it, and he was unable to market it properly. Jev used this strategy and did not fail.
Either way it's an analogy that's bound to be loose as Kahneman's modes are about humans.
Huh? I guess that depends on the exact definition of "classification", but I think the bulk of basic classification tasks we make every day to make sense of our surroundings, such as object recognition is definitely done using system 1. So is higher-level "stereotyping" or anything you could described with "I know it when I see it".
Because those responses can be incorrect or even harmful, you would sometimes make use of system 2 to correct them - but that doesn't change that the initial response is from system 1.
Those are generally the kind of tasks that require "System 2" in humans.
To be clear, I think the whole "System 1 vs System 2" framing is a pretty limiting way to think about AI (and thinking in general).
But with models, we can train them to answer such questions without verbal reasoning.
"System One" and "System Two" were coined in some pop science book...so back to its usage being a marketing ploy.
If you're comparing with something, you need to state 'fast' in relative terms. Jev is definitely fast, and if this Python takes the same time to get a decision then it's also fast. If it's 100* slower than Jev though, you shouldn't be calling it 'fast', because relatively speaking it's really, really slow.
So, fast in the LLM space and comparable with Jev.
Here is an archive: https://web.archive.org/web/20260923122959/https://www.nobod...
- By not being a optimised for chat, it can deliver confidence for answer and not for how an answer should be phrased
- Speed. It can take seconds for OpenAI to compile schemas, jev can respond before openAI has even begun thinking
- Token efficiency and price. I think its the output token they don't even charge for because they are negligible, and the tokens they do charge for are at a fraction of a comparable model.
If you are using structured output, I think those 3 together is a really big deal.
>But their example is classification but that would also be possible and faster with a classic BERT model.
I believe the things you can classify with ChatGPT without any tuning or training is way beyond what BERT can do.
If you accept the premise that there are use cases where you might ask a frontier model a classification-shaped question and expect an ok enough answer, rather than creating a purpose specific classifier on some dataset that you have, then it follows that this is quite an inefficient thing to do, because you're doing extra work to turn the output tokens into a structured output and mostly throwing them away. So then if you could instead train a frontier level model that skips the output tokens and directly returns the structured classification information, that would be more efficient, and that's what jev seems to be.
But a lot rides on that initial premise of whether this is a use case that makes sense. But if you find yourself asking a model like Opus arbitrary yes/no questions and then maybe you switch to a faster and cheaper model because it's too slow and expensive, it seems like jev might be a great replacement for that.
As far as I understand, the idea of Jev is zero-shot or few-shot classifier: it learns a lot of stuff at pre-training, but unlike a classic LLM it doesn't need to learn how to chat, so it can be much smarter at a particular size
> But their example is classification but that would also be possible and faster with a classic BERT model.
With BERT, you need a large, labeled dataset, and you have to train/fine-tune the model. Jev is pitched as a zero- or 'few-shot' model. You define the schema in code, give it instructions, and it works without a traditional training pipeline.
> So their pitch is a task specific smaller model or am I completely misunderstanding the whole thing?
Yup; that about sums it up: it is more or less an optimized, task-specific small model with the flexible understanding of a traditional LLM.
I think they were responding to this. You can use BERT to provide zero shot classification predictions.
Not particularly. There is still the problem of hallucinations and varying results across runs.
That's more of what type-safety means for their team. Every run gives the same results. It's type-safe
For three choices problem (A,B,C), what Jev guarantees is that it will give the choice in a defined schema (type-safe). It never guarantees that the choice is correct (hallucination).
My base case is that this will probably be pretty useful, and also not as useful as the current hype suggests.