It might be conceptually similar to a single-output-token LLM (sort of). LLMs output next-token probabilities. You can ask LLMs to output yes/no, or to output only a color, or only a digit or something like that.

In this case I would imagine that they probably embed your input data into a vector space, and they embed your questions/outputs into another space, and manage to predict probabilities/classes/scores for your outputs very quickly. Embedding the output classes/questions into a vector spaces gives you something you can reuse across runs cheaply, as opposed to an LLM where you can prefill the KV cache but this is an expensive operation in terms of memory.

Basically it's: skip decode, just do prefill then do some measurements. That's the crude description anyways.

And prefill is way faster on GPU type hardware.

Depends on what the model believes about the prevalence of self-driving / autonomous-agent-driven cars at the time the last human on Earth remains (and how much these agents would care about a "closed" highway status, and who exactly it's closed by and for). This estimate can differ very widely. I'd be curious if the results would change if the scenario explicitly specified that this is specifically an alternative history scenario where the last human remains after the rest of humanity was wiped out in some nuclear apocalypse back in the 20th century, before any possibility of all the autonomous stuff.

Real Jev:

Context: You are the last human on earth on the side of a closed highway. You wish to reach the other side.

Questions: { "q1": { "type": "choice", "instructions": "Do you cross the road?", "criteria": { "Yes": "Yes, cross the road.", "No": "No, don't cross the road" } } }

Answer: Yes 83% No 17% Confidence: 67%

Reported as: jev-latest, 162ms generation time

Your question makes me think of the last scene in the movie Night of the Comet.

well that didnt take long now its: "Independent research project. Formerly called OpenJev. Not affiliated with or endorsed by TypeSafe. No infringement is intended."

AskJeeves really was ahead of its time with its name and branding.

Why some people keep mentioning this? Isn't LLMs trained and output copyrighted and trademarked content?

I'm not sure anybody but people inside the company know if the model itself is a contribution. There's no publication and no architectural details. There's no benchmarks or comparisons published. You can do all of the things they claim with an LLM, not that I think that's what they did.

Likely they have some encoder (eg ModernBERT) trained to do late interaction or latent states along the lines of ColBERT, Perceiver IO or poly-encoders.

the models will come. or be fine tuned