Kev: Tiny Jev-like family of decision models built on top of Qwen3.5

Posted by tosh 15 hours ago

Kev: Tiny Jev-like family of decision models built on top of Qwen3.5(github.com)

377 points | 168 commentspage 2

jwr 14 hours ago|

I wonder how these would do filtering my spam. I have been using 27B-class models for a while now, and they are nearly perfect at determining what is spam and what isn't. The only disadvantage is computational cost.

walrus01 14 hours ago|

Take a look at Thomson 1.0-small, which is a variant of qwen 3.6 35b post trained by Thomson Reuters for text analysis. It classifies text content very well.

Mumps 11 hours ago||

Are you on the foundation research team for Thomson? (If so, hiya from B!) Why would you expect Thomson to be particularly good at spam clf? I figured your additional corpus was all news and legal?

walrus01 5 hours ago||

I have no connection with Thomson Reuters other than as an end user of a GGUF of the LLM I mentioned. That said, from my personal experience with this specific LLM, it's a decent improvement over a "base" Qwen 3.6 35B A3B Q8, and it does a good job of analyzing and categorizing documents on relatively small resources. It'll run fine in llama-server in pure CPU only on a 64GB RAM system with plenty of room to spare, takes something like 47GB with RAM reserved in llama-server for cache and full context size.

faangguyindia 14 hours ago||

On Gemma 4 12B, I am getting 220 ms per move or QS. I used it to play the Snake game locally:

prompt_eval=244 ms wall=245 ms schema_cache=hit generated=0

Move limit reached after 200 moves: score=16, length=19.

So, if a 12B dense model can offer this latency on a local old PC, then definitely you can scale it up with more powerful machines and get even lower latency.

prometheus1992 7 hours ago||

How are you forcing Qwen to answer in a structured way? I like this one better - https://github.com/deepanwadhwa/OpenDecision

akkad33 14 hours ago||

Can someone tell me what is the difference between Jev and a normal neural network that does classification ?

My understanding is: it takes text input and it does one shot classification (no training data)

crackalamoo 14 hours ago|

Yes, this is essentially it.

As a corollary, the output classes can be any set, rather than needing to be set before training.

akkad33 13 hours ago||

Can someone do a ELI5A of how they achieve classification over any user defined list of items? Normal neural networks do a softmax over a known output set to get probabilities

andy12_ 12 hours ago|||

You can achieve open-vocabulary classification by making the final weights in the softmax come from a category encoder instead of being fixed learned weights. So instead of

softmax(encode(input)*learned_weights)

You have

softmax(encode(input)*encode(categories))

I'm not sure if Jev does it this way, but it's how you get open-vocabulary zero-shot image classification with models like CLIP [1].

[1] https://openai.com/index/clip/

theodoretliu 13 hours ago|||

I can think of two possible approaches 1. Jev limits to 255 distinct options. So they can preprocess your set of options and “tell” the LLM via input tokens 1 = red, 2 = blue, etc then jev need only output softmax over 255 states while benefiting from pretrain of other LLMs 2. You allow the forward pass to output over the total token state but mask over the logits to limit to the user options. Less plausible? bc tricky when input is multi token which they clearly support.

My guess would be option 1. Didn’t read the kev repo here which would also explain

raahelb 14 hours ago||

Because these decision models do not have tool calling, the knowledge cutoff might become a problem. We'll either have to keep training continuously if we run locally or switch to the newer version every month or so when using a closed one like Jev

giuscri 14 hours ago|

even with knowledge cutoff set a second from now, you still want to provide as much info as you can if you’re using such tools for delegating decisions

cedws 11 hours ago||

I noticed it has a pretty small context window of only 32k. For most tasks I guess it would be enough with ample context.

algoth1 6 hours ago||

Jev feels more and more like a glorified if/else if block

k__ 8 hours ago||

Half-OT:

Is Jev a decoder (e.g., BERT) or is it some kind of encoder (e.g., GPT) that just happens to be trimmed down to only outputting a handful of tokens for the answers and their probability?

Alpha3031 8 hours ago||

You've got encoder and decoder reversed. BERTs and other models that primarily convert text (or other input) into latent representations are encoders. Models that convert their internal representations back into outputs are the decoders (in the case of GPT et al., autoregressive decoders because they perform this decoding based on past tokens).

npn 8 hours ago||

You got it reversed. Bert is encoder only and gpt is decoder only.

dunlin 15 hours ago||

Been hoping for something in this space. Jev-like decision models on Qwen3.5 could really simplify some of our internal routing logic.

loclol101 5 hours ago||

Using jev for data labeling would be interesting. I wonder how kev compares

floatrock 4 hours ago|

Apply it to this: https://minimallysufficient.com/posts/llm-classification-is-...

webprofusion 14 hours ago|

- https://github.com/logan-markewich/jeff - https://github.com/bespokelabsai/nimble

Schlagbohrer 8 hours ago||

Are these forks? Different orgs doing the same thing as the OP?

webprofusion 14 hours ago||

Why does nobody ever ship these as a docker image?

tacomagick 14 hours ago||

I guess you have AI to write your docker files and push your images now.

More comments...