Top
Best
New

Posted by gbayomi 1 day ago

Show HN: jevals – replacing LLM judges with typed Jev decisions(github.com)
43 points | 6 comments
jacksun788 10 hours ago|
Curious — how's this compare to laya?
gbayomi 4 hours ago|
It can also run with laya. The idea is to have an easy way to run common evals with decision models and easily switch the “backend” from jev/kev/laya etc
adityamishra241 12 hours ago||
Interesting idea. How do you handle cases where the decision depends on context that isn't captured by the typed decision type?
gbayomi 4 hours ago|
It’s very dependent on the underlying model right now, but it performs well for the current versions of jev/keV/laya!
sshussain270 1 day ago|
This is going to be a hot use case.
gbayomi 3 hours ago|
Agreed