important to note that the "confidence" score is... maybe not what people think it is - kind of useless, and just a convenience step from the probabilities.
from the docs: "confidence is a statistic computed from the probability distribution the answer already gives you." [0] I actually encourage people to visit the docs because it has a specific page on this with a little applet to really make this clear.
What is the use case for a classifier that works 90% of the time...? I feel like if I'm classifying something, I probably care enough that 90% ain't gonna cut it...
I guess the answer is just agential stuff that effectively gets double checked by the LLM in the driver seat, anyway? That tracks, though it means that jev is mostly just for the people making harnesses. Which is all of us but still!
Which begs the question, can Jev retest until it gets the right result? Can it tell how corellated two of its results are? 90% correct makes for a wonderful iterator, but a poor oracle.
I guess, in the end: I think it'll end up being fantastically useful for artificial engineers with their vastly superior ability to keep track of fine details and rapidly context switch, but fairly niche for any of us organic engineers that are left.
All that doesn't apply to low stakes stuff like games, though -- can't wait for the first truly open world game, NGL. A silver lining to the cobalt cloud?
"""
def fizz_buzz_encode(i):
if i % 15 == 0: return np.array([0, 0, 0, 1])
elif i % 5 == 0: return np.array([0, 0, 1, 0])
elif i % 3 == 0: return np.array([0, 1, 0, 0])
else: return np.array([1, 0, 0, 0])
me: That's enough setup, you're exactly right. [<--- !!] [...]
"""
Damn, Claude was there all along
So that's where in the training set current models get that phrase! /j
It works great, but maybe your implementation could save me some money. I’ll test it and report back.
The tests mock Jev.
I'll raise a PR which uses Jev to check if the target length is beyond this range
You pad the text.
Oh, my god.
Tech doesn't teach it's own history in a useful way, so we keep repeating it too.