Posted by aloukissas 10 hours ago
— "Let's build efficient classifiers"
— "Forget about the silly classifiers, LET'S BUILD A MACHINE GOD"
— "Ah, that machine god is often too expensive and underwhelming. Let's build efficient classifiers"
Now you can iterate on your 20 ticket categories in days against real workloads and not commit to training an efficient model until you've proven the prototype in production.
That's like... really basic LLM knowledge? Like how e.g. this article measures how often answers change based on order https://zachlandes.com/trolley-problem/#appendix-a:~:text=Ch...
Or is this the Claudish "One thing that suprised us" that just means "and here's another thing"?
Here's an interesting reference on this that gives a bit more background information:
-- 8< --
Because LLMs (and many other ML systems) are chaotic, it is possible to manipulate them into doing something unexpected through a small, apparently innocuous change to their input. These changes can be illegible to human observers, which makes them harder to detect and prevent.
For example, flipping a single pixel in an image can make computer vision systems misclassify images. You can replace words with synonyms to make LLMs give the wrong answer, or introduce misspellings or homoglyphs. You can provide strings that are tokenized differently, causing the LLM to do something malicious. You can publish poisoned web pages and wait for an LLM maker to use them for training. Or sneak invisible Unicode characters into open-source repositories or social media profiles.
-- >8 --
Source: https://aphyr.com/posts/412-the-future-of-everything-is-lies...
A modern model like Haiku 5.5 is still going to be overkill for many of these decision model use cases, and is probably at least 10x the number of parameters (Haiku 20-50B ??? vs Strand Decsions 2B Gemma 4).