Top
Best
New

Posted by get-inscribe 9 hours ago

Apple's new SpeechAnalyzer API, benchmarked against Whisper and its predecessor(get-inscribe.com)
409 points | 172 commentspage 2
port3000 8 hours ago|
I use Spokenly, offline-only mode with the Nvidia model. All local, totally free. Highly recommend
cootsnuck 3 hours ago|
For just desktop, I use Handy (open source): https://handy.computer/

And yea, Nvidia's Parakeet v3 is good enough for my own just local transcription most of the time.

When I need local transcription to be more reliable and I don't have the energy to proof read a long ramble, I still often just pop open chatGPT, dictate, cut, paste.

But we're pretty much already to the point where local transcription models can replace cloud ones for personal use. They're still a bit rough around the edges in terms of polish and latency, but plenty of people are fine with that to avoid yet another app subscription and not having to worry about wondering what's potentially happening with their data.

port3000 2 hours ago||
Thanks will check it out, hadn't heard of it
Tsarp 8 hours ago||
Any chance you can benchmark against whisper large and large v3 turbo? These run comfortably on older Macbooks and are still far more accurate in real life dictation compared to even the parakeet models( despite ASR leaderboards) with an RTF < 1.
wahnfrieden 8 hours ago|
Try MOSS-Transcribe-Diarize from a few days ago. I’m getting better results than those whisper models. And it’s very fast and small. Better suited to noisy audio too.
mcchen51 3 hours ago||
I run SuperWhisper on both my Mac (where is uses Whisper) and my iPhone (where it uses SpeechAnalyzer and have found that SA does indeed run faster and anecdotally more accurately. Super exciting!
gdonelli 8 hours ago||
I second that! Can you run your benchmarks against the iOS 27 beta?
endymi0n 6 hours ago||
Finally. I‘d be delighted though if they actually implemented language autodetection (like everywhere else) though. There’s little more frustrating in my day to day than having dictated half a page to find that it‘s complete gibberish because Apple forces you to select the right language first…
QGQBGdeZREunxLe 6 hours ago|
Same with the keyboard. Apple is completely incapable of taking context into account for the input mechanisms of the operating system.

If I start typing and the existing text is in Spanish, then a sensible default is to select the Spanish keyboard I have installed and let me adjust otherwise.

App developers should also be allowed to supply mini-dictionaries within a context to allow autocorrect to work correctly in that context, so for example in this thread [SpeechAnalyzer, API, Whisper, Parakeet, Nemotron] should be supplied so that these terms are autocorrected.

sudb 7 hours ago||
For my current purposes, I need a speech-to-text model/API to also emit word-level timestamps - for now, that makes ElevenLabs's Scribe v2 the best multiplatform, multi-language choice though it does look like this SpeechAnalyzer API provides them (although only for English).
dclowd9901 6 hours ago||
I'm always confused by these phrases:

> The new API cuts word error rate by 3.5 to 4x on the same audio: from 9.02% to 2.12% on clean speech

Shouldn't they have said "cuts error rate by 78%" or something?

clickety_clack 5 hours ago|
I don’t like it written that way either, and it always seems like the type of number you put on a slide for a head of sales or something. It rankles because:

- it implies that error could be increased n-times, but a 15x _increase_ in 9% error would be an error rate of 135%, which is nonsensical.

- a reduction from 90% error to 20% error is clearly a bigger improvement in rightness to a reduction from 9% to 2%. One is “almost all wrong to almost all right”, the other is “more right”, but they are both a 4.5x reduction in error which means that the 4.5 quantity doesn’t have a constant meaning.

The answer is something like log odds ratios, but that introduces the additional need for a reader to know what that is, and that would be unusual.

JV00 4 hours ago||
Whisper large v3 turbo works fine locally on recent iphones, weird to keep it out of the comparison
anonymouse008 4 hours ago||
Excited to get next years diarization models - ArgMax is pretty incredible but cost prohibitive (when compared to 0 price from iOS)
ks2048 8 hours ago|
Lots of comments are "you should compare against X and Y" - even better, just get the results on a standard benchmark, so you can compare against all,

https://huggingface.co/spaces/hf-audio/open_asr_leaderboard

jiehong 7 hours ago|
well, it's nice, but the multi-lingual diff is rather limited (only European languages).
More comments...