Top
Best
New

Posted by hidude562 3 days ago

Show HN: I made an audio-to-MIDI model 60x more efficient(kobi.music)
I released dettato, my open weight audio to midi transcription model. It beats muscriptor 1.3b on every set and is roughly 60x smaller, without training on copyrighted recordings. Feel free to ask questions
195 points | 36 commentspage 2
pier25 3 hours ago|
Very cool. This is the sort of stuff I want to see from AI.
dimmuborgir 2 hours ago||
Is there a way to force it to a single instrument? I uploaded audio files of pure piano music and the output midi files are split into piano, guitar and drum parts.
simedw 2 hours ago||
Very cool, I wonder if my MIDI generation model would improve if I trained it on the MIDI files your model generates.
oever 6 hours ago||
Audio-to-MIDI sounds very ambitious. How do you verify that the midi fits the audio? Does it result in useful beat detection? E.g. I'd very much like to have a way to map audio offset in a recording to labels in sheet music. But even beat detection in fairly standard jazz recordings fails often.

Does the model know when a piece (whole piece, not a section) is 3/4 or 4/4 th and can it give the onsets of all measures? Or all the midi notes just based on time and not organized rhythmically?

gunalx 4 hours ago|
Its definetly a problem where generating the trivial version of synthetic data is easy though.
jsphweid 6 hours ago||
I've been out of the loop since Magenta's MT3 model. This feels a lot better and significantly smaller. Very impressive, dude. And thank you!
gpugreg 5 hours ago||
Nice! Worked surprisingly well.

You might want to implement deduplication for your website so impatient users do not convert the same mp3 multiple times (at multiple times the price) when they click "run audio to midi" and the server takes a bit longer to respond.

cocodill 4 hours ago|
I guess impatient users could run audio to midi local.
mycall 6 hours ago||
I'm still waiting for someone to convert beatboxing to midi in realtime. That would be a game changer for that art.
maxgraf96 6 hours ago|
You should check out https://vochlea.com/ :)
mycall 5 hours ago||
> Trained on your own percussive sounds

This is probably where I would find it limiting range of sounds can be huge, much more than just 6 sounds (10000+) and not all of them percussive. Worth an eval though, thanks!

ch_sm 5 hours ago||
Just tried it with some home-recorded songs (pop/rock) — this works great!
gregzem 2 days ago||
Congrats with SOTA model. How good is it in recognition of the instruments which are overlapping by frequencies? Does is provide note offset and velocity in the transcribed MIDI? Thanks!
JodieBenitez 6 hours ago|
I need this in Bitwig !
pier25 3 hours ago||
yeah I was thinking that DAWs should be implementing this
thatmf 4 hours ago||
my exact thought. hmm. weekend project?
More comments...