Top
Best
New

Posted by hidude562 3 days ago

Show HN: I made an audio-to-MIDI model 60x more efficient(kobi.music)
I released dettato, my open weight audio to midi transcription model. It beats muscriptor 1.3b on every set and is roughly 60x smaller, without training on copyrighted recordings. Feel free to ask questions
195 points | 36 comments
alin23 15 minutes ago|
What a coincidence! I was just in the process of integrating MuScriptor on a personal macOS app [0] I work on and off for helping me practice my guitar, kaval and trumpet skills.

I'm so glad to see a smaller model being so accurate, MuScriptor is especially bad on the Romanian kaval, can't wait to try this in-app. I do it with already separated stems through BS-RoFormer so I'm hoping this can improve accuracy a bit.

[0] https://github.com/FuzzyIdeas/Muskey

gw32 1 hour ago||
This is fantastic. I love the commitment to train only on PD recordings. In my opinion, that is an essential to maintaining the favor of musicians - building technology not to take from their work and replace them, but rather to celebrate human-written music by making it accessible.

This is why I believe (AMT, OMR) and MIR-adjacent projects to be so much more meaningful than generative models (e.g. Suno). I'm looking forward to hearing more from kobi music - congrats on the launch!

peaseagee 6 hours ago||
Congrats on training on open and public domain recordings. Thanks for proving this kind of work can be done in a more ethical way.
piltdownman 6 hours ago|
TBF it's by far the best corpus for this use case anyway.

Classical Music as an academic concern is very pedantic in the best way, and has the lions share of the research hours and funding compared to any other genre. As a result the best 1:1 approximations of multi-instrument Audio into multi-channel MIDI files is done simply by OCR of the established Gesamtausgabe in each case. Computer based scoring and playback has been solid since Finale/Sibelius/Score came on the market, and democratized at a wide consumer level somewhere between Amiga mod-trackers and Macintosh Quicktime 2.0.

dekhn 1 hour ago||
I tried this on a Phish guitar track that I have and while GPU failed with an error, CPU eventually finished and it did a pretty good job! The recording is 45 minutes long, so getting GPU working is my next interest.

Unfortunately, Phish live recordings are copyrighted, so it's unlikely you can put them in the training corpus. Demonstrating that you can deconvolute and transcribe multiple different instruments with highly overlapping frequencies and sounds faster than an expert human would be a great demonstrate of the capabilities.

tough 29 minutes ago|
could you have personal use fine-tunes for copyrighted material as long as you don't publish it
thih9 3 hours ago||
Congrats on the launch!

The top illustration looks like an Angry Birds character with an ear. Fine for a one time blog post but if you plan to reuse it then you may want to change the design.

ohmahjong 2 hours ago||
It is a reference to an Angry Birds meme: https://knowyourmeme.com/photos/3226156-angry-bird-ear-angry...
epiccoleman 5 hours ago||
That is awesome! I'll definitely have to check it out. I was very impressed with muscriptor when I used it in a recent project (a conference talk I gave). It was an AI talk with a musical component, and the final "scene" in the talk involved my own guitar playing. I used Muscriptor to transcribe that playing for categorization as part of a "guitar solo generator" that recut a new solo out of my stock phrases each time. Really fun little demo for a bigger project.

Anyway, it would be awesome to be able to do those sorts of transcriptions better and faster - so I'll definitely give this a whirl. Maybe I can dig up the pipeline scripts from processing my licks for that talk and see how dettato slots in.

hdb2 1 hour ago||
I just signed up and fed it a piano solo from a Dixie Dregs tune, and it nailed it. this is very impressive for players like myself to transcribe things very, very quickly.
rogerdickey 4 hours ago||
all the examples seem to be classical music or largely instrumental stuff. how well does it work for things like pop music, rap?
SoleilAbsolu 2 hours ago||
Curious what's the performance for both note duration, and note pitch? How does it do for tunings/temperaments outside 12TET? Very refreshing to not violate copyright in this day and age amongst all the greedy grabbers!
locusofself 2 hours ago|
Very cool, I have been working on a DAW for several months that is getting close to a beta release. I'd love to incorporate this.

As a guitarist who can't play keys well, I'd love to use something like this to help me transcribe string arrangements.

More comments...