Audio to MIDI converter

An audio to midi converter capable of processing three types of sounds:

  1. Rhythmic, for drums and other trigger sounds, processed in a Drum Machine track.
  2. Melodic, for monophonic instruments and voice.
  3. Harmonic, for chord extraction in polyphonic instruments.

What problem(s) would this feature resolve?

Bitwig producers must rely on plugins or other DAWs to extrac MIDI from audio clips easily and reliably. Bitwig offers a Replacer device and a Zero Crossings module in The Grid, but their current usefulness for most producers is rather small.

How does this feature fit in Bitwig as a product?

Bitwig aims to Streamline your creative process and quickly evolve your ideas into complete songs, tracks, and compositions. Often ideas emerge in the brain and take shape as audio while humming, whistling, clapping, tapping on a table, or casually playing a basic instrument. They also emerge while listening to recorded audio. Getting the inspiring sounds in MIDI form is a frequent need in the creative process.

Is there already an alternative way to achieve this on Bitwig?

The Replacer device can detect triggers and convert them to midi at a set pitch. It won’t distinguish between e.g. a kick, a snare and a hi-hat.

The Zero Crossing module in The Grid can be used to detect pitch in a monophonic audio source, but it takes advanced Grid skills to get a working solution.

Could it be implemented using Bitwig components or APIs, without Bitwig team’s support?

No

Could it be provided by a VST or something else reasonably integrated with Bitwig?

Yes, Melodyne is probably the most popular plugin that extracts midi. There are also plugins that replace drum sounds as well.

Are there other products that offer this feature?

Ableton Live and Reason.

Relevant links (optional)

4 Likes

14 posts were split to a new topic: [Draft discussion] Audio to MIDI converter

(Moved from Pitch detection / correction inside the audio editing window)

For example, this is how the audio midi converter works in Studio One:
aaamp
The Melodyne effect is open on the audio track, notes are already recognized. And simply dragging and dropping the audio clip onto the midi track converts the Audio to Midi. But these will just be approximately recognized midi notes, without any nuances such as vibrato, legato, etc. Therefore, we need an MPE.

4 Likes

This is approximately how a midi-track converted from an audio track in Bitwig might look like:

3 Likes

we can go further and add a feature of recognizing the amplitude curve, and assigning it to note expression
2021-12-14_13-16-14

3 Likes

Damn this actually has quite the potential with the aforementioned MPE mapping.

2 Likes

Is there anything that can extract chords out of audio, sounds pretty complex to develop?

Yes, Celemony Melodyne can

2 Likes

Yes, but than we still need ARA2 support…

Polarity gets a step closer for monophonic audio using Note Grid. Not there yet. :slight_smile:

Polarity Music - why not sing your notes? - Bitwig Studio 4.2b3

1 Like

Could deCoda be an affordable workaround? Has anyone tried it? https://products.zplane.de/products/decoda/

1 Like

If they simply support ara, this could be achieved with ara and there’d also be all the additional benefits of melodyne. Of course that’d require people to get melodyne but it is really good.
Having a quick and dirty version like Ableton is no bad thing.

My vote goes here for now

1 Like

I love polarity musics latest effort of transforming beatboxing to midi in his recent video called “vochlea is done” :slight_smile:

Nevertheless this still isnt as “direct” and precise as Dubler2, as it seems. AND a Bitwig internal solution would surely make things easier when it comes to usability and precision! That’s why i still vote for a Bitwig solution, despite all the nice Grid-efforts.
I also mentioned audio to midi in my huge feature request Video here:
Minute 27:50 :slight_smile:

Ive been observing this sort of feature from afar for some time and considering purchasing plugins or software that can do it, but havent made any decision. I just bought my full Bitwig license yesterday after 3 weeks of learning, and now that I see this feature/tool being voted on for development inside Bitwig – eureeka!!! I need this feature!! Extract MIDI notes from audio, PLEASE!

Perhaps this can be a tool that can be pinned to the main toolbar, or brought forward somehow. I would use this all the time to hum and sing notes!!

For those who aren’t aware of it NeuralNote converts audio to midi and is free.
Download link: Release v0.0.1 · DamRsn/NeuralNote · GitHub

For drums it hasn’t given me any good results, but for melodic stuff it’s worked very well with everything I’ve thrown at it.

1 Like

NeuralNote demoed on Bitwig:

2 Likes

This feature request actually needs updating, because virtually almost every daw has some sort of feature like this. To the list need to be added Studio One,Cubase, FL Studio and Logic.

The heart of Bitwig’s development is experimentation and creative freedom. This feature fully goes in that direction and is much needed. I hope they will implement it soon

Anyone can edit the description and the tags. If you have the information, your update is welcome!

yep this is super useful, i use it in ableton all the time after humming a melody in the mic or beat boxing