| |
 |
|
MIDI Keyboard Image Parser
|
||
| Hands
and keys contour extraction from a MIDI keyboard image |
The MIDI
keyboard image parser consists of two modules, with one module
performing the extraction of the contour of a hand situated above a
musical instrument keyboard, and the other module performing the
computation of the geometry of the polygons which outline/represent the
individual keys of a musical instrument keyboard.
This project was ordered by a DAW software startup as part of a VST plugin solution aimed at providing a low-cost, image processing-based solution for extracting information on hand and/or finger gestures over a MIDI keyboard's keys, and then use the extracted information to send MIDI signals to a DAW in order to achieve various programmable sound effects (e.g. alter the tonality of the sound, introduce special effects such as tremolo or vibrato, sliding notes, etc).
Solutions targeting a similar feature set as the above-mentioned VST plugin already existed on the market at the time this project was started, but said solutions are all using various kinds of touch sensors placed on the keyboard, or they consist of special touch-sensitive membrane designs, etc. No existing solution on the market at the time of writing (october 2021) provides a similar feature set via digital image processing.
The videos below provide a glimpse into how various sound effects can be produced by a DAW by extracting gesture information from the fingers and/or hands motion over a MIDI keyboard (note that the DAW plugins utilized in the videos below require the use of specially crafted MIDI keyboards which contain touch-position detection via [various types of] hardware sensors).
This project was ordered by a DAW software startup as part of a VST plugin solution aimed at providing a low-cost, image processing-based solution for extracting information on hand and/or finger gestures over a MIDI keyboard's keys, and then use the extracted information to send MIDI signals to a DAW in order to achieve various programmable sound effects (e.g. alter the tonality of the sound, introduce special effects such as tremolo or vibrato, sliding notes, etc).
Solutions targeting a similar feature set as the above-mentioned VST plugin already existed on the market at the time this project was started, but said solutions are all using various kinds of touch sensors placed on the keyboard, or they consist of special touch-sensitive membrane designs, etc. No existing solution on the market at the time of writing (october 2021) provides a similar feature set via digital image processing.
The videos below provide a glimpse into how various sound effects can be produced by a DAW by extracting gesture information from the fingers and/or hands motion over a MIDI keyboard (note that the DAW plugins utilized in the videos below require the use of specially crafted MIDI keyboards which contain touch-position detection via [various types of] hardware sensors).