subpixels.com | SPXL.TVblog | contactgallery | videoflash | processingmyspace | facebook

spxlAudioToMidi

by subpixel, subpixels.com

Description

A small application made with Processing to produce MIDI control messages from a live audio feed. Auto-levelling frequency band meters are used to obtain "peak" output values across the entire audio spectrum analysed (whereas it is usual to have strong bass response with weaker treble response), and easing is applied to make the levels less erratic. In addition to tracking the high values, low values are tracked and the output range is maximised to span the low to high range. 10 separate monitor outputs are supplied, where each monitor is attached to one of the frequency band meters and that meter's output is scaled to the particular output range for the monitor, allowing for a reduced final output range as well as inverted ranges. Controls are provided to select the MIDI output device and the MIDI channel to use, as well as for selecting which monitors are connected to which meters and the monitor output ranges.

This app has been used to control the VJ applications Resolume and Neon v2, allowing simultaneous control from an external MIDI controller on the same channel (using MIDI-OX and MIDI Yoke for routing).

Download

2016-09-10 spxlAudioToMidi10: developed and tested with Processing 2.2.1 under Windows 10 Professional (64-bit). Unknown at this stage as to what changes may need to be made for Processing 3.x.

A copy of my sketchbook/libraries/promidi folder can be had here: promidi.zip (1MB).

Older version(s)

2014-06-08 spxlAudioToMidi08: developed and tested with Processing 2.0.3 under Windows 7 Professional (64-bit); (re-)built with Processing 2.2.1 under Windows 10 Professional (64-bit)

2010-06-25 spxlAudioToMidi04: developed and tested with Processing 1.1 under Windows 7 Professional (64-bit)

Demo Videos

Operation

Special keys: F3 = sample audio from "Microphone" mixer and F4 = sample audio from "Stereo Mix" mixer can only be used once each time the app is run due to an unsolved problem. Also, choose the MIDI channel (the numbers at the top left of the window) before selecting the MIDI device, since there is some separate problem with re-opening the MIDI devices.

The text at the top of the window controls the MIDI output. The left column of 30 bars are audio meters; each meter representing an average for a frequency band produced by FFT analysis with the low frequencies at the top of the display and the high frequencies at the bottom of the display. The meters are auto-levelling and provide normalised output values between 0 and 1 (inclusive). The right column of 10 bars are meter monitors, with each one connected to a particular meter. A monitor's output value is determiend by scaling the input meter value over the monitor's output range between 0 and 127 (inclusive); the lower and upper limits of each range can be adjusted, including to scale the meter input inversely, and the meter it is connected to (thus the frequency band it monitors) can be changed. If a MIDI output device is selected, each time a monitor output value changes, the value is emitted as a MIDI control message on the selected output channel. The control number is in the range 1 to 10, with the top monitor being control 1 and the bottom one being control 10.

Selecting MIDI output device

Click on the "SELECT OUTPUT" text to pop up the "SELECT DEVICE:" menu. Click a displayed device to select. Subsequently, clicking a displayed device name presents the "SELECT DEVICE:" menu again, and clicking the displayed channel number presents a "SELECT CHANNEL:" menu with channels 1 to 16.

Adjust monitor output ranges

Each monitor ouutputs values in the range 0 to 127 (inclusive) by default. The "minimum" and "maximum" limits can be adjusted by clicking (and dragging) the monitor output bar. Clicking in the top 3/4 of the bar adjusts the maximum; clicking in the top 1/4 of the bar adjusts the minimum. When dragging, if the mouse is dragged above or below the bar, the limit being dragged (min or max) snaps back to the previous value. Right-click the bar to reset to the default full range (0 to 127).

Changing monitor links

When the mouse is just off the left edge of a monitor bar the link to the associated meter is hilighted and "Drag to meter" is displayed. Click and drag the mouse pointer, releasing over to the desired meter; releasing when not over a meter leaves the link as it was previously. Right-clicking when "Drag to meter" is shown resets the link to the meter assigned by default when the application was started.

Keyboard controls

The "selected monitor" refers to the monitor that the mouse is over.

Space / P
[P]ause (and unpause)
M
[M]ute meter inputs; Shift+M for a mute shot (a few values output while muted)
E
Toggle [E]asing of meter values (on by default)
Shift+E
Reset [E]asing amount
UP / DOWN
Increase / Decrease easing amount
Z
Reset ("[Z]ero") the meter levels
Q
randomise links (to meters) for all monitors; Shift+Q to randomise link for selected monitor
A
reset links (to default meters) for all monitors; Shift+A to reset link for selected monitor
F
[F]lip range (swap min and max output values) for all monitors; Shift+F to flip selected monitor
V
in[V]ert range (sets min to 127-min, max to 127-max) for all monitors; Shift+V to invert selected monitor
R
[R]andom range (sets min and max output values) for all monitors; Shift+R to randomise selected monitor
S
re[S]et range to default (sets min to 0, max to 127) for all monitors; Shift+S to reset selected monitor
U
Random [U]pper range (sets max output values only) for all monitors; Shift+U to randomise selected monitor
Y
reset range upper value to default (sets max to 127) for all monitors; Shift+K to reset selected monitor
L
Random [L]ower range (sets min output values only) for all monitors; Shift+L to randomise selected monitor
K
reset range lower value to default (sets min to 0) for all monitors; Shift+K to reset selected monitor
1234567890
Set MIDI output channel (channels 1 to 10 only, 0 for channel 10)
!@#$%^&*()
Set the the MIDI input channel (channels 1 to 10 only)
H
Toggle display of values in [H]exadecimal
F1
Toggle display of meters
F2
Toggle display of monitors
F3
Analyse microphone input
F3
Analyse stereo mix

Screenshots


Default state

Select MIDI device

Link to meter part 1

Link to meter part 2

Hover to show value

Set max output
Click (and drag)

Set min output
Click (and drag)

Random links,
random output ranges

Select MIDI channel

Pause + mute