"Fly me to the moon, let me play among the stars"

Exploring a Musical Interface for Audiovisual Expression and Navigation in Astrovisualization

Abstract

This extended abstract presents ongoing work of a MIDI integration with the visualization software OpenSpace. Rather than only mapping commands to individual buttons or keys on a MIDI piano, the continuous playing of the piano drives the camera and the scene composition of OpenSpace in real-time. This creates a hybrid MIDI interface, where continuous features of the playing the piano (such as note density, velocity, register, and polyphony) control how fast the camera travels, how it is oriented, and which of scene layers are visible. Discrete features, such as which bass note was most recently played, controls towards which object the camera travels to. This creates an audiovisual congruence, in which the visuals and the played music become a single act, which not only immerses the audience, but can also inspire the performer during improvised play. The current mappings are described, while also reflecting on future implementations and use cases.