Gesture-Driven Interaction Using the Leap Motion to Conduct a 3D Particle System: Evaluation and Analysis of an Orchestral Performance

Authors

  • Joyce Horn Fonteles UNIFOR
  • Édimo Sousa Silva UNIFOR
  • Maria Andréia Formico Rodrigues UNIFOR

Keywords:

Gestural Interaction, 3D Particles, Orchestral Arrangement, Visualization, Performance Analysis, Evaluation, Leap Motion

Abstract

In this work, we present and evaluate an interactive simulation of 3D particles conducted by the Leap Motion, for an orchestral arrangement. A real-time visual feedback during gesture entry is generated for the conductor and the audience, through a set of particle emitters displayed on the screen and the path traced by the captured gesture. We use two types of data input: the captured left and right hand conducting gestures (some universal movements, such as the beat patterns for the most common time signatures, the indication of a specific section of the orchestra, and the cutoff gestures), which are responsible for setting the tempo and dynamics of the orchestra; and a MIDI file, which contains information about the score of an orchestral arrangement, defining which notes each musical instrument should play. As regards the gestural input of the conductor, we have considered two musical elements of expression: tempo and dynamics. Besides performing functional testing, we analyzed the simulation results focusing on the occurrence of false positives, false negatives, positional deviations and latency. Moreover, a professional conductor evaluated our system and provided qualitative feedback about it.

Downloads

Download data is not yet available.

Downloads

Published

2015-11-17

Issue

Section

Special Issue - SVR 2015