Massachusetts Institute of Technology
M A T R I X

Multipurpose Array of Tactile Rods for Interactive eXpression


The MATRIX is a new musical instrument that gives users a 3-dimensional tactile interface to control sound with their hand(s). It can be used as either a stand alone instrument or in conjunction with a traditional musical keyboard or microphone. The MATRIX generates musical output by mapping a performer's expressive gestures to a variety of sonic parameters. It acts as an input device that directly manipulates the parameters of a synthesis engine (eg. additive, granular, wave-terrain...), or an effect alogrithm (eg. delay, reverb, filter-banks...) in response to the changing shape of the interactive surface. In this way, the MATRIX provides a very intuitive method of manipulating sound with an amount of control that has never before been implemented in real-time.

One mapping uses the individual rods of the MATRIX to directly control the digital waveform of a sound, thereby "sonically sculpting" the timbre of the sound. An example of this technique is shown below in the excerpt from one of my compositions titled Dactylonomy. It is a duet for MIDI keyboard and MATRIX, and utilizes a custom synthesis technique that is actually a cross between Wave-terrain and Scanned synthesis (see "New Musical Mappings for the MATRIX" for more information). The video below shows the last 1:30 of the piece, which premiered August 17th, 2002 at the Woodstockhausen Electronic Music Festival in Santa Cruz, California.

  • Dactylonomy Excerpt - 8MB QuickTime, duet with Stefanie Ku.

    Below are some video clips of the MATRIX performing granular synthesis:

  • Granular synthesis video 1 - 15MB QuickTime, shows the MATRIX modifying my voice.
  • Granular synthesis video 2 - 23MB QuickTime, same synthesis engine with a female voice.
  • Granular synthesis video 3 - 10MB QuickTime, real-time use with a violin input signal.

  • Here are the same videos in Windows Media format (1-3MB): Video 1, Video 2, Video 3.

  • Audio only clip - 260KB mp3 (10 seconds) sample of the MATRIX modifying my violin sound.


    Please send comments and/or questions to me (Dan Overholt) at dano@media.mit.edu.

    The MATRIX interface is also used by Paul Nemirovsky in the Emonator project.*


    Publications

    Control of Signal Processing Algorithms using the MATRIX Interface (884KB PDF)
    Proceedings of the 114th Audio Engineering Society Convention '03 (Amsterdam, The Netherlands, March 22-25, 2003).

    New Musical Mappings for the MATRIX Interface (1.1MB PDF)
    Proceedings of the International Computer Music Conference '02 (Goteborg, Sweden, September 16-21, 2002).

    The MATRIX: A New Musical Instrument for Interactive Performance (140KB PDF)
    Proceedings of the International Computer Music Conference '01 (Havana, Cuba, September 17-22, 2001).

    Multipurpose Array of Tactile Rods for Interactive eXpression (160KB PDF)
    Conference Abstracts and Applications of SIGGRAPH '01 (Los Angeles, California, USA, August 12-17, 2001).

    The MATRIX: A Novel Controller for Musical Expression (888KB PDF)
    New Interfaces for Musical Expression Workshop at CHI '01 (Seattle, Washington, USA, April 1-2, 2001).

    My Master's Thesis (26MB PDF) from MIT describes the design and development of the MATRIX interface.


    Credits

    The MATRIX began as a collaborative project between the
    Hyperinstrument and Interactive Cinema groups at the MIT Media Lab.
    I would like to thank professors Tod Machover and Glorianna Davenport for their support.

    Design of the MATRIX interface & musical applications: Dan Overholt
    Data conversion, network, and some MIDI applications: Paul Nemirovsky
    Vocalator application & help with SuperCollider, MAX, and network programming: Tristan Jehan
    Data analysis, video applications, & MATRIX construction for Emonator project: Andrew Yip
    Help with data analysis & network libraries: Ali Rahimi
    Granular synthesis videos shot/edited by Ali Mazalek and Dan Overholt


    * The interface idea was originally conceived jointly by Dan Overholt and Paul Nemirovsky.
    Although the interface was initially called the Emonator, the project later separated into two main directions:
    (1) The MATRIX project focuses on the interface design and development of musical synthesis and signal processing applications, and
    (2) The Emonator project currently uses the MATRIX interface to explore the concept of emons as building blocks for audiovisual content creation.