Overview
The Emonator is one of the interfaces - and the most versatile at that - used to provide control input for the Emonic Environment, a real-time improvisational media system. Originally, the Emonator was conceived as a combination of software and hardware with the aim of exploring the concept of emons as the building blocks for aesthetic expression, content delivery, and information exchange. The Emonator was designed in collaboration with Dan Overholt, on whose page you can find details of the hardware design (Dan also later adopted the name MATRIX to refer to the device in his own, independent work). The rationale for the design of the Emonator was to generate emons of varying complexity and forms by interpreting pressure-like hand gestures.

The Emonator can serve as a multipurpose application for interactive control of music and video, inviting users to create aesthetic experiences using emotionally expressive gestures (particularly when used within the Emonic Environment). The aim is to allow everyone to be a composer / DJ / video editor without having to learn a complex system of mappings. Traditional performance instruments force the user into predefined ways of creating emotional expression (as well as existing electronic instruments). It is our goal to create a truly new and adaptive interface, which will acknowledge the natural ways of emotional expression for creating an aesthetic experience. This way we hope to make the Emonator into an affective performance instrument that will allow you to 'play' your feelings without having to learn a complex system of mappings.

Currently we are developing a line of applications that allow the Emonator to interpret gestures for a dynamic creation of music and video. The music controller provides three levels of music control - an audio shaper, a pattern composer, and a pattern navigator(not yet functional). The video controller allows users to navigate through and mix multiple live video streams (currently limited to 4).

The device hardware consists of a 12x12 array of closely positioned spring loaded plexiglas rods. The position of each rod (known to a 7-bit precision) is sensed using a pair of infrared light detectors and is interpreted using the underlying FPGA-based motherboard. Each rod's position is then transmitted to a connected PC, which uses these signals to generate music and video. We also plan to provide an additional layer of action feedback by using computer-controlled RGB-colored lights that will shine through the plexiglas rods. Full details about the hardware can be found in Dan's thesis (PDF 26MB).

Data Gathering and Analysis
The Emonator project serves two purposes in the overall development of the Emonic Environment:
At the first stage, we are using it as a measurement / analysis interface for premade emotional musical patterns (emons). A user is introduced to the Emonator application and asked to tangialize (move from the auditory to the physical domain) a series of emons (short musical phrases) that are played for him. The data from multiple subjects gathered above (pressure, speed, movement patterns, etc) is then analyzed to derive common physical patterns of response to each of the emons. Along with providing interesting scientific results (for interpretation of affect), this will also allow us to refine the Emonator's performance.

We hope that the Emonator will allow us to check the hypothesis that a person can take a coherent control of real-time sources of affect, and orchestrate a unique experience corresponding to their emotional state. The experimental data gathered at the first stage will allow us to create mappings that will be used by the performer in the creation of a free form musical / visual structure. Rather than relying on a user's expertise and performing ability, the Emonator will give its user the freedom of emotionally natural tangible expression.

The Emonator was the first of the Emonic Environment's components developed; now there are quite a few more. It is our hope that the Emonator will provide empirical evidence for the validity of our composition approach and techniques, and will help us increating large-scale art environments (such as the one described in the EmoWall proposal). It is our small-scale attempt at creating a coherent emotion-adaptive system of interactions between the Emonic Environment and the visitor. The Emonator has also been used as a controller in a museum installations.

Preliminary videos showing the MATRIX performing real-time Granular Synthesis over the Internet:
If you are on a high speed connection, try the QuickTime: video I, video II, video III. (10-23MB files).
If you are on a slow connection try the Windows Media copy: video I, video II, video III (1-4MB files).

NEW: a gesture-based application using all 12 rows (144 rods) (update soon).

Project Status
Current Emonator version: 0.9. Stay tuned.
Please e-mail your questions to pauln at media.mit.edu