Flavia Sparacino
flavia@media.mit.edu
The Improvisational Theater Space is an interactive stage where human
actors can perform accompanied by virtual actors. Virtual actors are modeled as
animated "Media Creatures" that are behavior-based automous software agents. It
uses real time computer vision, speech recognition and and speech anlaysis to
sense the perfomer's actions on stage. We used Artifical Life programming
methods and behavior-based design to avoid rigid scripting of user and content
interaction. The main result of this work is the construction of animated media
creatures endowed with intentionality and autonomous behaviors. Media Creatures
allow content to be active and to present itself to the user by dynamically
adapting to the context of the interaction. We used Media Creatures to create an
engaging Improvisational Theater Space where the user/performer is engaged in an
improvisational dialogue with a typographic actor.
The Improvisational Theater Space is an interactive stage where human
actors can perform accompanied by virtual actors. This space was created
following the model of the street theater, the mimes world and the
improvisational theater in general [Johnstone79].
We wanted to create an interactive stage where human and digital actors
meet to generate an emergent story through their interaction. An important
constraint was to bypass strict scripts of the encounter. We thought of an
Improvisational Theater Space not just as a performance space but also as a
playground for participatory theater, interactive storytelling, or museum
exploration.
By participatory theater we mean a situation where a user first watches a
performer interacting with a virtual actor and then is offered to take the place
of the human performer and experience an emergent story from a subjective point
of view. In this case it is important to avoid the need for the user to memorize
lines of text based on a one time observation of the performance. The user only
needs to roughly grasp the situation and the interaction modality in order to be
able to participate. This augmented storytelling environment responds to the
teller's gestures, words, and tone of voice by enriching the presentation with
sound, text, images, or graphics.
All of these scenarios need to satisfy the following necessary conditions
for the interactive experience to be truly engaging and immersive:
To satisfy the first condition, we created Theater Space as an
improvisational stage. A set of possible situations is set in advance and the
human actor is given the choice to change the order of the actions and the mood
in which scenes are played.
In order to have believable characters, we modeled the virtual actors
as Media Creatures. Media Creatures are autonomous agents with behaviors,
sensors and goals (internal motivations). A media creature knows whether its
content is text, image, a movie clip, sound or graphics and acts accordingly. It
also has a notion of its role and ``mood'', i.e. it can express basic emotions
like happy, sad, angry, scared etc. Media Creatures are modeled according to
Blumberg's tool kit for behavior-based animation of autonomous creatures
[Blumberg95]. The choreographic component of a media creature specifies its
behavior according to the context of the interaction with the performer. As
opposed to scripted animation that imposes a pre-defined sequence of actions to
a virtual actor, behavior-based animation, defines a tree of actions that are
driven either by the internal motivation of the creature or by the external
actions of the user/performer or by a combination of both. Hence the model of
the interaction between the human and the digital actor consists of a nearly
infinite tree of possibilities rather than a linear sequence of consecutive
actions. The autonomous structure of the Media Creatures allows them to exhibit
a behavior that "makes sense" to the user even when he/she is not "doing the
right thing". In addition the behavior system and sensors of the creatures
enable them to understand situations while interacting with the user/performer
and to act as improvisational theater performers. We use the term Media
Creatures or Media Actors or Media Tellers as synonymous, according to the
particular context in which they are interacting with the user/performer.
Finally for the third condition to be met the interactive stage uses
remote sensing technology to interpret the actions of the user/performer. This
work is premised on the assumption that immersive spaces require natural,
wireless interaction so that the user/performer is not encumbered with wires or
sensors. Most current virtual environments use bulky head-mounted displays, data
gloves and body suits with multiple cables. While such systems can be extremely
accurate they limit severely the freedom of movement of the user. This results
in an unnatural man-machine interaction and it is the first obstacle to creating
a truly engaging environment. The interactive Theater Space described in this
document use sensing devices which are embedded in the environment, endowing the
latter with perceptual intelligence. As people use primarily vision and audition
to communicate with each other, these interactive spaces use real time computer
vision and audition technology as their source of perceptual information.
In the past, the idea of creating performances that involve both humans
actors and inanimate objects as characters can be tracked back to the beginning
of the century. The avangard Italian Futurists movement inspired a new form of
theater called Synthetic Theater [Kirby71]. The theater of the Bauhaus also
experimented with a non-verbal, dance-inspired theater of objects with living
actors stylized as geometric, often cubic shapes, resembling modern marionettes
and automatons [Schlemmer71]. Although futurist performances did not have great
success with the public they can be seen as early tentatives of staging human
actors together with objects in a theater of situations that required
participation from the public.
We believe that the use of Media Creatures/Actors enhances possibilities
for communication in the performance art. For example although cinema has used
text in the past as a means to communicate the character's inner thoughts
instead of voice over - examples include Woody Allen's Annie Hall and Peter
Greenway's Prospero's Book - no similar effects have been used in theater to our
knowledge. The video accompanying this paper shows an example of use of a Text
Actor - modeled by a Media Creatures - that interacts with a human actor by
playing the role of his "alter ego".
Some authors ( [Pinhanez96]) have envisioned a computerized theater that
relies on the knowledge of the script to drive a computerized stage. The main
drawback of this approach is that it relies on the director and the actor to
rigidly follow a script for the system to be able to work. For instance it is
not uncommon in theater that both the actors and the director change the script
either during rehersals or even right before or during the final performance
[Brook95]. In our view this type of rule based system will not be able to
compensate for human errors or be responsive when some non planned ``magic''
between the actors happens on stage. It tends to force human interpreters to
rigidly follow a predefined track and therefore empoverishes the quality of the
performance.
In summary we believe that behavior-based media actors are a promising
approach to interactive theater for three main reasons:
1. Behavior-based vs
script based theater has room for improvisation, both in the case of the
improvisational or street theater in general or for classical scripted theater
that the director and the actors need to interpret and therefore modify.
2.
The system is tolerant to human error and actually encourages actors to enrich
or change the performance according to the reaction of the audience.
3. The
system can scale from a performace space to an entertainment space.
Behavior-based theater can allow for user participation either during or after
the performance without requiring the new users to learn all the script in
advance.