The Improvisational Theater Space is an interactive stage where human actors can perform accompanied by virtual actors. Virtual actors are modeled as animated "Media Creatures" that are behavior-based automous software agents. It uses real time computer vision, speech recognition and and speech anlaysis to sense the perfomer's actions on stage. We used Artifical Life programming methods and behavior-based design to avoid rigid scripting of user and content interaction. The main result of this work is the construction of animated media creatures endowed with intentionality and autonomous behaviors. Media Creatures allow content to be active and to present itself to the user by dynamically adapting to the context of the interaction. We used Media Creatures to create an engaging Improvisational Theater Space where the user/performer is engaged in an improvisational dialogue with a typographic actor.
The Improvisational Theater Space is an interactive stage where human actors can perform accompanied by virtual actors. This space was created following the model of the street theater, the mimes world and the improvisational theater in general [Johnstone79].
We wanted to create an interactive stage where human and digital actors meet to generate an emergent story through their interaction. An important constraint was to bypass strict scripts of the encounter. We thought of an Improvisational Theater Space not just as a performance space but also as a playground for participatory theater, interactive storytelling, or museum exploration.
By participatory theater we mean a situation where a user first watches a performer interacting with a virtual actor and then is offered to take the place of the human performer and experience an emergent story from a subjective point of view. In this case it is important to avoid the need for the user to memorize lines of text based on a one time observation of the performance. The user only needs to roughly grasp the situation and the interaction modality in order to be able to participate. This augmented storytelling environment responds to the teller's gestures, words, and tone of voice by enriching the presentation with sound, text, images, or graphics.
All of these scenarios need to satisfy the following necessary conditions for the interactive experience to be truly engaging and immersive:
To satisfy the first condition, we created Theater Space as an improvisational stage. A set of possible situations is set in advance and the human actor is given the choice to change the order of the actions and the mood in which scenes are played.
In order to have believable characters, we modeled the virtual actors as Media Creatures. Media Creatures are autonomous agents with behaviors, sensors and goals (internal motivations). A media creature knows whether its content is text, image, a movie clip, sound or graphics and acts accordingly. It also has a notion of its role and ``mood'', i.e. it can express basic emotions like happy, sad, angry, scared etc. Media Creatures are modeled according to Blumberg's tool kit for behavior-based animation of autonomous creatures [Blumberg95]. The choreographic component of a media creature specifies its behavior according to the context of the interaction with the performer. As opposed to scripted animation that imposes a pre-defined sequence of actions to a virtual actor, behavior-based animation, defines a tree of actions that are driven either by the internal motivation of the creature or by the external actions of the user/performer or by a combination of both. Hence the model of the interaction between the human and the digital actor consists of a nearly infinite tree of possibilities rather than a linear sequence of consecutive actions. The autonomous structure of the Media Creatures allows them to exhibit a behavior that "makes sense" to the user even when he/she is not "doing the right thing". In addition the behavior system and sensors of the creatures enable them to understand situations while interacting with the user/performer and to act as improvisational theater performers. We use the term Media Creatures or Media Actors or Media Tellers as synonymous, according to the particular context in which they are interacting with the user/performer.
Finally for the third condition to be met the interactive stage uses remote sensing technology to interpret the actions of the user/performer. This work is premised on the assumption that immersive spaces require natural, wireless interaction so that the user/performer is not encumbered with wires or sensors. Most current virtual environments use bulky head-mounted displays, data gloves and body suits with multiple cables. While such systems can be extremely accurate they limit severely the freedom of movement of the user. This results in an unnatural man-machine interaction and it is the first obstacle to creating a truly engaging environment. The interactive Theater Space described in this document use sensing devices which are embedded in the environment, endowing the latter with perceptual intelligence. As people use primarily vision and audition to communicate with each other, these interactive spaces use real time computer vision and audition technology as their source of perceptual information.
In the past, the idea of creating performances that involve both humans actors and inanimate objects as characters can be tracked back to the beginning of the century. The avangard Italian Futurists movement inspired a new form of theater called Synthetic Theater [Kirby71]. The theater of the Bauhaus also experimented with a non-verbal, dance-inspired theater of objects with living actors stylized as geometric, often cubic shapes, resembling modern marionettes and automatons [Schlemmer71]. Although futurist performances did not have great success with the public they can be seen as early tentatives of staging human actors together with objects in a theater of situations that required participation from the public.
We believe that the use of Media Creatures/Actors enhances possibilities for communication in the performance art. For example although cinema has used text in the past as a means to communicate the character's inner thoughts instead of voice over - examples include Woody Allen's Annie Hall and Peter Greenway's Prospero's Book - no similar effects have been used in theater to our knowledge. The video accompanying this paper shows an example of use of a Text Actor - modeled by a Media Creatures - that interacts with a human actor by playing the role of his "alter ego".
Some authors ( [Pinhanez96]) have envisioned a computerized theater that relies on the knowledge of the script to drive a computerized stage. The main drawback of this approach is that it relies on the director and the actor to rigidly follow a script for the system to be able to work. For instance it is not uncommon in theater that both the actors and the director change the script either during rehersals or even right before or during the final performance [Brook95]. In our view this type of rule based system will not be able to compensate for human errors or be responsive when some non planned ``magic'' between the actors happens on stage. It tends to force human interpreters to rigidly follow a predefined track and therefore empoverishes the quality of the performance.
In summary we believe that behavior-based media actors are a promising approach to interactive theater for three main reasons:
1. Behavior-based vs script based theater has room for improvisation, both in the case of the improvisational or street theater in general or for classical scripted theater that the director and the actors need to interpret and therefore modify.
2. The system is tolerant to human error and actually encourages actors to enrich or change the performance according to the reaction of the audience.
3. The system can scale from a performace space to an entertainment space. Behavior-based theater can allow for user participation either during or after the performance without requiring the new users to learn all the script in advance.