|
Flavia Sparacino Museum of the Future Media in Performance MIT Media Lab flavia@media.mit.edu |
Projects |
|
|
Flavia Sparacino Museum of the Future Media in Performance MIT Media Lab flavia@media.mit.edu |
Projects |
The museum wearable is a real time storytelling device: it is a museum
guide which in real time evaluates the visitor's preferences by
observing his/her path and length of stops along the museum's exhibit
space, and selects content from a large database of available movie
clips, audio, and animations. The museum wearable targets individual
visitors with special learning needs or curiosity, and offers a new
type of entertaining and informative museum experience, more similar
to immersive cinema than to the traditional museum experience. The
museum wearable identifies three visitor types: busy,
greedy, and selective, which have been selected as
the essential museum visitor types from the museum literature. It uses
a custom-made infrared location sensor to gather tracking information
about the visitor's path in the museum's gallery and uses this
information to introduce evidence in the dynamic bayesian network
which interprets the sensor information and delivers content to the
visitor. The network performs probabilistic reasoning under
uncertainty in real time to identify the visitor's type. It then
delivers an audiovisual narration to the visitor as a function of the
estimated type, interactively in time and space. The model has been
tested and validated on observed visitor tracking data using the EM
algorithm. Estimation of visitor preferences using additional sensors
is provided in a simulated environment.
|
Keywords
|
|
City of News is an immersive 3D web browser City of News is a
dynamically growing urban landscape of information. It is an
immersive, interactive, web browser that takes advantage of people's
strength remembering the surrounding three-dimensional spatial
layout. Starting from a chosen "home page", where home is associated
with a physical space, our browser fetches and displays URLs so as to
form skyscrapers and alleys of text and images through which the user
can "fly". The City is organized in urban quarters (districts) that
provide territorial regrouping of urban activities.City of News can be experienced either as a desktop 3d web browser or as a gesture-driven information space (immersive cinema) A computer vision stereo based interface is used to navigate inside the 3-D Internet city, using body gestures. A wide-baseline stereo pair of cameras is used to obtain 3-D body models of the user’s hands and head in a small desk -area environment. The interface feeds this information to an HMM gesture classifier to reliably recognize the user’s browsing commands. This work is based on our previous prototype, called the Hyperplex, in which people navigate a media-scape using pointing gestures.
|
Keywords
|
|
The presentation table is an interactive display table which narrates
a story guided by the position of the objects on it. The featured
example is Unbuilt Ruins, an exhibit space which shows a variety of
architectural designs by the influential XXth century american
architect Louis Kahn. This exhibition interactively features
radiosity-based, hyper-realistic computer graphics renderings of 8
unbuilt masterworks by Louis Kahn (side screens). It is housed in a
large room, and contains in the center a square table above which are
mounted a camera and a projector pointing downwards. By placing the
foor-plan selector-object in the center table, and moving a
cursor-object around the displayed floor plan the public can explore
hundreds of images which otherwise would have been impossible to house
in the limited space of a single museum.
|
Keywords
|
|
The Immersive Cinema is a large scale installation which uses two projection
surfaces: one vertical and one horizontal. The horizontal surface is a
large map projected on the floor. People physically walk onto
different locations of the floor map and trigger consequently the
front surface projection to show the correspondent visual and auditory
information. One can see the floor map projection like a mouse pad,
and the person walking onto the map like the mouse driving the
system. A computer controlled infrared camera detects people's
presence, location, and gestures on the floor map.The immersive cinema setup was used in a variety of installations.
|
Keywords
|
|
DanceSpace is an interactive performance space where both professional
and non-professional performers can generate music and graphics
through their body movements. As the dancer enters the space, a number
of virtual musical instruments are invisibly attached to their
body. The dancer then uses their body movements to magically generate
an improvisational theme above a background track. Simulatenously the
performer can paint with their body on a large projection screen,
thereby generating graphics shaped by their movement and the music
they produce. Dance Space uses unencumebring sensing: the performer
does not need to wear any special suits or markers. Motion tracking is
uniquely based on image processing from a camera pointed at the human
performer.
|
Keywords
|
|
Wearable computing provides a means to transform the architecture and
the space surrounding us into a memory device and a storytelling
agent. We assembled a wearable computer specifically aimed to the
purpose of mapping architecture into an experience comparable to that
of watching a movie from inside the movie set, or being immersed in an
information city whose constructions are made of words, pictures, and
living bits. The wearable is outfitted with a private eye which shows
video and graphics superimposed on the user’s real-surround view. It
uses real-time computer vision techniques for location finding and
object recognition. We describe two applications. Wearable City is the
mobile version of a 3D WWW browser we created called "City of News."
It grows an urban-like information landscape by fetching information
from the web, and facilitates the recollection of information by
creating associations of information with geography. Wearable Cinema
extends the previous system to generate an interactive audio-visual
narration driven by the physical path of the wearer in a museum space.
|
Keywords
|
|
Responsive Portraits challenge the notion of static photographic
portraiture as the unique, ideal visual representation of its subject.
A responsive portrait consists of a multiplicity of views -- digital
photographs and holographic 3D images, accompanied by sounds and
recorded voices -- whose dynamic presentation results from the
interaction between the viewer and the image. The viewer's proximity
to the image, head and upper body movements elicit dynamic responses
from the portrait, driven by the portrait's own set of autonomous
behaviors. This type of interaction reproduces an encounter between
two people: the viewer and the character portrayed. The sensing
technology that we used is a computer vision system which tracks the
viewer's head movements and facial expressions as she interacts with
the digital portrait; therefore, the whole notion of "who is watching
who" is reversed: the object becomes the subject, the subject is
observed.
|
Keywords
|
|
Progress in wireless and untethered body tracking techniques offers
today a variety of expressive opportunities for performers. We have
developed an interactive stage for a single performer using a wireless
real-time body tracker based on computer vision techniques. In
Improvisational Theater Space we create a situation in which the human
actor can be seen interacting with his own thoughts in the form of
animated expressive text, images, and movie clips projected on
stage. Such Media Actors are just like human actor able to understand
and synchronize their performance to the other actors' movements,
words, tone of voice, and gestures. Our work augments the expressive
range of possibilities for performers and stretches the grammar of the
traditional arts rather than suggesting ways and contexts to replace
the embodied performer with a virtual one.
|
Keywords
|
|
Virtual Studio is a 3D set inside which people can meet and interact
among themselves or with other 3D characters. As in a magic mirror,
advanced computer vision techniques allow participants to see their
full body video image composited in 3D space - without the need for a
blue screen background . Hence such setup can be used for
collaborative storytelling, visual communication from remote
locations, or game playing. The participant's image is subjected to
all graphics transformations that can apply to graphical objects,
including scaling. According to the participant's position in the
space, his/her image occludes or isoccluded by virtual objects in
respect to the 3D perspective of the virtual scene. Multiple people
can connect from remote locations, therefore turning the magic mirror
into a magic space
|
Keywords
|
|