Extensions

Extending the Model

Structure Engine

The keyword spread-weight mechanism described in the ConTour section underscores the potential and flexibility of a decentralized "story engine." However, it also raises the essential question of how such a mechanism would best be used in a storytelling system.

Simply giving control of individual keyword weights to viewers -- expecting them to directly manipulate them -- makes mechanisms of the story construction highly conspicuous and thus works against providing narrative immersion.

We might instead imagine the author defining the shape of a story by creating a kind of master template specifying the spread-weights of keywords at various times in the viewing experience. In doing this, however, we fundamentally limit narrative response, returning to a pre-determined structure with no capacity for incorporating aspects of a specific experience.

Ultimately, what's needed is a kind of "structure engine" -- a system module to manage aspects of the material selection and shape the experience without reducing the system's responsiveness. For instance, in ConTour, a structure engine could be used to manage the keyword spread-weights.

Structure Engine Sketch

Act I -- EXPLORE

Attempt to expose the viewer to the full breadth of available content. Accumulate knowledge to build a sense of the PROGRAM SCOPE.

Use "across the board" negative spread-weights for breadth. Use a large number of short sequences for a fast pace.

Act II -- DEVELOP

Attempt to expose the viewer to the full depth of content related to the PROGRAM SCOPE.

Prefer progressively longer sequences for decreasing pace.

Apply positive spread-weights to members of the PROGRAM SCOPE.

Apply negative spread-weights to all other keywords, or possibly just to members of a CONFLICT SET, and zero-weight (disable) all other keywords.

Act III -- RESOLVE

Synthesize closure to the experience by building a fast-paced summary of materials related to the PROGRAM SCOPE that were not presented.

In effect, build a "preview" of possible future experiences based on the viewer's history / interests.

scene-builder
The default operation of ConTour (all spread-weights positive) performs this kind of "scene-level" competency.

shot-builder
The "competency" at the shot-level is performed by the author by breaking their materials into discrete units of expression. Additional information might support variations of the material's presentation.

For instance, a video clip might have several available durations as well as having text equivalents (a full text transcript, a brief caption). In this way, the shot-builder might respond not only to temporal constraints, but to other conditions of the presentation (e.g. a low bandwidth internet connection).

In describing the "dual-temporality" of the material presentation effect in ConTour, we see the essence of a simple structure engine already in place. In the short term, ConTour attempts to maintain maximal "descriptive coherency" between individual materials -- providing a kind of "scene-level" competency. At the same time, the system accumulates smaller effects from every material presented -- maintaining in effect a slowly expanding scope of the story.

Extending from this model, we might imagine a structure engine comprising modules, in the style of Maes' Autonomous Agents, organized around a hierarchy of structural competencies: program-builder, act-builder, sequence-builder, scene-builder, and shot-builder.

In the hierarchy, each module exhibits a specific structural competency and only explicitly orchestrates the operation of its immediately subordinate module. For instance, program-builder might produce a program using three instances of act-builder based on the story functions: explore, develop, and resolve.

In addition, each module could be seen as exerting implicit control over every subordinate module by virtue of decreasing orders of magnitude of influence on the selection process. In other words, scene-builder would still operate in a "localized" way to ensure maximal coherency between two materials, but this operation would occur within the larger context and influence of the sequence-builder, in turn under the influence of the act-builder.

In the hierarchy, temporal constraints could "propagate downward." For instance, if the entire program is set to take place over thirty minutes, program-builder might indicate to act-builder that it has ten minutes to construct Act I. In turn, act-builder might use its allotted time to make five two-minute calls to the sequence-builder, and so on.

The final temporal constraint provided to shot-builder might be used to select among temporal variations of a particular shot. These variations might come either from an explicit stream-based representation of possible in and out points, or else might be computationally determined using perhaps computer vision techniques on the video image or automated pause-detection on the audio track.

A given module might exhibit its specialized behavior not continuously but rather only at specific points in the presentation. For instance, sequence-builder might operate by simply setting up specific story contexts for scene-builder to operate in. In this case, to move to a new scene, sequence-builder would temporarily disable the scene-builder, change the story context to "set the stage" for the next scene, then re-enable scene-builder and allow the presentation to continue.

Other Engines

Other useful engines that could be added to the storytelling system include the following: A "lookahead manager" could use the technique of limited lookahead to "scout out" the ramifications of available options before actually taking them. A "learning module" might incorporate the idea of reinforcement learning and allow pathways through the content to become "burned-in." The authoring model for this kind of system would consist of setting up a loose structure and then rehearsing the system until its performances were satisfactory.

Viewer Model

An extremely powerful component not directly addressed in this thesis is an explicit notion of a viewer model. By incorporating a dynamic model of the viewers knowledge over the course of an experience, a storytelling system could utilize concepts like suspense and surprise, in general making decisions based on "what the viewer knows" at a particular point in the narrative -- a key property in narrative genres like the mystery as well as in most documentaries.

Certain aspects of "viewer knowledge" -- for instance, what materials have been presented -- are present implicitly in ConTour. Making such knowledge part of an explicit and independent viewer module, however, enables that model to be stored after a particular experience then reused in future story presentations. For an Evolving Documentary, such a persistence of viewer knowledge would be essential to make the viewer's experience truly repeatable.

Future Forms

Story Scrapbooks & Spreadsheets

In addition to serving as a kind of "narrative engine" and back-end to a storytelling system, ConTour may be seen as the forerunner to a family of tools that add value to collections of video and other media by allowing viewers to play with variations on their presentation. We imagine two such applications, the Storytelling Scrapbook and the Story Spreadsheet.
[1] Amy Bruckman describes a range of such scenarios in "The Electronic Scrapbook: Towards an Intelligent Home-Video Editing System" MIT Masters Thesis. 1991. In the Storytelling Scrapbook, family members store photographs, text, and video snippets documenting the lives of family members. The Storytelling Scrapbook then supports viewing the "family archive" in a variety of ways; presentations might be organized around a specific event, or trace a recurring event or family member though time. [1] In a network environment, such a tool could facilitate the publishing of forms like the "video postcard" -- a self-presenting collection of materials that one sends to friends or distant family members.
As a business desktop application, one could imagine a kind of Story Spreadsheet -- a system that lets the user play "what-ifs" with an annotated media archive. For instance, one could imagine conducting market research by collecting a large amount of "testimonial" footage of consumer response to a range of products. These materials might then be "plotted" as a temporal presentation across a variety of demographic axes.

Scaling the Audience

The explosive growth of the internet and the World Wide Web suggests the storytelling system's potential to support not only scalable story content and telling, but also a scalable audience.

A key challenge becomes how to provide experiences that are responsive and meaningful at a personal level while at the same time are perceived as shared across a "society of audience." Currently in the Random Walk site, viewers may select a special "feedback" button to email the editors additional stories and related web URLs. In addition to this simple input mechanism, one could imagine a fully integrated "discussion" forum where viewers leave comments and communicate to others. Each communication could be indexed and accessed the same way other content in the site is.

In an imagined Dexter-like system, when viewers submit comments, the messages would be automatically tagged by the current story context. Viewers might also have the option to use the Concept Map to specify a new combination of keywords not already present in the database. One could imagine a second "discussion" materials listing appearing parallel to the "authored" listing. When keywords are activated on the Concept Map, both listings would simultaneously respond providing the viewer with a sense of both the topic's coverage in the database and audience response. Keeping the two sets of materials separate enables viewers to decide how much "outside commentary" they want to incorporate into their experience.

Extending from this example, one could imagine a whole new form of story somewhere between the documentary and the internet newsgroup. In the StoryGroup, the "author" plays the role of moderator and editor, constructing and maintaining the conceptual infrastructure of a story's site while allowing outside reaction and contribution. An author might choose at a certain point to relinquish control completely, allowing the story form to grow based purely on its collective and evolving audience.