We have described how agents work but have not yet described how the user specifies and controls them. The most important need for the interface is to allow the user to create and remove top-level tasks. In addition the user needs to be able to exert some control over the agent interpreter, such as cycling manually for debugging or controlling certain parameters that affect the operation of the interpreter.
Top-level tasks are those specified directly by the user. They are used to create the initial population of agents directly subordinate to the top agent. Top-level tasks can be created in a couple of different ways: they can be explicitly specified by the user by using task objects, or they can be automatically generated as a result of user actions such as dragging, which allows the user's intentions to be represented to the agent system as tasks.
Explicit top-level tasks are specified by means of task objects (see Figure 5.10 for some examples). Like other LiveWorld objects, these are created by cloning a prototype. The task is entered as the value of the object. Task objects also incorporate a toggle button, which allows them to be turned on or off, and icons that indicate the task's status. Relative box paths (see 4.5.4.2) may be used in the specification of tasks, which makes it possible to incorporate task objects into other objects which can then be used as prototypes for further objects (that is, the task objects will be cloned along with their container objects, and the new tasks will properly refer to the new object).
In order for DA to be used as an interactive constraint system, the user's actions must somehow be represented within the system. This is done by translating user actions such as dragging, size changing, or setting the value of a slot into goal tasks.
Auto-tasks generally have the form:
(<== (v <slot>) <value>)
That is, they use one-way equality to specify that the slot is to hold a certain value. In general auto-tasks persist only briefly, and are removed as soon as the user takes another action. For example, during a drag, each time the user moves the mouse, two tasks are created, one to hold the x position at a constant value and similarly for the y position. During the next move, these tasks are removed and new ones will take their place.
Lock tasks are similar in form to auto-tasks, in that they specify that a slot or slots are to be held to constant value. Lock tasks, however, are created explicitly by sending a slot a :lock command. This command simply creates the appropriate task and marks the slot with an icon.
Agent activity may be monitored through an agent display that shows the status of every active agent in the system. The graph uses LiveWorld's hierarchical box display to convey the hierarchical structure of the agent tree. This also allows the user to control the level of detail by closing uninteresting boxes. Each agent shows its state by means of anthropomorphic icons (see 5.2.2.2). When active, the agent display is dynamically updated as agents change state.
Figure 5.12: The agent display.
The agent system is designed to run concurrently with user interaction. There are several ways to do this, the best method being dependent upon on the type of problem being modeled. One technique is to have the agent system driven by an anima. This is best for domains in which the modeled system is basically independent of the user, such as an animal behavior simulation. For constraint domains, where objects do not move except in response to the user, a smoother interface is obtained by triggering the agent interpreter explicitly after each user action.
A number of user-settable switches are available to control the details of agent activation. These switches appear in the agent control panel (see Figure 5.13).
The remaining switches in the control panel control how much information is displayed but do not affect agent operation. show-auto-tasks?, for instance, control whether or not auto-tasks are turned into visible task objects. These switches exist only for the sake of allowing the user to save on both time and the consumption of screen real estate.
Figure 5.13: The agent control panel.
The control panel also contains methods that allow the user to manually initiate agent cycles, an anima-based agent driver (run-tasks), the prototype for task objects, and other agent-related miscellany.
The operation of a system of agents is a complex affair, with possibly hundreds of agents acting and interacting. It is a challenge to present this activity to the user in a comprehensible manner. Similar problems are present in any programming system, of course, but the concurrency and distributed control model of DA makes following a sequence of action even more difficult than it might be otherwise. The anthropomorphic metaphor underlying agents suggests that narrative forms might be useful for the purpose of conveying the underlying structure of activity (see section 3.4.3). Users of Dynamic Agents can generate storyboards, or comic-strip-like graphic representations, as one method of visualizing agent activity.
Storyboards have an advantage over other methods for representing temporal processes (such as animation) in that a storyboard maps time into space, so that a number of different moments of time may be seen together and compared visually. The comic-strip form, defined as "juxtaposed images in deliberate sequence" (McCloud 1993), has a rich vocabulary for expressing both temporal and spatial relations, at a variety of scales. Comics are also a narrative form, meaning that they lend themselves to expressing the "vicissitudes of intention" in Bruner's phrase. In plainer terms, stories deal with characters with goals, their attempts to realize those goals, and their success or failure--a format which nicely supports the activity of agent systems.
The storyboards automatically generated by DA only scratch the surface of the representational and narrative language available. A storyboard is composed of a sequence of panels, each of which represents some segment of time in the evolution of the agent system. Panels can contain a variety of relevant types of information, such as graphic snapshots of the changing world that the agents are manipulating, textual description of the action (modeled after comic-strip captions), or other representations of activity.
In general, there is too much going on in an agent system to put every agent and every action in a storyboard. Thus the problem of storyboard generation requires selectivity in terms of the information displayed, as well as the ability to translate events into suitable graphic representations. The system must select which moments to translate into panels, and what details of those moments are significant enough to take up space in the storyboard.
The strategy used to generate DA storyboards is to emphasize change and the agents responsible for it. Each panel of a storyboard represents a moment in which the world changes, and the panel includes both the results of that change (in the form of a snapshot of the relevant parts of the world) and information about the agents responsible for the changes. There are too many agents involved in a typical step to display them all, so further selectivity is called for. The generator chooses agents from the top and bottom of the agent hierarchy that reflect the why and how behind the actions.
This selectivity is in contrast with the more direct, lower-level representations used in the full agent display (see 5.3.3). These take different approaches to illustrating the same thing, namely agent activity. The agent graph display is intended to be a direct, transparent, and literal picture of the agents, while the storyboard is intended to tell a story about agent activity. Thus the displays are quite different, albeit with certain common features.
Figure 5.14: A storyboard. |
Figure 5.14 is an example of a fairly simple storyboard, illustrating a sequence of action from the graphic temperature conversion problem (see section 5.2.5.1). Each panel of a storyboard represents a major cycle, a reasonable choice given that a major cycle corresponds to a unit of world time. However, storyboard panel time does not include a panel for every major cycle of the agent interpreter. Only major cycles that actually result in change to the world are included, or in other words, major cycles that include no successful executions of worker agents are excluded. This is to conserve space, to keep the narrative interesting (by omitting panels where nothing visible happens) and to limit the display to the most salient information. For example, the sample storyboard shown has 6 panels, but would have 14 if all major cycles were included. The omitted major cycles include agent activity, but in these cycles all agents fail and so take no action in the world. This policy has interesting narrative consequences. As it stands, each panel of the storyboard represents a moment of change in the world. If the excluded steps were included, the story being told would be more "internal", reflecting not just the actions of the agents, but their internal starts, pauses, proposals, and failures. This level of detail might be of value under certain circumstances, but for the present purposes, the presentations of such internal detail will be minimized (as implemented, the storyboard generator has user-settable switches that control the level of detail presented). Each panel includes a snapshot of the changed parts of the world, which may include both changes to graphic actors or changes to the values of other boxes. Since the picture is not enough to convey the reasons and sources of change, panels must also include some information about the agents responsible for the action in the panel. Listing all agents involved in a single cycle would be too much detail. The system selects the agents that best convey the how and the why behind the action of the panel. In practice, this means including the worker agents that actually performed the action (the how) and the top-level agents that the workers are working for, which convey the purpose (the why) of the action. The effect is to include both the ultimate goal or purpose of the action and the concrete physical events that were taken to realize it, while omitting the intermediate stages of computation. |
Also included are any top-level agents that have experienced a change in their state as a result of the action of the major cycle, even if they themselves have not taken action. In practice this means agents that were satisfied but have become unsatisfied due to the action of another agent (that is, there has been a goal conflict; see section 5.2.6.2). The narrative rationale for including these is to illustrate the further effects of the panel's action, beyond the immediate physical effects. Generally such agents will become the active agents of the next panel, so this is also a device for linking the panels of the storyboard together.
Panels illustrate changes in agent state by using a variant of the usual agent icon notation. Each agent that is shown in a panel is shown with a pair of icon sets, separated with an arrow, illustrating the state of the agent before and after the action of the panel. Typically this information shows that some agents went from being satisfied to unsatisfied, or the reverse.
Figure 5.15: The first panel of the storyboard.
There are six panels in this example storyboard, which is a recording of activity in the constraint system shown in Figure 5.10. The action begins just after the user has made a change (by moving the position of the first bar) but before the agents have had a chance to respond to the change. This state is shown in panel-1. The "changes" shown reflect the first activation of the various top-level task agents involved in this system. All the agents except a7 have managed to satisfy themselves. a7's task is to keep the rectangle slots of bar consistent. Since bar/xpos has changed, a7 is no longer consistent.
Note that a4 is satisfied and has succeeded in panel 1, indicating not only that the value of bar/bottom is set at 200 (at the level of the baseline) but also that, since a4 has run, it will be attempting to hold the slot at that value throughout the computation. This has consequences for the next panel.
Figure 5.16: Panel 2.
In the next panel, a7 is activated and succeeds in becoming satisfied. By opening up the box representing a7, we can see that it was the worker agent a29 that brought this about by altering the ysiz slot of bar. This omits much of the internal agent activity involved in accomplishing the end state, such as an attempt to satisfy a7 by setting bar/bottom instead of bar/ysiz. This attempt failed because it came into conflict with the previously successful (and more determined) agent a4.
a29's action has satisfied a7 but has also caused a6, who relates the vertical size of bar to the value of the Fahrenheit slot, to become unsatisfied.
Figure 5.17: Panel 3.
In panel-3 a similar pattern is repeated. a6 satisfied itself by changing the value of Fahrenheit, causing a11, which relates the values of Fahrenheit and centigrade, to become unsatisfied.
Since Fahrenheit is not a graphic slot, the system cannot illustrate the action of this panel with a snapshot of the actors involved. Instead, it selects the appropriate boxes and their surrounding context.
Figure 5.18: The final panel of the storyboard.
This pattern of propagating goal conflicts and resolutions continues in successive panels. In panel-4, a11 sets centigrade to its correct value. In panel-5, the wave of change propagates back into the graphic part of the world, causing a change in bar-1's vertical size. Finally, in panel-6, bar-1's vertical position is adjusted without violating any further constraints, so the agents finally become quiescent, and the story is complete.
