MIT Media Laboratory
20 Ames Street
Cambridge, MA 02139
(617) 253-0608
mt@media.mit.edu
This unification enables the construction of an interface that achieves elegance, simplicity and power. Allowing graphic objects and internal computational objects to be manipulated through an integrated interface can provide a conceptual scaffolding for novices to enter into programming.
KEYWORDS: programming environments, objects, direct manipulation, visual object-oriented programming, agents, rules
The Vivarium project has generated some earlier software environments: Agar [13] and Playground [4]. In these systems the computational realization of an agent was something of a cross between a rule and a process. These previous efforts concentrated on simulating the environment, the specification of agents for the animate objects, and the interaction between the creatures and their environment. LiveWorld is an effort to improve upon these systems in several ways: first, by providing a flexible and powerful object system and interface, and second by incorporating a more developed notion of a computational agent, along with a theory of how such agents may be created and debugged. This paper deals only with the first of these goals.
The motivating simulation domain for these systems was animal behavior, but the larger goal was always to find fruitful new models for programming in general. Many other systems may be described in terms of objects with autonomous but interacting behaviors; i.e., video games, MUDs, and virtual worlds in general. I've coined the term animate system to refer to this class of environment consisting of multiple reactive objects.
Creating animate systems requires manipulating many different kinds of objects (worlds, actors, agents, and sensors, for instance). These must be accessible, manipulable, extensible, and combinable by the user. Yet we don't want to overwhelm novices with complexity. LiveWorld's solution is to use a novel object system, Framer [6], which permits objects to be viewed and manipulated at varying levels of detail. Complex objects may be treated as black boxes by beginning users, while more advanced users can open them up to see and change the parts inside. To accomplish this, all levels of parts are made out of the same basic structure, the frame.
The design values of LiveWorld include:
* tangibility -objects should be concrete, accessible to the senses, and capable of direct manipulation.
* reactivity - objects should be aware and capable of reacting to each other. The programs that drive objects should be running all the time, so that user interaction can trigger object behavior.
* improvisation - it should be easy to explore new possibilities by making incremental modifications to existing objects.
* learnability - the system should provide "conceptual scaffolding" that allows the learning of new skills to be rooted in existing skills.
Figure 1: A simple world with two graphic objects. Note that theater, stage, cast, objects, and slots are all realized from the basic frame/box.
The graphic counterpart to the frame is the box. From the user's perspective these two constructs are identical, i.e., it is possible for "the user to act as if the representation were the thing itself" [7, p97] for all components of the system. LiveWorld represents Framer's hierarchical structure by means of spatial containment.
Figure 1 illustrates a simple LiveWorld construction. Frame names are indicated in boldface, so x is an annotation of my-turtle, which is in turn an annotation of cast. Frame values follow the name, so the value of the x annotation is 24. Boxes may be open or closed, with this state being controlled by a handle in the shape of a triangle (styled after the hierarchical folder display in the Macintosh Finder [1]). A closed box displays only its name and value, while an open box will display the contained annotations, the prototype (in a recessed box in the lower-right) and a resize handle.
This is quite different from traditional object systems, where classes, slots and objects are very different sorts of entities. This unification permits LiveWorld to have a simple and uniform interface.
In addition to providing a simplification of traditional constructs, frames make it easy to build advanced programming constructs that are difficult or impossible to realize in conventional object systems. These include facets, demons, dependency networks, and histories. It also makes it easy to use frames to represent information about frames. For instance, display information for frame boxes is stored in (normally invisible) frame annotations to each displayed frame (%box-position, for instance). If you display these frames, they get their own display annotations, recur-sively.
It should be noted that LiveWorld and Framer are recursive only in the sense that the basic frame structure is defined recursively, and most functions that deal with it call themselves recursively to work their way up or down the frame hierarchy. It does not imply that a frame can contain itself.
The advantages of prototype-based programming are simplicity and concreteness. It eliminates a whole set of objects (class descriptors) from the environment, and simplifies the specification of inherited properties. In a sense it is specification by example. For example, the properties of elephants might be defined by a prototypical elephant (Clyde) who has color gray and mass heavy. Properties of Clyde become defaults for the spinoffs, so all elephants will be gray and heavy unless their default is overridden, as in the case of pink elephants. With prototypes, the specification of defaults and exceptions are done in exactly the same way, that is, simply by specifying concrete properties of objects.
There are some indications that mental representation of categories makes use of prototypes [9]. Whether this translates into any cognitive advantage for prototype-based computer languages is open to question. What prototypes do accomplish for programming is to smooth out the development process, by easing the transition from experimentation to system-building.
The system provides a graphics library of basic actors. These include shapes and text-actors, pictures, and turtles. The library itself is a theater and new objects are created by using the standard cloning commands, so there is no need for specialized palette objects.
The link between graphic and non-graphic objects may help novices who are experienced users of direct manipulation interfaces make the transition to programming, both by showing them a different view of their actions and by permitting them to interact with computational objects in a familiar way.
Figure 2: simple-world after dropping a sensor into the turtle and opening it. The sensor displays as a translucent range overlay; and recomputes its value as the objects move.
One interesting feature of LiveWorld is immediate redisplay of inherited values. For instance, if a graphic object is cloned, and then the original is resized, both the original and the clone will be updated interactively. This is a powerful illustration of inheritance and may have pedagogical value. On the other hand, it can cause surprises for users who aren't aware that values are being inherited.
By default, cloning is shallow. This means that if a frame with contained frame structure is cloned, only the top-level frame is copied. For most internal frames (i.e. those used as slots) this is adequate, since the slot values in the original will be accessible in the new one. But in some cases, internal objects must be copied. For instance, in Figure 2, if a clone of turtle is made, the slots xpos, ypos, and heading don't need to be copied (and aren't), but sensor must and will be, since the new turtle ought to have eyes of its own. Frames specify for themselves if they need to be copied when their container is copied. The frames that need to be copied are generally those that specify local sub-parts like sensors, or generate side-effects like animas (described below).
After a frame is cloned it must be installed (dropped) somewhere. Frames can have a fits-within slot that specifies what sort of object the frame may be dropped into; the interface will only allow the object to be dropped appropriately (i.e., sensors can only be dropped into actors). Frames can have install methods which allows them to specify side-effects that happen as a result of being dropped.
Framer only offers single inheritance, but because slots are objects in their own right it is often possible to simulate the effects of inheriting from multiple objects. For instance, if we have a turtle (which can inherit only from the prototypical turtle) and we want to add behavior that defines a mass and forces acting upon it, we can copy in the appropriate slots from the prototypical physical object (i.e., mass, gravity-agent). Effectively, the turtle can inherit properties from several objects.
Actions contain code written in Lisp extended with primitives for Framer and for message-passing (see Fig. 4 for an example). Future versions will also support actions programmed in other languages (such as Logo or an English-like scripting language). It's relatively easy to support multiple languages by making language interpretation an object property. In Lisp, messages are sent to objects through the ask primitive. The Ask menu lists the available messages for the selected object and serves both as documentation and interface.
Agents are specialized actions that arrange to be run repeatedly by virtue of containing an anima object (see below). A typical agent consists of a conditional that tests a sensor and conditionally takes an action (by sending a message). A variety of schemes exist to permit agents to control each other; these are beyond the scope of this paper. Further development will allow the specification of networks of agents, handling conflicts between agents, and support for agents with explicit goals.
LiveWorld provides a library of actions and agents. Beginning users can create animated creatures by cloning objects out of this library. The objects also provide a starting point and templates for more advanced users who are ready to start programming. The transitions between these stages (copying, modifying, and finally creating agents from scratch) are relatively smooth, with each stage providing scaffolding for learning the next.
Figure 3: Customizing a frame in order to define buttons.
Click tables inherit like anything else, so this technique may be used to define specialized objects such as buttons (see Figure 3). Here, an action object is specialized to have a different appearance (via the %border-width slot) and a special action to take on an ordinary mouse click (it will send itself a :act message). The button object can then be cloned and the clones given code of their own to define their action, as is done with the thrust button.
The image of LiveWorld is that there are many objects, many agents, and many actions going on at once. Objects are not controlled through a central locus, but determine their own actions. Ideally, every part of the system should be constantly aware, monitoring its conditions and reacting to it.
LiveWorld supports concurrency by enabling objects to be called in the background. The interface to this capability is through objects called animas, which repeatedly send a message to their containing frame. Animas may be turned on and off individually, or en masse. Inheritance may be used to control a group of animas.
Animas illustrate a classic tradeoff in object-oriented design. Animas are add-in objects which enable any containing object to become an agent or process. The alternative would be to make animacy a property of the containing agent itself. The advantage of this second approach is simplicity, since it results in fewer objects and object types. The anima solution, however, has the advantages of modularity in that a specific ability is encapsulated in a specific kind of object, and that object can be included anywhere); and of providing an interface or affordance to the user for adding or removing animacy. LiveWorld's hierarchical object system permits a compromise between these solutions. Since animas can be hidden in higher-level objects and automatically inherited, the user can ignore the fact of their existence at first, and discover them later when ready to "open the hood" of an agent.
Animas are normally running in the background, and it is common for the user to be making changes to the world or to code while other activities are going on. For instance, in the klinokinesis example below, you can move the food patch that the turtle is tracking and watch it adapt. This serves to lower the barriers between running and coding.
In the present implementation, animas are called in a round-robin fashion. This is adequate for most examples, but does not really qualify as true pseudo-parallelism. A facility to support pseudo-parallelism by performing all side-effects in parallel has been constructed and is undergoing evaluation. Framer's recursive annotation facility helps here, since it permits the system to store intermediate values locally.
Figure 4: The actors for the klinokinesis example. The turtle has two agents: one makes it go forward at a constant rate, the other makes it turn a random amount that depends on the food concentration gradient. The value of the last-distance slot is also computed by an internal agent (not shown).
Figure 5: The stage for the cast in figure 4. The turtle started from the point at the lower-left and found its way by "smell" to the food patch at the upper right
In general, there is a tension between simple, powerful, and general constructs and the interface goal of making specific structures for specific tasks. Take object palettes in a traditional program: because they are special objects, they may easily be given special properties such as floating or a special appearance. LiveWorld, though, has been carefully designed so that no special palette objects are necessary! We have achieved simplicity, but at the cost of being able to easily specialize the design of specific elements.
There are many unresolved issues revolving around the deletion and modification of prototypes. Framer by default does not allow the deletion of an object with spinoffs. The interface encourages loose patterns of cloning, but this can cause problems as dependencies are created which make later modification difficult. I have experimented with a modified delete function that permits deleting an object with spinoffs, by copying the slots of the prototype into the spinoffs. This preserves the characteristic of the spinoffs but loses type information.
User testing of the system is planned; some issues to be investigated are the learnability of the object system; whether the availability of all objects in graphic form is empowering or confusing, and whether the particular forms of conceptual scaffolding actually provide greater learnability.
In contrast, LiveWorld boxes are full-fledged objects, and the interface presents them as such. Boxes are dragged and copied with the mouse rather than with keyboard editing commands. Where the interface of Boxer is rooted in text editing (it is in fact based on the Emacs text editor), the interface of LiveWorld is rooted in object-oriented graphics. Both systems extend their root metaphor by allowing for recursive containment.
The advantages of LiveWorld's interface style over Boxer's is in implementing inheritance and a more modern interface (mouse-based rather than text-based) with better integration of graphics. The advantages of Boxer's style are that its textual metaphor makes it natural to impose an order over the items in a box, and it may be more accessible to some users with a textual orientation.
Self has an exploratory graphic interface [2] which, while having some commonality with that of LiveWorld, is exploring essentially different issues. For instance, both systems emphasize animation, but use it in very different ways. The Self interface animates its object representations (akin to LiveWorld's boxes) in a fashion that makes them seem real to the user, but this animation is outside the realm of the user's control. LiveWorld is more concerned with allowing the user to control the animation of graphic objects, leaving the more abstract, structural representation of objects unanimated.
ARK is another environment with goals similar to that of LiveWorld. In this case the commonality is in providing a lively, reactive feel, and in integrating graphic objects with computational objects. Other shared properties include prototypes (ARK's prototypes work more like those of Self), step methods to drive concurrent objects, and a special limbo state (called MetaReality in ARK) for objects in transition.
Both ARK and Rehearsal World, as well as many other visual environments, rely on a separation of user levels. Components are built by sophisticated users, generally outside of the visual environment, while more naive users are allowed to connect these components together graphically but not to open them up or modify them. LiveWorld, on the other hand, tries hard to make every object openable and accessible.
The fact that all objects are presented graphically by default, that is, without the cognitive overhead of having to request their display in an inspector, is a subtle yet powerful change in the way that the programmer relates to objects. In LiveWorld, every object and every slot is an affordance for action as well as a prototype for improvised modifications.
2. Chang, B.-W. and Ungar, D. Animation: From Cartoons to the User Interface. In Proceedings of UIST `93. (November 3-5, Atlanta, Georgia), 1993, pp. 45-56.
3. diSessa, A.A. and Abelson, H. Boxer: A Reconstructible Computational Medium. Communications of the ACM 29, 9, 1986, 859-868.
4. Fenton, J. and Beck, K. Playground: An Object Oriented Simulation System with Agent Rules for Children of All Ages. In Proceedings of OOPSLA `89. 1989, pp. 123-137.
5. Finzer, W. and Gould, L. Programming by Rehearsal. Xerox Palo Alto Research Center Research Report SCL-84-1, 1984.
6. Haase, K., Framer. Unpublished software. 1992.
7. Hutchins, E.L., Hollan, J.D., and Norman, D.A., Direct Manipulation Interfaces, in User Centered System Design, D.A. Norman and S.W. Draper, Editor. 1986, Lawrence Erlbaum Associates: Hillsdale NJ.
8. Kay, A. Vivarium Papers. Unpublished essays, 1990.
9. Lakoff, G. Women, Fire, and Dangerous Things. University of Chicago Press, Chicago, 1987.
10. Lieberman, H. Using Prototypical Objects to Implement Shared Behavior in Object Oriented Systems. In Proceedings of First ACM Conference on Object Oriented Programming Systems, Languages & Application., 1986.
11. Minsky, M. Society of Mind. Simon & Schuster, New York, 1987.
12. Smith, R. Experiences with the Alternate Reality Kit: An Example of the Tension Between Literalism and Magic. In Proceedings of SIGCHI+GI'87: Human Factors in Computing Systems. Toronto), 1987, pp. 61-67.
13. Travers, M., Animal Construction Kits, in Artificial Life: SFI Series in the Sciences of Complexity, C. Langton, Editor. 1988, Addison-Wesley, Reading, Massachusetts, pp. 421-442.
14. Ungar, D. and Smith, R.B. Self: The Power of Simplicity. In Proceedings of OOPSLA `87. (October 4-8. 1987, Orlando, Florida), 1987, pp. 227-241.