Traditional digital interactive entertainment has centered around characters for years. Most entertainment products contain computer characters that represent the user, and the user puppeteers the character in its environment. In Petz products, the characters are completely autonomous, the user is represented as a hand cursor, and the pets perceive the user as just another socially intelligent autonomous character. In this way the Petz and the user are on essentially equal footing.

The virtual Petz concept has been a good way for us to explore the complicated issues in creating interactive, real-time autonomous characters. Choosing dogs and cats as our agents helps us in a few ways. People understand what pets are and essentially know how to interact with them. This is critical for producing believable characters. Pets are also a good choice because they are expected to behave in ways that we can successfully implement. They have relatively simple gestures, language and cognitive abilities. Also, animated dogs and cats have well established cartoon archetypes. We base the look and behavior of the Petz on these archetypes.

The goal of the Petz is to build an intimate relationship with user. Therefore the pet's primary motivation is to receive attention and affection. They feed off of this interaction. Without it they become lethargic, depressed, and if ignored long enough, they will run away.

The most direct way the user shows affection is through petting. Using the left mouse button users can pet, scratch and stroke. The pet immediately reacts in a variety of ways depending on what spot on its body is being petted, how fast, and how the pet feels at the time. We call this direct interaction. The fact that you can touch the character is a very effective way of building an intimate relationship. It also goes a long way in creating an illusion of life.

We've tried to make the Petz have equal footing in their relationship with the user. The toys and objects in the pet's environment have direct object-like interaction for both the user and the Petz. Petz have full access to the toy shelf, and if they really want something, they will just go and get it themselves. This helps express the unpredictability and autonomous nature of the Petz. It also requires users to share control of the environment with the Petz. The act of sharing the space is a further strengthens the owner-pet relationship.

By picking up and using a toy, the user can initiate play. For example, throwing a ball may initiate a game of fetch, or holding a tugtoy in front of a pet may initiate a game of tug-of-war. Similarly a pet can get its own toy and bring it to the user to initiate play. This cooperative decision-making helps build the relationship.

Petz communicate with the user in a variety of ways. Emotion can be expressed through different facial expressions (eyebrows, mouth, ears), styles of movement and body language (sad walks, happy trots, various postures, a variety of tail motions), and sounds (excited playful barks and meows, sad whines and whimpers, yelps of pain, etc). They get fat when they eat too much and get skinny when hungry.

Petz communicate their intentions and desires through their actions. We have created a broad base of behavior for the Petz -- they can eat, sleep, play, attack, groom, hiss, explore and so on. When a pet wants attention they may start barking or meowing. If hungry they may go searching for food, hunting mice, or start begging. When they are upset or scared they may run and hide, act aggressively, or cower and shiver.

It is important to note that the only way the user can understand what the Petz are feeling is to interpret their actions and physical cues, in the same way an audience interprets an actor's performance. We do not display bar graphs or text messages describing the pet's internal variables, biorhythms or emotional state. By forcing a natural interpretation of the pet's behavior, we don't break the illusion of a relationship with something alive.

The user can communicate to their pet in a variety of ways. In addition to petting users can gesture to their Petz. By double-clicking in an empty space on the screen the hand cursor animates a "come here" motion and plays a whistling sound. Petz tend to stop what they're doing and come to the user's cursor.

Users can train their Petz to do tricks for treats. When holding a treat, users can make the hand cursor do directional hand motions (up, down, left, right) to which a pet might jump, sit, lie down, rollover, and so on. If you shake the treat over a toy, a pet may attempt a trick using that toy, such as balancing on a ball or fetching.

In general users can reward or discipline any behavior. Methods of rewarding include a giving treats, feeding, petting, or presenting its favorite toy. Discipline is achieved by squirting a pet with a water spray bottle. (This is the SPCA-approved method of discipline.) In this way users can modify a pet’s overall behavior through positive and negative reinforcement.

In our newest product, Petz II, released in October 1997, multiple characters can interact with the user and each other at the same time. When two Petz first meet each other, typically as puppies or kittens, they cautiously investigate each other. As they wrestle and play they form relationships and attitudes towards one another. Some will become best friends and will often groom each other, playfully nip at each other, sleep side by side, bring each other treats, and so on. Other pets may become enemies and will fight, chase and terrorize each other throughout their lives. A dominant dog might steal food and toys from a submissive dog.

Adult Petz tend to nurture and protect younger Petz. They will comfort them if they get scared and carry them to a pillow to sleep. Younger Petz will follow the adults around and attempt to copy their behavior. However once adolescent, a pet will form an adult relationship with the other adults.

Most Dogz and Catz tend to dislike each other. Typically they will growl, bark, hiss, fight and chase each other. However if they are introduced to each another at a young age and grow up together, they may become friends.

The interplay between the variety of personalities gives rise to many dramatic situations. For example, a tired old dog won't be able to go to sleep when a young playful kitten is bouncing around him. The dog may try to take a nap after distracting the kitten with a bowl of food, but the kitten's short attention span defeats this strategy.

Catz are always on the lookout for mice. If the user takes out the cheese and puts it in front of the mouse hole, the mouse may sneak out for a nibble. The cat will immediately begin stalking it for an attack. If the user then brings out a dog, a three-way chase ensues. This usually ends with the dog chasing the cat away and the mouse getting a free meal.

Petz begin as puppies or kittens and age over time. Users witness their pet develop and can affect what kinds of personalities they turn out to have. However at a certain point the Petz stop aging and do not die.

This has been an issue which our design team has fought over back and forth. Some argued that if the pet eventually dies it makes the lifecycle more important. Immortality may devalue the experience by making the user's time with their pet less precious.

On our website we recently conducted an informal poll asking users if they thought virtual Petz should die. 65% of the responses said they should not die. 30% said they would like to be able choose if their Petz should die. Only 5% said they wanted their Petz to eventually die. After long discussions with our marketing department, it was decided that the disadvantages of death outweighed the advantages.

To create these socially intelligent agents on a personal computer, we had to develop new display techniques. Our high-degree of interactivity and lifelikeness requires the use of real-time 3D animation. Pre-rendered animation, however beautiful it may be, cannot be as responsive, dynamic, or as varied as a we require. But to deliver this real-time animation on a PC, drastic optimizations on the display level are required. Our proprietary method of achieving this is unique. We chose to err on the side of frame rate versus a photo-realistic image that moves poorly.

The Petz products use a network of basic 3D body animation which has a high resolution of branch points. For example, from any basic pose such as standing or sitting, the pet can transition to any other pose. As the pet's general body animation moves across this network, additional animation is layered on top. This allows the pet to look in all directions no matter what its body doing. Also we layer breathing, blinking, meowing, head shaking and even character specific posture. In fact the look of the each breed and their ages are layers. The subsequent number of permutations is enormous and therefore passes a critical threshold in human perception. This method greatly enhances our ability to create an illusion of life.

Using this display technology we have developed many behavior mechanisms that operate simultaneously. There are small mechanisms to control blinking and breathing and a variety of cognitive mechanisms to keep track of multiple goals, different plans to execute each goal, multiple emotions towards other characters and objects, attitudes towards other characters and objects, and personality-specific quirks.

In general to achieve a real, unpredictable, non-repetitive quality in these mechanisms, we use constrained randomness. For example, there are rules for each individual pet that determine which toys it most likes to play with, but there is some randomness in that decision. This results in the possibility that a pet might play with any toy, not just the ones it likes the most. A lot of development time is spent tweaking the constraints on the randomness to achieve a good balance between consistent personality and unpredictability. We are focusing on the user's perception of the character and trying not to over-engineer the artificial intelligence.

The Petz products are an experiment that turned out to be relatively successful. The product is equally popular among girls and boys, kids and adults. This is unusual in consumer entertainment software. In the program, users can take a bitmap "photo" of their Petz which they can save to disk. We encourage users to e-mail their photos to us, which we post on our website. We've had a tremendous response, receiving thousands of photos, some with personal stories attached and some with elaborately painted scenes of their Petz on Mars, or dressed up in tuxedos, or windsurfing, or flying World War II planes. We receive many letters from these kids describing their relationships with their individual Petz. Some kids we know get up especially early every day to feed them. Some of the stories describe feelings and complex behavior that we never put in the product. It seems that through combining a direct interaction interface, strong and recognizable personalities, and some fundamental social intelligence, an intimate owner-pet relationship can arise.

We've found that developing socially intelligent agents requires people with interdisciplinary skills from both technology and the arts. Until recently, building lifelike artificially intelligent characters has been treated as a computer science problem. Character development, in any medium, is fundamentally an artistic process which combines elements of drama, storytelling and expressive action. Lifelike computer characters are ultimately perceived through the filter of human interpretation. No computer character can be perceived as truly alive and intelligent unless its builders always keep in mind what the final user will perceive.

We thank Rob Fulop, John Scull, Jonathan Shambroom, Alan Harrington, Ted Barnett, Jeremy Cantor, and the rest of the PF. Magic Petz team.