Time, For a Change
Robert C. Burke
rob@media.mit.edu
13 September, 2000
Introduction
This essay is about time, and how a Society of Mind
might be designed to accommodate, work with, and possibly understand the
passage of time. It is divided
into three sections: The first
considers ways in which our minds remember, and how time affects our memories.
The second describes how the passage of time should moderate the ways a
character learns and develops in its environment.
The third suggests how these ideas might be woven into creatures now
being developed in the Synthetic Characters group.
As well as representing my final report for Dr
Minsky’s The Society of Mind course, it is my hope that I will
continue to pursue this investigation as I research my Master’s thesis this
year.
Places, Subjects, Events, Impressions
Mona asked Dave to describe his visit to Norway
last summer. Dave first recalled his impression that Norway is a
wonderful country. He found the Norwegian people friendly, and their
lifestyle refreshingly balanced. He remembered one friend, Annette, who
on several occasions introduced him to traditional Norwegian foods. He
especially enjoyed eating a reindeer dinner with her one evening in August.
But he couldn’t remember the Norwegian name for the dish. Dave was
sure he knew it, but he just couldn’t remember the name.
Somehow, our Minds have a way of filtering through all
of our experiences and transforming them into impressions.
Not only do we remember events and sequences, but we also form
impressions about the things that are important in our lives.
Our Minds form these impressions on various scales and
levels of detail. In Dave’s
case, he has remembered numerous types of information about his trip.
Some parts, like the details of his reindeer dinner, are very specific.
Other parts of his memory, like his impression of Norwegian people, are
very general. Although Dave
almost certainly didn’t have a chance to meet every person in Norway, one of
his first recollections is a positive impression of Norwegians in general.
Somehow, without conscious thought, Dave’s mind has generalized
specific episodes into an impression.
Trying to describe the structure of a general
impression is a challenge. In
Dave’s rather typical impression, we have at least three kinds of elements:
places like Norway; subjects like Annette; and events like eating reindeer one
August evening. (In Ad
Herennium, the anonymous philosopher divided memory into places, images,
things and words. Here, we
concentrate on the first three. Perhaps
words can serve as labels for the others?)
As time passes, we continue to expand our range of
experiences. But our capacity to
recall episodes, like Dave’s dinner with Annette, is limited. As
Dr Minsky points out, our capacity to store memories is only useful if we can
re-member those memories when they are needed or useful.
Both Dean Piaget and William James agree that a main function of the
memory is to forget useless information. Remembering
dozens of potentially useless facts would necessitate that our minds spend
time sifting through these useless memories.
What do we forget, and what can we remember?
Piaget suggests that we
conserve the memories that are most frequently used.
This indicates that we can’t treat memory like a passive blackboard.
When storing memories, other parts of our Mind can indicate to us which
sensory information seems most salient. Similarly,
over time, the memory can seek advice about which of its contents continue to
be the most important. Our memories will be much more effective if they
collaborate with the rest of our intelligence.
But it isn’t simply an issue of what we can remember
and what we’ve forgotten. Dave
can’t recall the Norwegian name for reindeer.
But he knows that it’s somewhere in his memory; if only he could
recall it! Knowing that you’ve
forgotten something is a strange state of mind.
It suggests that you know that information is stored in your memories,
but you don’t know how to access it. Perhaps
Dave will continue thinking about his experiences, and something else will
trigger the memory. Or maybe
nothing he thinks of will provide a strong enough association to allow him to
recall the dish’s Norwegian name.
So there are several important things that fall out of
Dave’s response. His memories
of Norway include a web of impressions of subjects, events and places.
They also include specific information that he associates with the
events. He also is aware of what
he knows, even if he can’t remember it.
His memories have been filtered by a year in which he has forgotten
many of the details that he might have recalled if asked the same question a
year ago. But his impressions
remain, and will continue to change as Dave stores new experiences and
memories.
Change and Growth
And all the lives we ever
lived and all the lives to be
Are full of trees and changing leaves.
- Virginia Woolf
Dr Minsky describes in The Society of Mind how
children, over time, develop agencies in their minds.
Some, like the Society of More, provide them with concepts like volume.
Although the brain doesn’t “shut down” its ability to learn in a
particular way, studies of children suggest that periods of intense learning
and development occur in stages.
It is certainly no accident that our minds and bodies
develop in stages. Consider how
important it is for young children to have a plastic, flexible understanding
of language. And although as
adults we can lament how frustrating it is to learn a new language, we should
be pleased that our minds no longer work quite the same way.
A mother teaching a child how to pronounce words in her native language
shouldn’t be learning equal amounts of babble in exchange!
Our stages of development assist us in learning the right things at the
right times in our lives.
When designing systems that learn in a variety of ways,
we may be able to ease the burden on the learning mechanism by this temporal
division of labor. We might build
a Society of Mind that starts with some agents very plastic, and others very
rigid. Then, as the Society
develops, we can change which of the agencies are allowed to make substantial
changes to their structure. By
working in stages, we can reduce the number of agencies developing
simultaneously, and thus reduce the risk of conflict between different types
of growth.
It is possible that the order in which developmental
stages occur is important. Suppose
that the onset of puberty was to occur, on average, at a much earlier age in
children. A heightened sense of
sexual awareness would probably hinder other aspects of early childhood
development. A young child is too
busy developing other aspects of its mind and body that are necessary to
accommodate sexuality at some future point.
We may find similarly that a thoughtful ordering of developmental
stages assists in the growth of an artificial Society of Mind.
How and when do our minds adapt the ways in which they
learn? The “B-Brain” is
described in the Society of Mind as the parts of the brain that monitor other
core brain functionality (that is, the A-Brain), and provide feedback to
encourage effective development. This
sort of self-monitoring could be used on a short-term basis, to help the
memory decide what to remember and what to forget.
Over a long timescale, the B-brain could be used to mediate various
stages of development. It could
even be used to turn specific places, subjects and events into impressions in
memory. On many time scales,
having the brain monitor its own development will assist its continued growth.
As a final note, we have good intuition for designing
systems that learn in various ways. Some
kinds of classifiers, for example, adjust their rates of learning over time.
But we aren’t as good at imagining a system developing in a variety
of ways all at once. If we were
to go about designing a complex Society of Mind, developmental stages may not
only make the mind work more like our own, but it may give us a chance to
understand and monitor the process.
Time, For a Change
Starting in January of 2000, the Synthetic Characters
group went about re-designing their behaviour-based artificial intelligence
system to include a variety of learning mechanisms. As of the following summer, the action-selection mechanism
was able to learn in a variety of ways:
The system can generate new
“actiontuples,” or, very loosely, vectors that define how rewarding it is
to perform an action in a given context.
By building new actiontuples with more specific contexts, the creature
is able to form more concise hypotheses about which important aspects of the
context, when followed by the action, produce a reward.
The system uses
back-propagation to determine the value of the actiontuples – that is, how
rewarding it is to activate an actiontuple and thus perform an action in a
given context.
Each of the actions may
themselves be parameterized, so that the creature can learn to modify specific
aspects of an action to make it more rewarding.
The system attempts to learn
the optimal length of the interval over which an action should remain active.
Each of these learning mechanisms was demonstrated to
work (in the context of a dog who could be trained to perform tricks on voice
command). However, the system was
prone to falling into undesirable states from which it couldn’t escape.
Most obviously, the lack of a method for culling actiontuples led to a
bloat in the number of actiontuples in the system.
Eventually, the learning mechanism slowed to a halt when choosing
between dozens of possible context-action pairs.
Moreover, when a creature is not “doing the right thing,” it was
very difficult to determine why the dog behaved a certain way.
The resulting system was too unstable to trust in a
“critical” situation. When
demonstrating the dog, we disabled all of the learning mechanism (with the
exception of the acoustic pattern matching) and showed a pre-trained creature.
If we didn’t do this, we would up with a creature that – even if it
learned the right things – would slow the system to a halt after less than
half an hour. We are encouraged
by the fact that each the mechanisms was proven to work, and that even real
creatures can be trained the wrong things sometimes.
But a system that models a dog should show greater longevity.
There are also a number of tasks that the current
system is incapable of performing that would be of interest to someone
attempting to build a complete Society of Mind.
The creature now represents its knowledge completely by the values of
the various parameters of its actiontuples, and the actiontuples themselves.
(The existence of very specific hypotheses suggests that the creature
suspects that something in its world is important.)
However, because the creature has no sense of history or memory, it
could not indicate to you why it is performing a certain action.
Why does a dog sit when it hears the word “sit”?
The best answer the system could give right now is that, on average,
doing so tends to lead to a better world state. A more interesting answer would be that just a moment ago,
the creature was told to sit, did so, and was given a treat as a reward.
Perhaps the element that binds together these concerns
about the memories of our characters is time. Humans have struggled to understand their own sense of time.
How much could we improve the system if we built our creatures with an
eye on their understanding of its passage?
Suppose we start building the creature with the notion
that it has a lifespan, and that, like a living creature, it will grow, learn,
change, and eventually die. Then
we immediately begin to have new expectations for the creature.
It should behave and learn differently as it grows.
It should have a sense of its past – perhaps in terms of the events,
sequences, and impressions discussed above.
It should have mechanisms, perhaps similar to what Dr. Minsky calls a
“B-Brain” in the Society of Mind, that allow it both to remember and
forget, so that it doesn’t become burdened with an excess of useless and
uninteresting memories.
To implement even part of this tall order will require
a variety of new mechanism. Perhaps
most pressing is a richer knowledge representation.
The representation would have to facilitate action selection, but also
provide for the sorts of associations between actions, events, sequences and
impressions that we have been discussing.
Equally important is a design for a B-Brain that monitors and checks
growth in the Mind. The B-Brain
would have short-term functions, like culling unimportant information from the
Mind. It would also have
long-term functions, like forming impressions, and facilitating temporal
division of labor among the various learning mechanisms, thus allowing the
creature to grow.
We have characters that learn how to use the current
context to “do the right thing.” But
we have never designed a character that can last more than a couple of hours,
or one that, at the end of those hours, could recount what it had just
experienced. Perhaps by putting
learning and development in the context of passing time, we can solve some of
these problems and bring our characters closer to asking, like humans, where
all the time has gone. And at
last, the passage of time will decay a character’s memory as well, causing
him to know there’s a bone buried somewhere in the yard; if only he could
recall where!
References
Here are some of the other sources I have consulted
along with The Society of Mind and rough drafts of The Emotion
Machine while researching this essay.
Anderson, John R., Learning and Memory, 1995.
Blumberg, Bruce M., Old Tricks, New Dogs, 1996.
Piaget, Jean and Inhelder, Barbel, Memory and
Intelligence, 1973.
Schank, Roger C., Dynamic Memory Revisited,
1999.
Schank, Roger C., Dynamic Memory, 1983.
Yates, Frances A., The Art of Memory, 1966.
As well as numerous entries from James McConkey’s The
Anatomy of Memory, 1996.