6.868 Final Project, Seum-Lim Gan and Jeff Norris
The human mind is quite adept at accomplishing many simultaneous tasks. A person is quite capable of speaking, running, bouncing a ball, and maneuvering around his opponents while playing basketball, all while maintaining a steady heartbeat and breathing rhythm. However, isn't it fascinating that when you are reading, it is nearly impossible to also have a verbal conversation with someone, or even to understand what someone is saying on the radio? Have you ever noticed that when you begin to daydream in a boring lecture, you seem to stop seeing what is in front of you in favor of whatever you are daydreaming about? Often, when you return from your daydreaming state, you are surprised to find that the scene in front of you has changed!
It appears that our senses require some of the same agencies in order to inform us about the world around us as are required to help us remember what the world of yesterday looked/smelled/tasted/felt/sounded like. These examples of conflicting activities can be very helpful in understanding the mind because they give us a clue about what actions of the mind are treated in a similar fashion. Perhaps, by inspecting the activities which we are unable to accomplish simultaneously, we may be able to identify operations which require exclusive access to a shared agency.
In particular, we will analyze two examples of "conflicting" activities. First, we examine the fact that the process of imagination (visual and auditory) seems to temporarily override our senses in these areas. We will build a plausible technical model and mechanism that would cause this conflict, drawing from the anatomy of the brain and Minksy's theories on K-Lines and cross-exclusion. Second, we analyze the fact that it is very difficult to read and understand another set of spoken words simultaneously. For this example, we expand our previous model to include a more detailed description of the organization of certain types of agents critical to conflicting tasks. Again we will refer heavily to the anatomy of the brain and Minsky's discussion of the Language Agency.
All of us have daydreamed in a boring lecture, or "spaced-out" while watching a boring film. Daydreaming, or imagination, gives us the ability to simulate scenarios in our mind, putting all of the appropriate agents to work as if they were actually operating in the scenario we are imagining. In addition to specifying new responsibilities for Minsky's K-Line agents, the mechanism we propose to explain this substitution of memorized data includes a new type of agent and a division of the rest of the mind's agents into two groups. After explaining these additions, we will give a description of our model, an example of correct operation, and situations where it fails.
Our model includes a new type of agent, a Sensory Gate, for each of the classical human senses (sight, smell, hearing, touch, and taste). All sensory data is sent directly to the correct Sensory Gate, where it is then typically directed to the agents responsible for processing the data. However, each Sensory Gate has another input, from the agents responsible for forming and maintaining K-Lines (as discussed in the Society of Mind), giving the mind the ability to substitute memorized sensory information for real sensory information. A Sensory Gate can be thought of as a traffic light, stopping traffic from the senses and allowing memorized sense data to turn onto the "road" (see figure). Note that this allows memories to be "triggered" directly by sensory data. When the sensory input is sufficiently strong (a bright flash or a sudden boom), imagination is cut off and overridden by sensory input (preventing us from daydreaming through an earthquake).
The ability to simulate a scenario in your mind wouldn't be very valuable if you were always tricking yourself into believing that what you were imagining was real (although, as we will describe later, failures of our model produce exactly this result!). There must be a division between the agents which think about what to do and the agents that actually do it. In our model, we call these processing and action agents, respectively. Typical processing agents would include BUILD-TOWER and MAKE-SANDWICH, and typical action agents would include GRASP-BLOCK and DROP-PICKLE. This division between planning and acting would allow us to activate only the processing agents for a task when we were imagining something, preventing us from waving our hands around when we were thinking about a dance.
To accomplish this division, we define a system of agents called an PA-Barrier which exists between the processing and action agents. All communications between the processing agents and action agents must go through the PA-Barrier. It should be noted that this barrier is one-way, from processing agents to action agents, as it would be ridiculous for a low-level action agent like BLINK to call upon the services of WRITE-SOM-ESSAY. When the K-Line agents take control of a Sensory Gate, as mentioned above, we assert the data they substitute is much "weaker" than data coming directly from the senses. This weaker signal still drives the processing agents as desired, but normally lacks the energy to leap the PA-Barrier. Indeed, memories usually DO seem weaker than real-time experiences, which helps us to discern between reality and imagination. The failures of this system, described later, include situations when one of these weak signals does make the jump from processing to action agents.
There is great support for the existence of this sort of division between processing and action agents in the anatomy of the brain. It has been shown that the lobes of the Cerebral Cortex are exclusively responsible for higher level operations like vision, problem solving, behavior, and so forth, while motor coordination and learned physical actions like balance and posture exist entirely in the Cerebellum. We will discuss these physical divisions in greater detail in section 3. The following figure summarizes our model:
While you are sitting in the most boring lecture known to man, your mind wanders and you begin to think about all of the things you have to do that day. You remember that you had planned to rearrange your bedroom to fit a new stereo system, so you begin imagining what your room would look like with the furniture in different locations. At this point, your K-Line agents have switched off your visual sense and are helping you visualize the shape and size of your room, as well as helping you remember configurations you have considered in the past. This data is being fed into your processing agents, which consider each possibility and come up with better solutions, which are committed to memory. One of these solutions involves you lifting your desk by yourself, and you imagine one possible way to lift it. When the K-Lines feed this information to the processing agents, they act upon it as if you were actually standing in front of your desk preparing to grasp it, but they aren't able to send this weak, memory driven signal over the PA-Barrier to the action agents. This is very fortunate, because the person in front of you would be very confused when you hoisted his desk, with him in it, into the air! At this point, the lecturer trips and falls, which is a significantly interesting visual spectacle to snap you out of your daydreaming and back to reading the scribbles on the chalkboard.
Sometimes, if a person is thinking about something that makes him extremely angry, he will actually clench his fists and maybe even shake a bit. It's clear that these actions aren't particularly useful unless the thing that makes him extremely angry is within striking distance, but nonetheless, his mind has been slightly tricked into exciting itself when it was unnecessary. In our model, this occurs when a memory driven signal is strong enough to appear to be directly from the senses, and it makes the jump across the PA-Barrier to the action agents. Another such example is a person driving a borrowed automatic shift car who repeatedly reaches for the manual gearshift that isn't there.
Our model assumes that all imagination activities go through the K-Line agents. This may be true, but it definitely is a limitation of our model. Are we capable of imagining something that doesn't depend upon any of our memories? This a very subjective question, and one that is not addressed in our model.
It can be very disconcerting when someone tries to talk to you while you are reading something. It's as if reading and listening to your friend are competing for the use of one agency in your mind. Minsky touches on this bottleneck in his diagram in section 19.2 of The Soceity of Mind. We expand our model in the previous section and consult further sources on the anatomy and function of the brain to form a model that describes this conflict among processing agents. In addition, we report on the results of a simple experiment we performed to test part of our theory.
The four lobes of the Cerebral Cortex, which is responsible for nearly all higher thought, each have specialized purposes (see figure). Therefore, it isn't unreasonable to group agents with these specialized purposes together as well. It is interesting to note that all of the examples we have mentioned of conflicting activities all rely on processes which lie in the same lobe. Of course, grouping similar agents together means that the pathways to these groups of agents may become saturated if many different agents attempt to use the same agency. Our language example focuses on the functions present in the Temporal Lobe. We believe that the language agency is an example of an agency which is often used by many different agents. This wouldn't be surprising, considering the complex tasks that the language agency is responsible for; the brain may simply not have the room for more than one interface to this intricate agency.
If only one agent can be allowed to use the language agency at a time, there must be some mechanism to prevent multiple agents from gaining access to the language agency simultaneously. In our model, we call this agent the Traffic Cop. The Traffic Cop sits between all of the agents in a particular area (not necessarily the entire Temporal Lobe, there may be many traffic cops in a lobe) and acts as a buffer between often used agencies and the agents on the outside. So, when you are reading and someone says something to you, the Traffic Cop blocks the signal, perhaps re-routing it to agents which become active when someone is talking nearby. As in the previous model's Sensory Gates, the Traffic Cop at the Temporal Lobe could be overwhelmed if someone says your name nearby, generating a powerful enough signal to distract your language agency from reading.
It seems obvious that it is more difficult to read something while understanding what someone is saying to you, but how much more difficult is it? To accurately determine the answer to this question would require an extensive set of experiments. However, we decided to perform a simple experiment to give us a rough estimate. The Recall Experiment works as follows.
1. Beforehand, the experimenter prepares 4 lists of 10 unrelated words, each word consisting of seven or less letters.
2. The test subject is shown each word in the first list, one at a time and one per second, using a small moving window so only one word is visible at any time. He is then asked to record as many words as he can remember.
3. The test subject is then dictated each word in the second list, one per second. He is then asked to record as many words as he can remember.
4. Step 2 and 3 are now performed simultaneously, using the third and fourth lists. The subject is again asked to record as many words as he can remember.
After several iterations on two subjects, we found the results which are summarized in graph 1. Clearly, we would need many more test subjects and more iterations to rigorously prove our point, but it can be seen in this limited sample set nonetheless that there is a marked decline in the number of words remembered when the subject is shown words and dictated to simultaneously, even though there are twice as many words presented to him! The internal conflict between these two actions has made the subject much less effective at recalling words than before. It's somewhat surprising, but the subject would have performed better if he had closed his eyes or plugged his ears during the last part of the experiment!
There are some functions which exists in the same lobe of the brain which do not seem to conflict. For instance, the Temporal Lobe is also responsible for musical analysis and appreciation, and many people have little difficulty listening to music while carrying on a conversation or reading. Of course, this could be because the agencies for music are not included in the same Traffic Cop group as the agencies for language, allowing them to operate somewhat independently.
We have analyzed two examples of conflicting activities of the mind. Since the brain rarely seems to evolve in pointless directions, perhaps it is worthwhile to ask if there are important reasons for certain actions to conflict. The ability to imagine a future reality while retaining knowledge of what is real and imagined is an amazingly powerful skill. If we really are using the same agents for imagined realities as we are for actual realities, as it seems, then the act of imagination must interrupt the act of sensing, or confusion would result. Similarly, perhaps our systems for memorizing language are serialized, and therefore need sentences to be contiguous and uninterrupted. Hence, our mind could have evolved Traffic Cops to provide this guarantee. Regardless, there is clearly much to learn from not only studying all the things that the mind can do, but also everything that it can't.
The Brain Tumor Soceity. Anatomy of the Functional Areas of the Brain. 1997.
Marymount College Psychology Department. The Human Brain. 1996.
Baumhaker, Dr. Yossi. The Mystery of the Human Brain. Tel-Aviv University. 1995.
Johnson, Keith A., M.D. et al. The Whole Brain Atlas. Harvard University. 1995.
Minsky, Marvin Lee. The Society of Mind. 1985.