Commonsense Course Projects Gallery
Flash Visualization for Knowledge in Open Mind
We decided that instead of having the results from OpenMind displayed
Acquiring Common Sense from The Sims
One question that I keep coming back to when trying to figure out what to do with common sense is: What is common sense? It seems like there are a number of different opinions about what should and shouldn't be included in common sense, and what it means for a computer to "use" common sense. If I have a big database of common sense knowledge with a great representation that allows a computer to answer seeminly common sense questions (like the ones mentioned on the website henry sent around), does this constitute the computer having or using common-sense? It gets back to the various motivation of AI researchers, some want computers to be intelligent, others want computers to seem intelligent for our purposes.
One way to think about common sense is shared knowledge or implicit knowledge. This is easy to think of in the context of communication. What goes unsaid in a conversation is implicitly understood to be shared knowledge or common sense. In the physical world, "things fall down not up", is implicitly understood to be shared knowledge given that we all share the same physical world. This idea extends into smaller circles of shared knowledge, where the smallest circle of common sense could be 2 people or a person and a computer. If a group of people were at an event. They can talk about or refer to the event assuming that the others share a common ground.
Extending the idea that common sense is what goes unsaid, it is interesting to think about how to acquire all this implicit knowledge. Even though we can't remember not knowing much of what is considered common sense, at some point we all had to learn these things either by being told by a parent or teacher or just through observation and experience. So, one way a computer might be able to go through this learning process is through simulation or a virtual world like the SIMs. Basically this is just taking advantage of the fact that there is this world that the computer can sense and affect. One thing that might be fun to do is implement a "child" or "butler" avatar that would follow around the user's avatar, trying to learn by observation, learn by asking questions about the current activity/surroundings, and maybe suggesting things to do and trying to anticipate things to do for the user (like a good butler). So basically letting the user and the agent interact in this virtual space turns into an interactive learning session for the agent, allowing the computer to learn in a common sense sort of way. Learning what is common by watching multiple interactions, learning about a new context or situation by recognizing action/environment similarities and dissimilarities and then asking questions.
So, for my class project, I'd like to look into the feasibility of making this learning butler character for the SIMs. I did however have some other thoughts about possible class projects. The following are a few different ideas around using common sense knowledge and reasoning to improve some type of human computer interaction.
1) A context-aware map with common sense about phys places, could a map
with common sense be better at giving directions? Current mapping systems
(mapquest, the talking car maps) are not very user friendly, and this
tends to be because they don't have enough common sense to say "take
a U-turn" instead of "take a left, go .01 miles, take a left,
go .01 miles, and take a left". If it does some inference about "collocation"
and "hasa" to give more common sense directions.
3) Another example of the interactive learning idea mentioned with theSIMs is the 20questions game. The computer asks questions to try and guess the animal/vegetable/mineral the user is thinking of. When it guesses wrong it asks for user input about the difference between the guess and the actual answer, and then later it can use this to ask questions to discriminate between these things. The next level up is for it to get better at asking "good" questions, or most discriminating questions, thus getting to the answer most efficiently. It would be interesting to look at whether or not kids do this when they first learn the game. For example, do they start off just guessing different animal features, but then learn that if you ask "does it have 4 legs" or "does it have fur" you eliminate a lot of choices. Or is it natural to ask more discriminating questions before more particular ones?
4) My laptop gets carried around with me all the time. Could it start to have some common sense about its functionality by understanding something about the people I'm around and places I go. What would it do for me? It could make information about shared experiences more readily available, visible, or applicable to current task. When I'm sitting in class, maybe it puts some "relavent" documents and information on the desktop ready to be used because it has a common sense about this place from the last time I was there.
The Pro-Con Engine
When groups of individuals are planning on doing activities, they make their decisions using a collective understanding of each other's preferences. These preferences have to be verbalized, either through spoken or written communication. While some members of the group may be more attuned to the preferences and particularities of different individual members of the group, often times, the group does not have a complete picture of the constraints and preferences of all members. This lead to either prolonged discussion of pros/cons or a dictator type approach to decision-making which leads to different members of the group having their preferences ignored.
How can common sense help? A data source containing common sense knowledge about each user's schedule, constraints on activities, and preference for different activities, then the data source can be used to articulate the pros and cons of a particular decision. A common sense application which is able to identify the issues relating to a decision can save time for the group (less time required to building a base understanding of the group's preferences) and can assist the group in making decisions which take into account each group member's needs and preferences.
As an example scenario, imagine that I were to ask the Pro/Con Engine, "Should my husband, dad, and I go apple picking this weekend?"
Rather than me having to guess the constraints and preferences of my husband and father, or having to ask each of them for what to them is obvious information, it would be preferable to ask a pro/con engine to enumerate the different pros and cons of the decision.
A possible result coming from the Pro/Con Engine would be:
- this is one of the few times you can do this activity
- dad has to catch a plane at 5pm
- you will need to drive to go apple picking.
The facts held in the database will be both of a worldly and personal nature. Some of the facts relate to time and geography. Others relate to facts about the activities. And a third category of information relates to personal preferences. This area of personal preferences is where I think this application will have the most benefit, because it will highlight the factors that each individual thinks is obvious, yet the collective group is not aware of.
Issues with this project idea:
- does this application propose to do too much?
This project proposes building a recommendation engine based around the
preferences of multiple people. But this differs from previous work in
that it does not make a suggestion of an activity, but rather presents
a first-order analysis of whether or not a particular activity is a good
or bad thing to do. By presenting arguments to the group of individuals,
the group is better equipped to understand the pros and cons and make
a well-informed decision.
Foreign Language Assistant
The application that we are proposing to build for our final project
Currently travelers can only rely on basic phrase books, which are however
The system will have four functionalities, and the user may choose which
1.Word translation: the user enters a word in English. The system uses
2.Phrase translation: the user enters a phrase in English. The system
3.Generating relevant words: the user enters a word in English describing
4.Generating relevant phrases: the user enters a word in English describing
Acquiring Culturally Contextualized Common Sense
Commonsensical Meal Adviser
Our idea is the creation of an application that will specifically deal with offering the optimal solutions given specific or peculiar sets of scenarios and domains.
We will build a certain set of rules for the specific knowledge database and have it query the specific knowledge (common sense) that we will spawn of open mind.
We will create our own set of knowledge that is oriented towards our problem domain.
The specific case here is a MEAL ADVISER. This is an application that uses common sense and comes up with suggestions on whether we can have dinner, what we can have, and how to go about it.
Open Mind Database Input
1. I have 2 gallons of milk
It will mainly deal with what, where, when, who and how issues?
1. Dinner (is equal) food (not equal) pork
1. A calendaring database
We would type INPUT " I am having Carla and Roger for dinner tomorrow
Using Common Sense to Add Pictures to Short Stories
Description of the program:
The program will use common sense knowledge to suggest pictures to be added to a short story.
This approach is similar to the one used in ARIA, however, there are two important differences:
1) Pictures will be drawn from a large picture database in the Internet (Yahoo! Picture Gallery), and no annotations will be used.
2) The program will be able to use common sense knowledge from previous sentences to suggest relevant pictures. This is possible because the program will function in a static manner (i.e., the user will provide the short story beforehand).
An important application of this program will be to add pictures to children stories. For example: In some situations, a child may be biased towards not reading a short story because it has no pictures in it; therefore, if an adult can use the program to add pictures to the story, then the child could be leaned towards reading the story while getting some visual notion of what he is reading. In addition, the child could use the program to select the pictures that match the story that he is developing in his head while he is reading the story.
A short story in the form of a file.
The short story file with suggestions of pictures between sentences, so that the user can select the pictures that he likes. Then, after the user selects the pictures, the program can covert the final version of the story with the pictures to HTML.
With time of using the program, the limitations of the Yahoo! Picture Gallery to capture all situations be apparent. This observation will specialize the types of short stories on which the program will work better, and on which the program will specialize.
|Ask Jon: A Common Sense Profiling
and Recommendation Engine
Tien-Lok J. Lau, Austin Wang
Through the use of the Open Mind Knowledge base, we aim to tackle two major challenges in the field of software agents: user profiling, and content recommendation. Ask Jon starts by profiling each user, and then recommends relevant information to the user based on a topic they specify. The project assumes that members of the Open Mind community have subconsciously embedded information about common stereotypical personalities and tastes into Open Mind; the goal is to mine this meta-information and utilize it in a provocative way.
Open Mind Commonsense is an ambitious project aimed at teaching computers all those things an average person knows but takes for granted. Its approach differs to other commonsense databases by allowing complete freedom in terms of input syntax, and is open to the public via the Internet. However, having a trove of random commonsense knowledge is merely the first step in the quest to develop a smarter computer. Therefore, we are looking to develop a brand new interactive application based on the knowledge embedded in Open Mind that will not only demonstrate the power of Open Mind, but also to test the potential of introducing computers to commonsense.
After some experimentation and analysis of the of information stored in Open Mind, we have come to the conclusion that much of the information considered "common sense" by the general public is actually common stereotypes of various cultures, ethnicity, religions, age groups, and professions. This may be a reflection on common sense in general, or a reflection on the information that people choose to put into Open Mind. Regardless, we believe that this information will be especially useful in the tasks of user profiling and content recommendation. Both of these tasks are not new in any way, and businesses have been working of these problems for a long time now. However, we are suggesting a novel approach to these important tasks by using the common sense knowledge in Open Mind. As the knowledge in Open Mind continues to grow, our profiler and recommendation engine will only grow stronger and more accurate, reflecting the common sense of the public.
Catherine is interested in knowing what type of movie Open Mind would
recommend her based on her profile. She is interested in knowing if the
collective common sense of Open Mind can correctly identify a type of
movie that she would enjoy. Catherine fires up Ask Jon and enters information
about herself. Figure 1 depicts a sample interface that she would see.
She tells the system that she is a twenty-one year old Chinese engineer
who does not have any religious affiliation.
There have been many examples of integrated user profiling and recommendation
systems in the field of software agents. Here are a few of them categorized
by their chosen approach:
Recommendation is carried out by calculating the keyword proximity of
the user's query and their accumulated knowledge about the query with
the relevant information resources. Probability is employed to combine
the knowledge contexts of different users.
Once the user interest space has been populated, any new document can be classified in the same way and drawn onto the interest vector space. If the new document vector is sufficiently close to an existing interest vector cluster, it is recommended to the user.
Ask Jon will inherit design features from many of the past applications above, but will demonstrate something fundamentally new. It will try to mine the implicit knowledge of people's preference within Open Mind and integrate it with the recommendation process.
The performance of the past keyword matching systems has been limited due to the lack of contextual information. Ask Jon hopes to harness the context-rich environment of Open Mind and combine keyword/concept matching and statistical modeling to achieve better performance.
The application shall be an interactive tool that the user can use to gather information about their topics of interest. The tool can also extend beyond personal content recommendation, since it uses very little a priori knowledge. For example, a user can create a profile using someone else's information, and then try to predict things that would appeal to them, like what flowers they would likes, or their favorite TV shows.
Due to the nature of Open Mind, it is important for the user to understand
that the application may return baffling results. Therefore, it is part
of our design goal to make them aware of the application's dynamic and
The content recommendation screen will have current searches and past
searches. The user can rate the past results if they wish. While the application
is processing information from Open Mind during the topic search function,
the information from Open Mind is visualized in a stimulating and intuitive
fashion. The intension of the visualization is two-fold. Primarily, it
is to keep the user interested while he waits for the results. Secondly,
it gives him an idea of how the application generates the results, so
that the results will not come as a complete surprise.
Several user profiles can be stored on the local machine and can be retrieved and loaded as the user wishes. The search results for each user are also saved along with the profile.
Krulwich, B. and C. Burkey (1996)."Learning user information interests
through extraction of
For the final project, we are interested in experimenting further with alternate (i.e., visual) representations of results from the database and how they affect people's positive and negative views of assertions. We would like to expand on the implementation of the first project that resides at http://www.tv28.com/openmind/. One idea we have is to modify our first project and make it a game of some sort, where users can, for example, click the "good" common-sense sentences that come up. A counter can be maintained to keep track of how many clicks a certain piece of knowledge has received (ie., "good" feedback) as well as how many times it might have been clicked but wasn't (i.e., "bad" feedback). After the "good" and "bad" feedback counters reach a certain ratio and minimum count (e.g., 15 bad feedbacks and only 2 good ones), that statement could be removed from the database because it would be considered to have a low quality level. These counters can be implemented as extra fields for each entry in the database an additional field would act as a flag to determine if the assertion should still be used.
Other visual representations (e.g. text-only, html form) for labeling assertions will be determined and implemented for comparison.
We intend to make some sort of incentive for users to play this game, including a high-score list or something similar displayed by the list of people who have entered the most assertions. We feel that the addition of this sort of feedback mechanism will allow users to evaluate items in the database, and eventually less-useful statements will be purged from the database. The end result would be an OpenMind database with less "garbage," (e.g. Catherine Havasi is not a fire-breathing chipmunk) which would then be more useful for other applications.
We will use Flash to implement this project. The database will have to be altered to accommodate the additional fields and also properly deal with deleted assertions. All of the backend and interface scripts will be written in Python.
| 6 Degrees of Separation
Josh Juster & David Gottlieb
Here's an example of how our implementation would work.
Tomorrow I am going to the circus to see the clowns with my friend.
1) get keywords (minus stopwords)
2) link all keywords via their shortest paths
tomorrow-circus: 4 sentences tomorrow-clown: 3 sentences
This means that the database can link these words in x number of sentences.
clown-friend: 2 sentences:
Clowns are funny looking. I have a friend that is really funny.
3) find shortest path that connects all keywords:
first sentence: tomorrow-friend link (1 sentence)
this links all words tomorrow-friend-circus-clown in 4 sentences
result: a paragraph with somewhat relevant information about what you