I am a game developer and AI researcher at the MIT Media Lab. Prior to enrolling at MIT, I spent a decade developing award-winning AI in the game industry. I recently completed my PhD with the dissertation: Collective Artificial Intelligence: Simulated Role-Playing from Crowdsourced Data Thesis advisor: Deb Roy COMPLETE ABSTRACT | RESEARCH BIO

My goal is to simulate remarkably open-ended, natural, unscripted dialogue and social interaction in virtual environments. To accomplish this, I have developed a platform for capturing and simulating social behavior from recorded human performances. Collective Artificial Intelligence (CAI) is my method, building on techniques from Artificial Intelligence (AI) and crowdsourcing, to automate characters from a massive database of recorded content. Data-driven simulated role-playing has the potential to revolutionize digital entertainment, online education, and consumer engagement. Applications include new experiences in videogames, as well as new classes of training simulations, therapeutic applications, customer service support, tools for recruiting and interviewing, and social robots.

Collective Artificial Intelligence (CAI) simulates behavior and dialogue using data recorded from thousands of human performances, mined for inter-related patterns. The CAI process combines crowdsourcing, pattern discovery, and case-based planning. This process produces characters capable of open-ended, face-to-face interaction and dialogue with humans. Characters can interact with the 3D virtual environment, and converse with humans via typed text or speech.

As a proof of concept, I developed The Restaurant Game, and recorded over 16,000 people role-playing as customers and waitresses. The restaurant is a setting that everyone can understand, yet demonstrates the enormous range of behavior and dialogue in everyday interactions. My thesis evaluates interaction between human subjects and an AI-waitress driven by this data. Videos to the right show examples of the original human-human interaction (top), and interaction with the AI-waitress, via typed text and speech. Scroll down to see a visualization of action sequences observed in 5,000 recorded games. Two additional games have been developed from the same codebase: Improviso records players on the set of a low-budget sci-fi film, and Mars Escape captures human-robot interaction on a space station. Data from Mars Escape has been transfered from the virtual world to power a physical robot.

The Restaurant Game, Improviso, and Mars Escape (from left to right).

Related Publications:            

Understanding speech in interactive narratives with crowdsourced data. (AIIDE 2012).
Crowdsourcing HRI through online multi-player games. (AAAI Fall Symposium 2010).
Behavior compilation for AI in games. (AIIDE 2010).
Semi-automatic task recognition for interactive narratives with EAT & RUN. (INT3 2010).
Learning meanings of words and constructions, grounded in a virtual game. (KONVENS 2010).
Semi-automated dialogue act classification for situated social agents in games. (AAMAS AGS 2010).
Automatic learning and generation of social behavior from collective human gameplay. (AAMAS 2009).
The Restaurant Game: Learning social behavior and language from thousands of players online. (JOGD 3(1)).
Learning Plan Networks in Conversational Video Games. MS Thesis. MIT, 2007.

Related Press:                   

Crowdsourced Online Learning Gives Robots Human Skills - New Scientist (July 26, 2011)
Collective AI: A Conversation with MIT’s Jeff Orkin - h+ Magazine (April 5, 2011)
Play MIT's New Video Game to Help Train Smarter Robots - Popular Science (April 21, 2010)
Singularity - Rocketboom (November 4, 2009)
Video Game Helps Artificial Intelligences Learn to Learn - Popular Science (July 30, 2009)
Games That Design Themselves - h+ Magazine (July 27, 2009)
Bots Get Smart - IEEE Spectrum (December 2008)
Toward More Human Video Game Characters - IEEE Intelligent Systems (July/August 2008)
Decisions, Decisions - The Boston Globe (April 14, 2008)
One-to-One - SEED (The Design Issue, April 2008)
Go Get a (Virtual) Life - NPR Talk of the Nation: Science Friday (August 31, 2007)
Game Designers Test the Limits of Artificial Intelligence - The Boston Globe (June 17, 2007)
Human Players Being Tapped to Enhance Artificial Intelligence - Garage Games (May 1, 2007)
Virtual Dining Provides Tips for Cyber Characters - New Scientist (March 24, 2007)
The Restaurant: a Game Development Conversation - The Guardian (March 8, 2007)

Awards & Recognition:                   

	AIIDE 2012 paper selected as one of the top research papers of 2012 by AIGameDev.com.
	2012 Ernie award for Improviso, as a stand-out independent game.
	Improviso selected as a finalist for the 2011 IndieCade Festival of Independent Games.
	Improviso selected as a Freeware Game Pick by IndieGames.com.
	AAMAS 2009 paper selected as one of the best research papers of the year at GDC 2010 Game Studies Download.
	The Restaurant Game is featured in Expressive Processing: Digital Fictions, Computer Games, and Software Studies (MIT Press).

Below: Visualization of all action sequences observed in 5,000 human-human games.