Irfan A. Essa

Presently Assistant Professor at Georgia Institute of Technology, College of Computing . For more up-to-date information, please go to my real home page: http://www.cc.gatech.edu/fac/irfan

Vismod Research Staff

Research interests

Machine and Human Vision, Multimedia, Human Movement, HCI, AI, Facial and Gestural Perception, Computer Graphics, Mathematical Modeling, Numerical Analysis, Cognitive Sciences

Education

Ph. D. Media Arts and Sciences
Massachusetts Institute of Technology, Cambridge, MA, 1994
Ph. D. Advisor: Sandy Pentland

S. M. Engineering
Massachusetts Institute of Technology, Cambridge, MA, 1990

B.S. Engineering
Illinois Institute of Technology, Chicago, ILL, 1988

Experience

Research Scientist
MIT Media Laboratory, Vision & Modeling Group, Cambridge, MA, 1994-present

Research Assistant
MIT Media Laboratory, Vision & Modeling Group, Cambridge, MA, 1988-1994

Research Scientist/Intern
Thinking Machines Coorporation, Cambridge, MA, 1990

Research Description (as of Jan 1995)

I am developing computer vision systems that use measurements from video for detailed facial modeling and tracking of facial expressions. Accurate facial modeling involves fine details of geometry and muscle coarticulation. By coupling pixel-by-pixel measurements of surface motion to a physically-based face model and a muscle control model, we have been able to obtain detailed records of both the displacement of each point on the facial surface and the muscle control required to produce the observed facial motion.

Tracking facial expressions, in contrast, typically involves fast, accurate characterization of a relatively small set of predetermined facial expressions. By coupling sparse, fast visual measurements with our physically-based model via an interpolation process (based on radial basis functions), we have produced a system capable of real-time facial expression tracking. This work has applications in teleconfrencing, human-machine interaction, surgical planning and computer animation. ( more information )

Selected Publications

Coding, Analysis, Interpretation and Recognition of Facial Expressions

Irfan A. Essa and Alex P. Pentland
Submitted: IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), April 1995. Also available as Vision & Modeling TR # 325

Animation of Facial Expression using Input from Video., To Appear,

Irfan A. Essa, and Alex P. Pentland. ``Modeling and Interactive To Appear in Proceedings of Computer Animation 1996 Conference, Geneva, Switzerland, June 1996.

Motion Regularization for Model-based Head Tracking.

Sumit Basu, Irfan Essa and Alex Pentland. In Review (Jan. '95), Proceedings of the International Conference on Pattern Recogntion 1996, Vienna, Austria, August 1995. Also available as Vision & Modeling TR # 362

Task Specific Gesture Modeling using Interpolated View

Trevor Darrell, Irfan A. Essa and Alex P. Pentland
Submitted: IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), April 1995. Also available as Vision & Modeling TR # 364

Facial Expression Recognition using a Dynamic Model and Motion Energy

Irfan A. Essa and Alex P. Pentland
In Proceedings of the International Conference on Computer Vision (ICCV 95), Cambrdige, Massachusetts, June 20-23 1995. Also available as Vision & Modeling TR # 307.

Casual Analysis for Visual Gesture Understanding

Matthew Brand and Irfan A. Essa
In Proceedings of the AAAI Fall Symposium on Computational Models Integrating Language and Vision), Cambridge, Massachusetts, November 1995. Also available as Vision & Modeling TR # 327

Facial Expression Recognition using Visually Extracted Facial Action Parameters

Irfan A. Essa and Alex P. Pentland
In Proceedings of the International Workshop on Automatic Face and Gesture Recognition 1995, Zurich, Switzerland, June 26-28 1995.

Looking at People: Extracting Human Movements

I. Essa, T. Darrell, A. Azarbeyajani, S. Scalroff and Alex P. Pentland
In Proceedings of International Workshop on Computer Vision and Parallel Processing 1995, Islamabad, Pakistan, January 2-5 1995.

Analysis, Interpretation, and Synthesis of Facial Expressions

Irfan A. Essa
Ph. D. Thesis, Massachusetts Institute of Technology, Cambridge, MA. 1994. Also available as Vision & Modeling TR # 303

Tracking Facial Motion

Irfan A. Essa, Trevor Darrell and Alex Pentland
In Proceedings of IEEE Workshop on Nonrigid and Articulated Motion, Austin, TX., November 1994. Also available as Vision & Modeling TR # 272

Correlation and Interpolation Networks for Real-time Expression Analysis/Synthesis.

Trevor Darrell, Irfan A. Essa, and Alex Pentland
In G. Tesauro, D. S. Touretzky, and T. K. Leen (Editors), Advances in Neural Information Processing Systems (NIPS) 7, Denver, CO., MIT Press 1995. Also available as Vision & Modeling TR # 284

A Vision System for Observing and Extracting Facial Action Parameters

Irfan A. Essa and Alex Pentland
Proceedings of IEEE Computer Vision and Pattern Recognition Conference, pages 76-83, June 1994. Also available as Vision & Modeling TR # 247

Visually-Guided Animation

A. Pentland, T. Darrell, I. Essa, A. Azarbayejani, and S. Sclaroff
In Proceedings of the Computer Animation 1994 Conference, Geneva, Switzerland May 1994. Also available as Vision & Modeling TR # 266

Physically-based Modeling for Graphics and Vision

Irfan A. Essa, Stan Sclaroff, and Alex Pentland
Ralph Martin, editor, Directions in Geometric Computing. Information Geometers, U.K., 1993. Also available as Vision & Modeling TR # 184

A Unified Approach for Physical and Geometric Modeling for Graphics and Animation

Irfan A. Essa, Stan Sclaroff, and Alex Pentland
Computer Graphics Forum, The International Journal of the Eurographics Association, 2(3), 1992.

Vision-based Modeling: An Application of a Unified Approach for Physical and Geometric Modeling for Graphics and Animation

Stan Sclaroff, Irfan A. Essa, and Alex Pentland
In Proceedings of the Workshopp on Animations and Simulations, Eurographics 92, Cambridge, UK. 1992.

The ThingWorld Modeling System: Virtual Sculpting by Modal Forces

A. Pentland, I. Essa, M. Friedmann, B. Horowitz, and S. Sclaroff
ACM Computer Graphics, Proceedings of the Second Interactive 3-D Graphics Symposium, Snowbird, Utah. 1990.

Some relevant people

Undergraduate:

Alex Kuperman , Aaron Thomason, Mike Sy, and Nathan J. Williams

Staff:

Erik Trimble .

Graduate:

Sumit Basu (with Alex Pentland) .

Presently Collaborating with:

Alex (Sandy) Pentland , Matthew Brand , Nancy Etcoff, Nancy Alvarado and Justine Cassell .

Former Collaboraters:

Trevor Darrell , Stan Scalroff

References:

J. K. Aggarwal, Norm Badler, Aaron Bobick, Larry S. Davis, Nancy Etcoff, Thomas Huang, Takeo Kanade, Marvin Minsky (Occasionaly) , Alex (Sandy) Pentland, Rosalind (Roz) Picard , John Williams.

Interests

Biking, Hiking, Reading (metaphysics, philosophy, fiction) Swimming, Photography and playing Squash.

Birthplace

Karachi, Pakistan (1965)

A favorite book

CRC Standard Math Tables (1987) by W. H. Beyer

Most wishes to have dinner with

Anybody from years 3000 AD or 3000 BC Or Someone from another planet (No that does not mean that I don't wanna have dinner with people in the present living in good ole terra

Personal Web Page

Vismod Home Page