Whole-Hand Input
by
David J. Sturman, Ph.D.
February 1992

Copyright Massachusetts Institute of Technology, 1992.
All rights reserved.

Abstract

This dissertation examines whole-hand input: the full and direct use of the hand's capabilities for the control of computer-mediated tasks. It presents the subject as a distinct study, independent of specific application or interface device. It includes a comprehensive discussion of the ideas, issues, and technologies relevant to the field.

Whole-hand input is a powerful tool for the real-time control of complex computer-mediated tasks that require the manipulation and coordination of many degrees of freedom. By taking advantage of the innate naturalness, adaptability, and dexterity of the hand, whole-hand input techniques can provide performance superior to that of conventional devices (such as dials, mice, and joysticks) when applied to complex tasks.

The important problems of whole-hand input involve appropriateness of use, control design, and device selection. The dissertation addresses these with a design method for whole-hand input by which an interface designer can discuss, develop, and evaluate techniques and devices for using whole-hand input in a particular application. Three experiments illustrate use of the design method and validate the principles of the thesis.

A testbed and software library for investigating whole-hand input techniques is described. The testbed allows easy development and testing of whole-hand input with application simulations. The library is based on an abstract whole-hand input device type providing a standard interface to different physical whole-hand input devices. It features techniques for device calibration, posture recognition, and gesture recognition.

Three prototype applications using the testbed, and one musical performance application demonstrate a variety of whole-hand input techniques including master-slave control, controlling task variables with hand shape, and gestural command input.

The text concludes with detailed recommendations for future work to forward the understanding of the direct use of the hand as an input device.

An accompanying videotape demonstrates the three experiments, the prototype applications, and shows a short section of the musical performance.