A little blurb about Matt Hirsch.
Dr. Hirsch is a co-founder and Chief Technical Officer of Lumii. He worked with Henry Holtzman's Information Ecology Group and Ramesh Raskar's Camera Culture Group at the MIT Media Lab, making the next generation of interactive and glasses-free 3D displays. Matthew received his bachelors from Tufts University in Computer Engineering, and his Masters and Doctorate from the MIT Media Lab. Between degrees, he worked at Analogic Corp. as an Imaging Engineer, where he advanced algorithms for image reconstruction and understanding in volumetric x-ray scanners. His work has been funded by the NSF and the Media Lab consortia, and has appeared in SIGGRAPH, CHI, and ICCP. Matthew has also taught courses at SIGGRAPH on a range of subjects in computational imaging and display, with a focus on DIY.
What are they doing now?
After graduating from the MIT Media Lab, Matt, along with co-founders Tom Baran and Daniel Leithinger created Lumii , to change the way we build displays. You can see a little demo of one of our light field display prototypes below.
Right now at Lumii you can print out a 3D Holiday Card on your home printer! Just go to lumiidisplay.com/holiday . It will look something like this when you're done with it.
MIT Media Lab Projects
Compressive Light Field Projector
For about a century, researchers and experimentalists have strived to bring glasses-free 3D experiences to the big screen. Much progress has been made and light field projection systems are now commercially available. Unfortunately, available display systems usually employ dozens of devices making such setups costly, energy inefficient, and bulky. We present a compressive approach to light field synthesis with projection devices. For this purpose, we propose a novel, passive screen design that is inspired by angle-expanding Keplerian telescopes. Combined with high-speed light field projection and nonnegative light field factorization, we demonstrate that compressive light field projection is possible with a single device. We build a prototype light field projector and angle-expanding screen from scratch, evaluate the system in simulation, present a variety of results, and demonstrate that the projector can alternatively achieve super-resolved and high dynamic range 2D image display when used with a conventional screen.
View details »
Focus 3D
We present a glasses-free 3D display design with the potential to provide viewers with nearly correct accommodative depth cues, as well as motion parallax and binocular cues. Building on multilayer attenuator and directional backlight architectures, the proposed design achieves the high angular resolution needed for accommodation by placing spatial light modulators about a large lens: one conjugate to the viewer's eye, and one or more near the plane of the lens. Nonnegative tensor factorization is used to compress a high angular resolution light field into a set of masks that can be displayed on a pair of commodity LCD panels. By constraining the tensor factorization to preserve only those light rays seen by the viewer, we effectively steer narrow high resolution viewing cones into the user's eyes, allowing binocular disparity, motion parallax, and the potential for nearly correct accommodation over a wide field of view. We verify the design experimentally by focusing a camera at different depths about a prototype display, establish formal upper bounds on the design's accommodation range and diffraction-limited performance, and discuss practical limitations that must be overcome to allow the device to be used with human observers.
View details »
ASP Light Field Camera
We propose a flexible light field camera architecture that
is at the convergence of optics, sensor electronics, and
applied mathematics. Through the co-design of a sensor
that comprises tailored, Angle Sensitive Pixels and
advanced reconstruction algorithms, we show that—contrary
to light field cameras today—our system can use the same
measurements captured in a single sensor image to recover
either a high-resolution 2D image, a low-resolution 4D
light field using fast, linear processing, or a
high-resolution light field using sparsity-constrained
optimization.
View details »
8D Display
Imagine a display that behaves like a window. Glancing
through it, viewers perceive a virtual 3D scene with correct
parallax, without the need to wear glasses or track the
user. Light that passes through the display correctly
illuminates the virtual scene. We contribute a new,
interactive, relightable, glasses-free 3D display. By
simultaneously capturing a 4D light field, and displaying a
4D light field, we are able to realistically modulate the
incident light on rendered content. Our hardware points the
way towards novel 3D interfaces, in which users interact
with digital content using light widgets, physical objects,
and gesture.
View details »
Tensor Display
We introduce tensor displays: a family of glasses-free 3D
displays comprising all architectures employing (a stack of)
time-multiplexed LCDs illuminated by uniform or directional
backlighting. We introduce a unified optimization framework
that encompasses all tensor display architectures and allows
for optimal glasses-free 3D display.
We demonstrate the
benefits of tensor displays by constructing a reconfigurable
prototype using modified LCD panels and a custom integral
imaging backlight. Our efficient, GPU-based NTF
implementation enables interactive applications. In our
experiments we show that tensor displays reveal practical
architectures with greater depths of field, wider fields of
view, and thinner form factors, compared to prior
automultiscopic displays.
View details »
SIGGRAPH 2012 Paper »
CGA 2012 Paper »
ISDH 2012 DOF »
ISDH 2012 Real-Time »
ISDH 2012 Calibration »
Polarization Fields
We introduce polarization field displays as an
optically-efficient design for dynamic light field display
using multi-layered LCDs. Such displays consist of a stacked
set of liquid crystal panels with a single pair of crossed
linear polarizers. Each layer is modeled as a
spatially-controllable polarization rotator, as opposed to a
conventional spatial light modulator that directly
attenuates light.We demonstrate interactive display using a
GPU-based SART implementation supporting both
polarization-based and attenuation-based
architectures. Experiments characterize the accuracy of our
image formation model, verifying polarization field displays
achieve increased brightness, higher resolution, and
extended depth of field, as compared to existing
automultiscopic display methods for dual-layer and
multi-layer LCDs.
View details »
High-Rank 3D
Today's 3D display are not only light deficient, but
rank deficient. We have developed a 3D display that
eliminates the need for special glasses, while solving both
light and rank deficiency. Until now, the commercial
potential of glasses-free 3D displays, particularly those
based on liquid crystal displays (LCDs), has been primarily
limited by decreased image resolution and brightness
compared to systems employing special eyewear.
In the Camera Culture group at the MIT Media Lab, we have
found a way to increase the brightness and resolution of
LCD-based, glasses-free 3D displays using a method they
call Content-Adaptive Parallax Barriers . We call
our new display technology High-Rank 3D
or HR3D , since our display is capable of
displaying a full-resolution light field.
View details »
BiDi Screen
The BiDi Screen is an example of a new type of I/O device
that possesses the ability to both capture images and
display them. This thin, bidirectional screen extends the
latest trend in LCD devices, which has seen the
incorporation of photo-diodes into every display
pixel. Using a novel optical masking technique developed
at the Media Lab, the BiDi Screen can capture
lightfield-like quantities, unlocking a wide array of
applications from 3-D gesture interaction with CE devices,
to seamless video communication.
View details »
Bike Commute Archive
A Media Lab researcher has been kind enough to share his
daily bicycle commute for research and entertainment
purposes. These videos are offered under a creative commons
license. The archive covers about 2.5 years of
commuting.
View details »
Twitter Laundry
This page describes how we turned some electronic junk we
found in a spare parts bin into a twittering waching machine
and dryer. With any luck, twitter will one day be filled
entirely with the banal updates of machines.
View details »
Kaidan Turntable
The Kaidan
Magellan Turntable (MDT-19) is a motorized turntable
originally intended for scientific imaging. We have one of
these in
the Camera
Culture group , which has been passed down from
generation to generation, and mostly neglected along the
way. Here I host some python code to get the table running
again.
View details »
Build Your Own 3D Display
This course provides attendees with the mathematics,
software, and practical details they need to build their
own low-cost stereoscopic displays. Each new concept is
illustrated using a practical 3D display implemented with
off-the-shelf parts. First, the course explains
glasses-bound stereoscopic displays and provides detailed
plans for attendees to construct their own LCD shutter
glasses. Then the course explains unencumbered
auto-multiscopic displays, including step-by-step
directions to construct lenticular and parallax-barrier
designs using modified LCDs. All the necessary software,
including algorithms for rendering and calibration, is
provided for each example, so attendees can quickly
construct 3D displays for their own educational,
amusement, and research purposes.
Course Website »
@SIGGRAPH 2010 »
@SIGGRAPH Asia 2010
Build Your Own Glasses-Free 3D Display
At SIGGRAPH 2010, the Build Your Own 3D Display course
demonstrated how to construct both LCD shutter glasses and
glasses-free lenticular screens, providing Matlab-based
code for batch encoding of 3D imagery. This follow-up
course focuses more narrowly on glasses-free displays,
describing in greater detail the practical aspects of
real-time, OpenGL-based encoding for such multi-view,
spatially multiplexed displays.
The course reviews historical and perceptual aspects,
emphasizing the goal of achieving disparity, motion
parallax, accommodation, and convergence cues without
glasses. It summarizes state-of-the-art methods and areas
of active research. And it provides a step-by-step
tutorial on how to construct a lenticular display. The
course concludes with an extended question-and-answer
session, during which prototype hardware is available for
inspection.
Course Website »
@SIGGRAPH 2011 »
Computational Displays
This course serves as an introduction to the emerging
field of computational displays. The pedagogical goal of
this course is to provide the audience with the tools
necessary to expand their research endeavors by providing
step-by-step instructions on all aspects of computational
displays: display optics, mathematical analysis, efficient
computational processing, computational perception, and,
most importantly, the effective combination of all these
aspects. Specifically, we will discuss a wide variety of
different applications and hardware setups of
computational displays, including high dynamic range
displays, advanced projection systems as well as
glasses-free 3D display. The latter example, computational
light field displays, will be discussed in detail. In the
course presentation, supplementary notes, and an
accompanying website, we will provide source code that
drives various display incarnations at real-time
framerates, detailed instructions on how to fabricate
novel displays from off-the-shelf components, and
intuitive mathematical analyses that will make it easy for
researchers with various backgrounds to get started in the
emerging field of computational displays. We believe that
computational display technology is one of the "hottest"
topics in the graphics community today; with this course
we will make it accessible for a diverse audience. While
the popular, introductory-level courses "Build Your Own 3D
Displays" and "Build Your Own Glasses-free 3D Display",
previously taught at SIGGRAPH and SIGGRAPH ASIA, discussed
conventional 3D displays invented in the past, this course
introduces what we believe to be the future of display
technology. We will only briefly review conventional
technology and focus on practical and intuitive
demonstrations of how an interdisciplinary approach to
display design encompassing optics, perception,
computation, and mathematical analysis can overcome the
limitations for a variety of applications.
Course Website »
@SIGGRAPH 2012 »
Hirsch, M., Wetzstein, G., & Raskar, R. (2014). A Compressive Light Field Projection System. ACM Transactions on Graphics (SIGGRAPH), 33(4).
Maimone, A., Wetzstein, G., Hirsch, M., Lanman, D., Raskar, R., & Fuchs, H. (2013). Focus 3D: Compressive accommodation display. ACM Transactions on Graphics (TOG), 32(5), 153.
Hirsch, M., Sivaramakrishnan, S., Jayasuriya, S., Wang, A., Molnar, A., Raskar, R., & Wetzstein, G. (2014, May). A switchable light field camera architecture with Angle Sensitive Pixels and dictionary-based sparse coding. In Computational Photography (ICCP), 2014 IEEE International Conference on (pp. 1-10). IEEE.
Hirsch, M., Lanman, D., Holtzman, H., & Raskar, R. (2009, December). BiDi screen: a thin, depth-sensing LCD for 3D interaction using light fields. In ACM Transactions on Graphics (TOG) (Vol. 28, No. 5, p. 159). ACM
Lanman, D., Hirsch, M., Kim, Y., & Raskar, R. (2010, December). Content-adaptive parallax barriers: optimizing dual-layer 3D displays using low-rank light field factorization. In ACM Transactions on Graphics (TOG) (Vol. 29, No. 6, p. 163). ACM.
Wetzstein, G., Lanman, D., Hirsch, M., & Raskar, R. (2012). Tensor displays: compressive light field synthesis using multilayer displays with directional backlighting. ACM Transactions on Graphics (TOG), 31(4), 80.
Lanman, D., Wetzstein, G., Hirsch, M., Heidrich, W., & Raskar, R. (2011, December). Polarization fields: dynamic light field display using multi-layer LCDs. In ACM Transactions on Graphics (TOG) (Vol. 30, No. 6, p. 186). ACM.
Maimone, A., Wetzstein, G., Hirsch, M., Lanman, D., Raskar, R., & Fuchs, H. (2013). Focus 3D: compressive accommodation display. ACM Transactions on Graphics (TOG), 32(5), 153.
Lanman, D., Wetzstein, G., Hirsch, M., Heidrich, W., & Raskar, R. (2012, February). Beyond parallax barriers: applying formal optimization methods to multilayer automultiscopic displays. In Proc. SPIE (Vol. 8288, p. 82880A).
Hirsch, M., Izadi, S., Holtzman, H., & Raskar, R. (2012, November). 8D display: a relightable glasses-free 3D display. In Proceedings of the 2012 ACM international conference on Interactive tabletops and surfaces (pp. 319-322). ACM.
Hirsch, M., Izadi, S., Holtzman, H., & Raskar, R. (2013, April). 8d: interacting with a relightable glasses-free 3d display. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 2209-2212). ACM.
Lanman, D., Wetzstein, G., Hirsch, M., & Raskar, R. (2013, February). Depth of Field Analysis for Multilayer Automultiscopic Displays. In Journal of Physics: Conference Series (Vol. 415, No. 1, p. 012036). IOP Publishing.
Wetzstein, G., Lanman, D., Hirsch, M., & Raskar, R. (2013, February). Real-time Image Generation for Compressive Light Field Displays. In Journal of Physics: Conference Series (Vol. 415, No. 1, p. 012045). IOP Publishing.
Hirsch, M., Lanman, D., Wetzstein, G., & Raskar, R. (2013, February). Construction and Calibration of Optically Efficient LCD-based Multi-Layer Light Field Displays. In Journal of Physics: Conference Series (Vol. 415, No. 1, p. 012071). IOP Publishing.
Karbeyaz, B. U., Naidu, R. C., Ying, Z., Simanovsky, S. B., Hirsch, M. W., Schafer, D. A., & Crawford, C. R. (2008). Variable Pitch Reconstruction Using John's Equation. Medical Imaging, IEEE Transactions on, 27(7), 897-906.
Hirsch, M., Wetzstein, G., Raskar, R. “Methods and Apparatus for Light Field Projection,” Asignee: Massachusetts Institute of Technology, United States patent application US 20140300869 A1.
Wetzstein, G., Lanman, D., Hirsch, M., Raskar, R. “Tensor Displays,” Asignee: Massachusetts Institute of Technology, United States patent application US 2014/0063077 A1.
Lanman, D., Wetzstein, G., Hirsch, M., Heidrich, W., Raskar, R., “Polarization Fields: Dynamic Light Field Display using Multi-Layer LCDs,” Asignee: Massachusetts Institute of Technology, United States patent US 2013/0176704 A1.
Lanman, D., Hirsch, M., Kim, Y., Jakubczak, S., Raskar, R., “Content Adaptive Parallax Barriers for Automultiscopic Display,” Asignee: Massachusetts Institute of Technology, United States patent application US 2012/0140131 A1.
Hirsch, M. W., Raskar, R., Holtzman, H., Lanman, D., “Bi-Directional Screen,” Assignee: Massachusetts Institute of Technology, United States patent US 2011/0019056 A1.
Naidu, R., Ying, Z., Simanovsky, S., Hirsch, M. W., Crawford, C. R., “Method of and system for classifying objects using histogram segment features of multi-energy computed tomography images,” Assignee: Analogic Corporation, United States patent US 2007/0031036 A1.
Ying, Z., Naidu, R., Simanovsky, S., Hirsch, M. W., and Crawford, C. R., "Method of and system for classifying objects using local distributions of multi-energy computed tomography images," Assignee: Analogic Corporation, United States patent US 2007/0014472 A1.
Ying, Z., Hirsch, M. W., Desai, P., Simanovsky, S., and Crawford, C. R., "Method of and system for 3D display of multi-energy computed tomography slices," Assignee: Analogic Corporation, United States patent US 2006/0274066 A1.
