Home | News | Join Us | People | Projects | Publications | Talks | Courses
Home | News | Join Us | People | Projects | Publications | Talks | Courses
Ramesh Raskar, Associate Professor, MIT Media Lab
Douglas Lanman, Postdoctoral Associate, MIT Media Lab
Gordon Wetzstein, Postdoctoral Associate, MIT Media Lab
Matthew Hirsch, MIT Media Lab
Wolfgang Heidrich, Professor, University of British Columbia, Department of Computer Science
Yunhee Kim, Postdoctoral Fellow, MIT Media Lab
Szymon Jakubczak, MIT Computer Science and Artificial Intelligence Laboratory (CSAIL)
To create "holographic" 3D displays supporting videos requires capturing, storing, transmitting, and decoding terabytes of data per second. This is simply not possible with consumer technology in the near term. To overcome these bandwidth limitations, our group has invented a series of techniques to exploit the coherence in nearby views of 3D scenes. We employ these methods to depict 3D scenes, without the need for eyeglasses, using optimized optical hardware and co-designed "compressive" image-encoding algorithms. To date, we have explored the use of such "compressive displays" that are primarily composed of multiple layers of high-speed liquid crystal displays (LCDs): a well-established display technologiy that may lead to near-term commercial 3D displays. Most recently, we have introduced "tensor displays": a family of compressive 3D displays comprising all architectures employing a stack of time-multiplexed, light-attenuating layers illuminated by uniform or directional backlighting (i.e., any LCD-based, glasses-free 3D display). We show that the set of views of the 3D scenes, as emitted by an multlayer, high-speed tensor display, can be represented using tensor algebra (a mathematical framework previously applied to compress complex, high-dimensional data sets). Using this representation, we introduce a unified optimization framework, based on nonnegative tensor factorization (NTF), encompassing all tensor display architectures and leading to enhanced glasses-free 3D displays with current LCD technology.
Current commercial 3D displays typically require the viewer to use special eyewear. For example, theatrical systems require either polarized glasses or specialized color filters. In the home, users must similarly wear polarized eyewear or shutter glasses. Emerging glasses-free 3D displays eliminate the need for such eyewear, but also introduce a host of limitations not encountered when using specialized eyewear. In particular, glasses-free 3D displays often limit the field of view: the viewer must be located within a narrow cone directly in front of the display. In addition, glasses-free 3D displays often produce imagery with a limited depth of field: as objects float away from the display surface, they become blurred. Researchers have attempted to address these limitations with a host of optical solutions, including holographic elements, spinning or other high-speed moving parts, or explicitly tracking the location of viewers, increasing cost and reducing practicality. To date, these approaches have not yielded robust commercial systems that can approach the price and performance of simpler designs requiring eyewear—explaining their persistence in a market that would otherwise embrace glasses-free 3D displays.
Our group is pursuing a complementary approach advocating refined optical hardware, but also the simultaneous adoption of novel compressive display algorithms. While compression algorithms are routinely applied for efficient transmission and storage of terabytes of data (e.g., for high-definition video), their application at the time of display represents a departure from conventional display designs. Traditionally, display engineers design screens to faithfully reproduce the brightness of each pixel. However, for 3D displays, accurate rendition of multiple views places demands on the display hardware, decoding software, and transmission bandwidth that simply cannot be met with current technology. Thus, our group has pursued a series of projects demonstrating that "compressive displays" (i.e., those that exploit correlation between views at the time of display) can overcome these constraints using current display hardware, albeit configured in new ways (e.g., layered, high-speed LCDs).
Tensor DisplaysWe introduce tensor displays: a family of compressive light field displays comprising all architectures employing stacks of time-multiplexed, light-attenuating layers illuminated by uniform or directional backlighting (i.e., any low-resolution light field emitter). We construct a prototype and show interactive display using a GPU-based implementation. Files: |
|---|
Polarization FieldsWe introduce polarization fields as an optically-efficient construction for light field display with layered LCDs. Such displays contain a stack of liquid crystal panels with a single pair of crossed polarizers, with layers acting as spatially-controllable polarization rotators. We construct a prototype and show interactive display using a GPU-based implementation. Files: |
|---|
Layered 3DWe optimize automultiscopic displays composed of compact volumes of light-attenuating material. Inexpensively fabricated by stacking transparencies, the attenuators recreate a light field when back-illuminated. Tomographic optimization resolves inconsist views, leading to brighter, higher-resolution 3D displays with greater depth of field and higher dynamic range. Files: |
|---|
High-Rank 3D (HR3D)We stack a pair of modified LCD panels. Rather than using heuristically-defined parallax barriers, we demonstrate that both layers can be simultaneously optimized for the multi-view content. This process leads to 3D displays with increased brightness and refresh rate. Files: |
|---|
G. Wetzstein, D. Lanman, M. Hirsch, and R. Raskar. Tensor Displays:
Compressive Light Field Synthesis using Multilayer Displays with Directional Backlighting. ACM Transactions on Graphics (SIGGRAPH 2012), August 2012
[paper, project page]
D. Lanman, G. Wetzstein, M. Hirsch, W. Heidrich, and R. Raskar. Beyond Parallax Barriers: Applying Formal Optimization Methods to Multi-Layer Automultiscopic Displays. In SPIE Stereoscopic Displays and Applications XXIII, January 2012
[paper]
D. Lanman, G. Wetzstein, M. Hirsch, and R. Raskar. Depth of Field Analysis for Multilayer Automultiscopic Displays. In OSA International Symposium on Display Holography (ISDH 2012), June 2012
[paper]
G. Wetzstein, D. Lanman, M. Hirsch, and R. Raskar. Real-Time Image Generation for Compressive Light Field Displays. In OSA International Symposium on Display Holography (ISDH 2012), June 2012
[paper]
M. Hirsch, D. Lanman, G. Wetzstein, and R. Raskar. Construction and Calibration of LCD-based Multi-Layer Light Field Displays. In OSA International Symposium on Display Holography (ISDH 2012), June 2012
[paper]
D. Lanman, G. Wetzstein, M. Hirsch, W. Heidrich, and R. Raskar. Polarization Fields: Dynamic Light Field Display using Multi-Layer LCDs. ACM Transactions on Graphics (SIGGRAPH Asia 2011), December 2011
[paper, project page]
G. Wetzstein, D. Lanman, W. Heidrich, and R. Raskar. Layered 3D: Tomographic Image Synthesis for Attenuation-based Light Field and High Dynamic Range Displays. ACM Transactions on Graphics (SIGGRAPH 2011), August 2011
[paper, project page]
D. Lanman, M. Hirsch, Y. Kim, and R. Raskar. Content-Adaptive Parallax Barriers for Automultiscopic 3D Display. ACM Transactions on Graphics (SIGGRAPH Asia 2010), December 2010
[paper, project page]
We thank the entire Camera Culture group for their unrelenting support, as well as members of the Information Ecology group at the MIT Media Lab and the Imager Laboratory at the University of British Columbia.
This research is supported by research grants from MIT Media Lab sponsors. Additional sponsorship is provided by Samsung Electronics, NVIDIA, and Dolby. Douglas Lanman was supported by NSF Grant CCF-0729126 and IIS-1116452, as well as the DARPA MOSAIC program. Gordon Wetzstein was supported by a UBC Four Year Fellowship and the DARPA SCENICC program. Yunhee Kim was supported by NRF of Korea Grant 2009-352-D00232. Wolfgang Heidrich is supported under the Dolby Research Chair in Computer Science at UBC. Ramesh Raskar is supported by an Alfred P. Sloan Research Fellowship (2009) and a DARPA Young Faculty Award (2010).
Technical Details
Ramesh Raskar, Associate Professor, MIT Media Lab
raskar (at) media.mit.edu
Press
Alexandra Kahn, Senior Press Liaison, MIT Media Lab
akahn (at) media.mit.edu or 617/253.0365
|
|
|
|
|
| Cameras |
6D Display Lighting and Viewpoint aware displays |
Bokode Long Distance Barcodes |
BiDi Screen Touch+3D Hover on Thin LCD |
NETRA Cellphone based Eye Test |
Home | News | Join Us | People | Projects | Publications | Talks | Courses