Project Image

A Switchable Light Field Camera Architecture with Angle Sensitive Pixels and Dictionary-based Sparse Coding

IEEE ICCP 2014 Best Paper

Matthew Hirsch1

Sriram Sivaramakrishnan2

Suren Jayasuriya2

Albert Wang2

Alyosha Molnar2

Ramesh Raskar1

Gordon Wetzstein1

1MIT Media Lab

2Cornell University

Teaser Image

Prototype angle sensitive pixel camera (left). The data recorded by the camera prototype can be processed to recover a high-resolution 4D light field (center). As seen in the close-ups on the right, parallax is recovered from a single camera image.


We propose a flexible light field camera architecture that is at the convergence of optics, sensor electronics, and applied mathematics. Through the co-design of a sensor that comprises tailored, Angle Sensitive Pixels and advanced reconstruction algorithms, we show that—contrary to light field cameras today—our system can use the same measurements captured in a single sensor image to recover either a high-resolution 2D image, a low-resolution 4D light field using fast, linear processing, or a high-resolution light field using sparsity-constrained optimization.



Note: Slideshare does not convert animations properly. I recommend downloading the slides from the slideshare site for the full experience!


    Paper [pdf]
    Video [mp4]
    Presentation Slides [slideshare]


M. Hirsch, S. Sivaramakrishnan, S. Jayasuriya, A. Wang, A. Molnar, R. Raskar, G. Wetzstein. A Switchable Light Field Camera Architecture with Angle Sensitive Pixels and Dictionary-based Sparse Coding. Computational Photography (ICCP), 2014 IEEE International Conference on. IEEE, 2014.


  title={{A} {S}witchable {L}ight {F}ield {C}amera {A}rchitecture with {A}ngle {S}ensitive {P}ixels and {D}ictionary-based {S}parse {C}oding},
  author={Hirsch, Matthew and Sivaramakrishnan, Sriram and Jayasuriya, Suren and Wang, Albert and Molnar, Alyosha and Raskar, Ramesh and Wetzstein, Gordon},
  booktitle={Computational Photography (ICCP), 2014 IEEE International Conference on},

Additional Information

ASP Diode Figure

Schematic of a single angle sensitive pixel. Two interleaved photodiodes capture a projection of the light field incident on the sensor (left). The angular responses of these diodes are complementary: a conventional 2D image can be synthesized by summing their measurements digitally (right).

Frequency Domain Intuition Figure

Illustration of ASP sensor layout (left) and sampled spatio-angular frequencies (right). The pictured sensor interleaves three different types of ASPs. Together, they sample all frequencies contained in the dashed green box (right). A variety of light field reconstruction algorithms can be applied to these measurements, as described in the text.

Impulse Tile Figure

Microscopic image of a single 6 × 4 pixel tile of the ASP sensor (left). We also show captured angular point spread functions (PSFs) of each ASP pixel type (right).

Comparison of Methods Figure

Comparison of different reconstruction techniques for the same captured data. We show reconstruction of a 2D image (bottom right), a low-resolution light field via linear reconstruction (bottom left and center), and a high-resolution light field via sparsity-constrained optimization with overcomplete dictionaries (top). Whereas linear reconstruction trades angular for spatial resolution — thereby decreasing image fidelity — nonlinear reconstructions can achieve an image quality that is comparable to a conventional, in-focus 2D image for each of 25 recovered views.

Results Overview Figure

Overview of captured scenes showing mosaics of light fields reconstructed via sparsity-constrained optimization (top), a single view of these light fields (center), and corresponding 2D images (bottom). These scenes exhibit a variety of effects, including occlusion, refraction, specularity, and translucency. The resolution of each of the 25 light field views is similar to that of the conventional 2D images.

Best Paper Award

Authors accept the best paper award

The authors accept the best paper award at ICCP 2014. Right to left, Ramesh Raskar, Suren Jayasuriya, Matthew Hirsch, Gordon Wetzstein, and Sriram Sivaramakrishnan

Site Details



The best way to get in touch with me is email.


This website uses twitter's Boostrap 3 CSS framework.


The theme for this site is based heavily on the How to Integrate Simple Parallax with Twitter Bootstrap tutorial offered by Otherwise, they had nothing to do with this, so don't blame my lack of style on them! Thanks for being awesome and sharing!


This site was last updated on May 9, 2014.

Copyright © IEEE 2014. This is the author's version of the work. It is posted here by permission of IEEE for your personal use. Not for redistribution.