Ramesh Raskar
| | | Publications | | | Courses | | | Talks | | | Patents | | | Press | | | Art&Demos | | | Personal | | |
MIT Media
Lab , Associate
Professor
[raskar@media(.)mit(.)edu,
Room E15-324, 20 Ames St, Cambridge, MA 02139]
Center for Future Storytelling, Co-director
Computational Photography
Previous Homepage at MERL for Details
| Fall
2008 course: MAS 963 (H) Computational Camera and Photography (6-3-3) Fridays 1:30-4:30pm in E15-283A (Roth room) |
Spring 2009 course: MAS 964 (H)
Special Topics in Media Technology:
Camera Culture (0-9-0) W 3pm-5pm in E15-235
Projects
Computational Photography
Computational Photography is an emerging multi-disciplinary field that is at the intersection of optics, signal processing, computer graphics+vision, electronics, art, and online sharing in social networks. The field is evolving through three phases. The first phase was about building a super-camera that has enhanced performance in terms of the traditional parameters, such as dynamic range, field of view or depth of field. I call this Epsilon Photography. Due to limited capabilities of a camera, the scene is sampled via multiple photos, each captured by epsilon variation of the camera parameters. It corresponds to the low-level vision: estimating pixels and pixel features. The second phase is building tools that go beyond capabilities of this super-camera. I call this Coded Photography. The goal here is to reversibly encode information about the scene in a single photograph (or a very few photographs) so that the corresponding decoding allows powerful decomposition of the image into light fields, motion deblurred images, global/direct illumination components or distinction between geometric versus material discontinuities. This corresponds to the mid-level vision: segmentation, organization, inferring shapes, materials and edges. The third phase will be about going beyond the radiometric quantities and challenging the notion that a camera should mimic a single-chambered human eye. Instead of recovering physical parameters, the goal will be to capture the visual essence of the scene and analyze the perceptually critical components. I call this Essence Photography and it may loosely resemble depiction of the world after high level vision processing. It will spawn new forms of visual artistic expression and communication. Related work
Coded Time (Exposure)
Coding in Space
Coded Illumination
Coded Wavelength
Coded Sensing
Flutter Shutter Cam
Coded Aperture Optical Heterodyning
Multi-flash Camera
Agile Spectrum
Gradient Processing
2006
2007
2007
2004
2008
2005
Traditionally, a projector is considered a display device that maps a 2D image onto a larger flat screen. In our research, we have questioned this restrictive consideration. By treating projector as the dual of a camera – that is, a 3D projective device that maps 2D pixels to 4D rays – we have shown that programmable illumination can be used in a versatile fashion for display, augmentation, communication and mobility. Previous related work
Disclosed Product Impact:
Acute Angle Seamless Multiprojector Display
MPC Polgyonal 3D scene display
Multiprojector quadric curved immersive display
Six Projector 3D rendering on curved display (A detailed press release)
Radio Frequency Identity and Geometry (RFIG), location using photosensing RFID
We bring RFID and wireless tags into the realm of computer vision, graphics and HCI by supporting precise geometric analysis of tagged objects and surfaces using a composite RF and optical communication. Related work
Non photorealistic Rendering
Features such as silhouette edges, ridges, valleys and intersections of geometric primitives are useful in conveying shape in illustrations and art. We have developed a proactive imaging method to capture such features in real world scenes of various scales. One area of focus is shape clarifying multi-flash imaging for endscopes. For polygonal scenes, our focus has been real-time rendering of these features without requiring the connectivity information. Previous related work
Intelligent user interfaces
We are developing user interaction methods with novels gadgets and sensors. We have also exploited variational implicit surfaces for free form sketching and manipulation. Previous related work
Publications
2008
Paper: Glare Aware Photography: 4D Ray Sampling for Reducing Glare Effects of Camera Lenses
R Raskar, A Agrawal, C Wilson, A Veeraraghavan
SIGGRAPH 2008
First 'single-shot' approach to classify and reduce glare. We show that glare manifests as an outlier in 4D ray space and use a statistical approach to remove it by 4D sampling without reconstructing a 4D-lightfield.
Paper: Towards 6D Displays
M Fuchs, Raskar, HP Seidel, H Lensch
SIGGRAPH 2008
A method for embedding 4D and 6D data into 2D films and employing lenslet arrays so that an observer experiences encoded objects as if they are lit by real-world incident illumination.
Paper: Shield Fields: Heterodyned Shadows for Single-Shot Capture and Visual Hull Reconstruction of 3D Occluders
Lanman, Raskar, Agrawal, Taubin
SIGGRAPH ASIA 2008
A method for capturing shadows created due to multiple light sources using a mask and converting shadows into a 3D visual hull.
2007
Paper: Coded aperture and Optical Heterodyning : A mask-based approach for Digital Refocusing and Light Field Acquisition by Conventional Cameras
A Veeraraghavan, R Raskar, A Agrawal, A Mohan, J Tumblin
SIGGRAPH 2007
We exploit Fourier domain remapping of 4D ray-space to capture 4D light field on a 2D sensor in a conventional camera using a transmissive mask. No lens array is used.
R Raskar, H Nii, B de Decker, Y Hashimoto, J Summet, D Moore, Y Zhao,J Westhues, P Dietz, M Inami, S Nayar, J Barnwell, M Noland, P Bekaert, V Branzoi, E Bruns
SIGGRAPH 2007
A high speed optical motion capture also measures orientation and incident illumination via photosensing tags. Imperceptible tags work in natural lighting, to support on-set mocap or real-time broadcast of virtual_sets.
Course: Computational Photography, Course details and Slides
Course: Spatial Augmented Reality:Merging Real and Virtual Worlds
Emerging Technologies : Illumination Sensitive Dynamic Virtual Sets, [More Details]
|
2006 |
R Raskar, A Agrawal, J Tumblin
SIGGRAPH 2006
This method flutters the camera shutter open and closed with a binary pseudo-random sequence during exposure. Fast-moving objects are blurred, but sequenced shuttering preserves all spatial frequencies, simplifying the deblurring process.
| Paper: Fast
Separation of Direct and Global Components of a Scene
Using High Frequency Illumination S Nayar, G Krishnan, M Grossberg, R Raskar SIGGRAPH 2006 This work proposes a fast capture method for separating a scene into its direct and global illumination components. The method can handle a variety of global effects including inter-reflections, subsurface scattering, and volumetric scattering. |
Course: Spatial Augmented Reality: A Modern Approach to Augmented Reality
Emerging Technologies Booth showing real-time demonstration of a new 500 Hz Motion Tracking which costs only a few hundred dollars, [More Details]
2005
A Agrawal, R Raskar, S Nayar, Y Li
SIGGRAPH 2005
We address three problems: over or under illumination at a given flash brightness, unwanted reflections in glass, and attenuation over depth. And show a strikingly simple technique based on local gradient analysis.
Course: Spatial Augmented Reality: A Modern Approach to Augmented Reality
Panel: The Ultimate Display(My position: Pocket Projector)
2004
R Raskar, K Tan, R Feris, J Yu, M Turk
SIGGRAPH 2004 Flashlights cast annoying slivers of shadow around the subject. We locate shape contours by analyzing these cast shadows. We then use these silhouettes for artistic effects. But the contours can be used for many Computer and Machine vision applications.
Emerging Technologies Booth showing real-time demonstration
Siggraph 2004 cover
Highlighted Paper
Highlighted Emerging Technology activity, (More)
Electronic Theater (conference video highlight)
Student Poster (Specularities)
R Raskar, P Beardsley, J vanBaar, Y Wang, P Dietz, J Lee, D Leigh, T Willwacher
SIGGRAPH 2004 It is difficult to precisely locate wireless tags using RF alone. But by adding a photosensor to these tags, we can locate using a hybrid RF+Optical scheme. We show this can be implemented in RFID tags and locate these tags using a pocket projector.
Emerging Technologies Booth showing real-time demonstration of handheld iLamps projectors
Highlighted Paper
2003
R Raskar, J vanBaar, P Beardsley, T Willwacher, S Rao, C Forlines
SIGGRAPH 2003
2002
2000
Matusik, Buehler, Raskar, Gortler, McMillan
SIGGRAPH 2000
R Raskar,G Welch, M Cutts, A Lake, L Stesin, H Fuchs,
SIGGRAPH 1998
Other Journal and Conference Papers
A Mohan, R Raskar, J Tumblin
Eurographics 2008
A Agrawal, R Raskar
CVPR 2007 (Oral)
Setlur, V., Takagi, S., Raskar, Ramesh., Gleicher, M., and Gooch, B.
Mobile and Ubiquitous Multimedia (MUM) 2005, ACM Press 2005.
Hongcheng Wang, Ning Xu, Ramesh Raskar and Narendra Ahuja
Graphical Models Volume 69, Issue 1, Pages 1-88 (January 2007)
A Agrawal, R Raskar, R Chellappa
CVPR 2006, New York, NY
A Agrawal, R Raskar, R Chellappa
ECCV 2006 (Oral), Graz, Austria
R Feris, R Raskar, L Chen, K Tan, M Turk
ICCV 2005 (Oral), Beijing
A Agrawal, R Chellappa, R Raskar
ICCV 2005, Beijing
R Raskar, P Dietz, P Beardsley, J vanBaar
Communication of the ACM (Invited Article) Sept 05
J Tumblin, A Agrawal, R Raskar
CVPR 2005, San Diego, CA
R Raskar, J vanBaar, T Willwacher, S Rao
Eurographics 2004, Grenoble, France
R Raskar, J vanBaar, P Beardsley
SID (Society for Information Display) 2004
(Comparison of Compute Grid with Projector-based Display Grid)
H Wang, R Raskar, N Ahuja
ICPR 2004
R Raskar, A Ilie, J Yu
ACM Nonphotorealistic Animation and Rendering (NPAR) 2004, Annecy, France ( Results, Project)
R Raskar
Invited Talk at Pro-Cams workshop, Nice, France, Oct 2003 (Slides)
R Raskar, K Low
Pacific Graphics 2002, Beijing, China
Bimber, Gatesy, Witmer, Raskar, Encarnacao
IEEE Computer, Sept 2002
O Karpenko, John F Hughes, R Raskar
Eurographics 2002, Saarbruken, Germany
R Raskar, R Ziegler, T Willwacher
NPAR 2002, Annecy, France
R Raskar, P Beardsley
IEEE CVPR, Lihue, 2001
R Raskar, G Welch, K Low, D Bandyopadhyay
Eurographics Workshop on Rendering, London, 2001
R Raskar
Siggraph/Eurographics Graphics Hardware, LA, 2001 Source Code
R Raskar, J vanBaar, J Chai
ACCV 2002
R Raskar, K Low
ACM Afrigraph, 2001
D Badyopadhyay, R Raskar, H Fuchs
IEEE ACM Int. Symp. on Augmented Reality (ISAR), New York, 2001
G Welch, H Fuchs, R Raskar, M Brown, H Towles
IEEE Computer Graphics and Applications, November 2000
R Raskar
IEEE VR 2000, March 2000, NJ
R Raskar, M Brown, R Yang, W Chen, G Welch, H Towles, B Seales, H Fuchs
IEEE Visualization 99, Oct 1999, San Francisco, CA
R Raskar, G Welch, W Chen
Second Int Workshop on Augmented Reality (IWAR'99), October 1999, San Francisco, CA
R Raskar
ACM Siggraph 99 Sketch
R Raskar, Michael Cohen
Symposium on Interactive 3D Graphics, Atlanta GA, April 1999 Source Code
R Raskar, Matt Cutts, Greg Welch, Wolfgang Stuerzlinger
Ninth EuroGraphics Rendering Workshop,(Appeared in Drettakis, G., Max, N. (eds.), Rendering Techniques '98. Proceedings of the Eurographics Workshop in Vienna, Austria, June 1998
R Raskar, G Welch, Henry Fuchs
Fourth International Conference on Virtual Systems and Multimedia, Gifu, Japan. Nov 1998.
R Raskar, G Welch, H Fuchs
First International Workshop on Augmented Reality, San Francisco CA, Nov 1998.
H Fuchs, M Livingston, R Raskar and others
First International Conference on Medical Image Computing and Computer-Assisted Intervention 1998.
R Vujosevic, R Raskar, N Yetukuri
International Journal of Production Research, November 1995
Ninth International Congress of Cybernetics and Systems, New Delhi, January 1993.
Courses
Spring 2009 course: MAS 964 (H)
Special Topics in Media Technology:
Camera Culture (0-9-0) W 3:00-5:00pm in E15-135
Fall 2008 course: MAS 963 (H)
Computational Camera and Photography (6-3-3)
Fridays 1:30-4:30pm in E15-283A (Roth room)
Spring 2008 course: MAS 964 (H)
Special Topics in Media Technology:
Camera Culture (0-9-0) W 10:30-12:30pm in E15-235
Elsewhere
Advanced Topics in Computational Photography
R Raskar, J Tumblin
Siggraph 2008 [ Course Notes ]
Gradient Domain Techniques
A Agrawal, R Raskar
ICCV 2007, CVPR 2008 [ Course Notes ]
Computational Photography
R Raskar, J Tumblin
Siggraph 2006 and 2007 [ Course Notes ]
Spatial Augmented Reality
O Bimber, R Raskar
Siggraph 2006 [ Course Notes ]
Computational Photography
R Raskar, J Tumblin
Siggraph 2005 [ Course Notes ]
Spatial Augmented Reality: A Modern Approach to Augmented Reality
O Bimber, R Raskar
Siggraph 2005 [ Course Notes ]
Projectors: Advanced Graphics and Vision Techniques
R Raskar
Siggraph 2004 [ Course Notes in PDF ]
Modern Approaches to Augmented Reality
O Bimber and R Raskar
Eurographics 2004 [ Course Notes in PDF ]
Alternative Augmented Reality Approaches: Concepts, Techniques, and Applications
O Bimber and R Raskar
Eurographics 2003 [ Course Notes in PDF ]
Projectors: Advanced Geometric Issues in Applications
R Raskar
Siggraph 2003 [ Course Notes in PDF ]
Projector-based Graphics
R Raskar, E Lantz
Siggraph 2002 [ Course Notes in PDF ]
Patents and Applications
Issued
(34+)
1 7,218,792 Stylized
imaging using variable controlled illumination
2 7,206,449 Detecting silhouette edges in images
3 7,154,395 Interactive wireless tag location and identification
system
4 7,137,707 Projector-camera system with laser pointers
5 7,103,227 Enhancing low quality images of naturally illuminated
scenes
6 7,102,638 Reducing texture details in images
7 7,068,274 System and method for animating real objects with
projected images
8 7,038,185 Camera for directly generating a gradient image
9 7,019,748 Simulating motion of static objects in scenes
10 7,001,023 Method and system for calibrating projectors to
arbitrarily shaped surfaces with discrete optical sensors mounted at
the surfaces
11 6,930,681 System and method for registering multiple images
with three-dimensional objects
12 6,834,965 Self-configurable ad-hoc projector cluster
13 6,811,264 Geometrically aware projector
14 6,795,069 Free-form modeling of objects with variational
implicit surfaces
15 6,793,350 Projecting warped images onto curved surfaces
16 6,781,591 Blending multiple images using local and global
information
17 6,764,185 Projector as an input and output device
18 6,755,537 Method for globally aligning multiple projected
images
19 6,741,248 Rendering geometric features of scenes and models by
individual polygons
20 6,733,138 Multi-projector mosaic with automatic
registration
21 6,729,733 Method for determining a largest inscribed
rectangular image within a union of projected quadrilateral
images
22 6,715,888 Method and system for displaying images on curved
surfaces
23 6,709,116 Shape-adaptive projector system
24 6,677,956 Method for cross-fading intensities of multiple
images of a scene for seamless reconstruction
25 6,527,395 Method for calibrating a projector with a
camera
26 6,520,647 Automatic keystone correction for projectors with
arbitrary orientation
Public
(USPTO link for before last 18 months, 45+)
Pending ( 12+)
Press
Books
Computational
Photography: Mastering New Techniques for Lenses, Lighting, and Sensors
Presentations and Talks
Keynote Addresses & Invited Presentations
ProCams October 2003, Nice,
France, 'The Large Display Dilemma: Projectors are Dead' (Slides)
Symposium
on Virtual Reality October 2004, Sao Paulo, Brazil, (Slides)
Graphics Interface
May 2005, Victoria, Canada, 'Spatial Augmented Reality: New Problems in
Human Interface' (Slides
[17MB])
Ciencia and Arte
April 2005, Caxias do Sul, Brazil, (Slides)
Flat Information Display November 2005, San
Francisco, CA, 'Pocket Projector: All-in-one Solution for Mobility,
Augmentation, Communication' (Slides)
Active RFID Summit
November 2005, San Antonio, TX, 'R F I G : RFID + Geometry' (Slides)
Imagina
February 2006, Monaco (Slides)
International Symposium on Mixed and
Augmented Reality October 2006, San Diego, 'The Poor Man's Palace:
Special Effects in the Real World' (Slides)
Vision
Modeling and Visualization Conference, November 2007, 'Coded
Imaging', (Slides)
Asian
Conference on
Computer Vision, November 2007, 'Less is More: Coded Computational
Photography', (Slides)
Spring Conference on
Computer Graphics, April 2008, 'Can Computer Graphics Impact Our
Daily Life'
Universities and Research Laboratories
2008
U of Sao Paulo, Brazil (October, Host:
Romero Tori, Carlos Morimoto)
Ecole Normale Supérieure, Paris, France (June, Host: Jean Ponce)
ETH Zurich, Switzerland (February, Host: Marc Pollefeys)
Georgia Tech, Atlanta, Georgia (March, Host: Irfan Essa)
2007
Microsoft Research, Seattle (May, Host: Sing Bing Kang) Video
MIT Media Lab, Cambridge (June) Video
U of Pennsylvania, Philadelphia (Oct, Host: Jianbo Shi and Kostas Daniilidis)
U of Delaware, Newark (Oct, Host: Jingyi Yu)
Boston U., Boston (Nov, Host: Stan Sclaroff)
2006
Technical University Graz, Graz, Austria (May, Host: Dieter
Schmalstieg)
New York University (NYU), New York City (March, Host: Prof Ken Perlin)
2005
Harvard University, Cambridge, MA (February, Host: Ramakrishna C)
Osaka University, Osaka, Japan (August, Host: Prof Yoshifumi Kitamura)
Trinity College, Dublin, Ireland (April, Host. Prof. Carol O'Sullivan)
University of Illinois at Urbana Champaign, IL (March, Host. Prof.
Yizhou Yu and Narendra Ahuja)
Northwestern University, Chicago, IL (March, Host: Prof. Bruce Gooch
and Jack Tumblin)
Purdue University, IN (March, Host: Prof. Dan Aliaga)
SENAC, Sao Paulo (April, Host: Prof. Romero Tori)
University of Kentucky, Lexington, KY (November, Computer Vision for
Interactive Intelligent Environments Link)
2004
University of Maryland at College Park, MD (Host: Prof Amitabh
Varshaney)
Brown University, Providence, RI (Host. Prof. Gabriel Taubin)
North Eastern University, Boston, MA (Host: Prof. Ravi Sundaram)
Georgia Institute of Technology, Atlanta, GA (Host: Prof. Irfan Essa)
University of North Carolina at Chapel Hill, NC (Host: Prof. Greg
Welch)
Indian Institute of Technology, Mumbai, India (Host: Prof. Sharat
Chandran)
International Institute of Information Technology, Hyderabad (Host:
Prof. P. J. Narayanan)
U of Washington, Seattle, WA (Host: Prof. Steve Seitz)
Woods Hole Oceanographic Institution , Cape Cod, MA (Host: Prof.
Hanumant Singh)
2003
Stanford University, CA (Host: Prof Terry Winograd ) Video
University of California at Berkeley, Berkeley (Host: Prof Carlo H.
Séquin)
Honda Labs, Mountain View CA (Host: James Davis)
SRI International, Menlo Park, CA (Host: Motilal Agrawal)
nVidia , Santa Clara HQ, CA (Host: Wei-chao Chen)
University of California at Irvine, Irvine (Host: Prof Gopi
Meenakshisundaram)
Siggraph Boston Chapter, Boston
2002
Siggraph Boston Chapter, Boston
2001
MIT Media Lab, Cambridge
Siggraph Boston Chapter, Boston
U of Tokyo, Tokyo (Host: Prof Tomoyuki Nishita)
Sony Computer Science Lab, Tokyo (Host: Frank Nielsen)
Mitsubishi Electric VIL (UK Research Lab), London (Host: Miroslaw
Bober)
2000
IBM Research, T. J. Watson, New York (Host: Gabriel Taubin)
AT&T Labs, New Jersey (Host: Shankar Krishnan)
Bell Labs, Murray Hill, New Jersey (Host: Dan Aliaga and Ingrid
Carlbom)
Stanford University, Palo Alto, California (Host: Prof Pat Hanrahan and
Prof Marc Levoy)
U of California at Berkeley, California
Intel Research, Santa Clara, California (Host: Jean-Yves Bouguet)
Princeton University, New Jersey
1999
Brown University, Providence, Rhode Island
1998
California Institute of Technology (CalTech), Pasadena, California
(Host: Prof Peter Schröder)
U of Utah, Salt Lake City, Utah
Panels
Symposium on Computational
Photography and Video at MIT, What Will a Camera Look Like in 20 Years ? May
2005 (Organizer)
Siggraph 2005, The
Ultimate Display: What Will It Be?, August 2005
Papers Committee:
Siggraph
2007, Siggraph 2008
Papers Committee: Eurographics ( EG)
2005, Eurographics 2008, Eurographics 2009
Area Chair: CVPR 2009
Program Committee: International Conference on Computer
Vision (IEEE ICCV)
2005,
2007, 2008
Program Committee: Computer Vision and Pattern Recognition (IEEE CVPR) 2005,
2006, 2007, 2008
Program Committee: European Conference on Computer Vision (ECCV) 2006, 2008
Program Committee: Computer Graphics International ( CGI) 2005
Program Committee: Computer Vision and Pattern Recognition (IEEE CVPR) 2004,
2005, 2006
Program Committee: Non-photorealistic Animation and Rendering (ACM NPAR) 2004, 2006
Program Committee: High Performance Computing - Virtual Reality (HPC)
2004
Program Committee: Sketch-based
Interfaces (with Eurographics) 2004
Judging panel member for Global Indus Technovator Awards at MIT, 2004
Reviewer: SIGGRAPH(1999-2004-), Eurographics(2001-2003), EGRW/EGRS,
I3D(2001,2003), IEEE Visualization(1999,2000,2002,2003), ISMAR (Mixed
and Augmented Reality, 2003,2004), ACM Multimedia, NPAR (2002,2004),
UIST(2001,2002), ACM TOG, IEEE CG&A, IEEE TVCG
- Mitsubishi Electric Valuable Invention Award, 2006 (For
Photosensing Tags, RFIG)
- Mitsubishi Electric Valuable Invention Award, 2006 (For Dome Screen Projector Display)
- Dome Screen Projector for prestigious 'Best 10 New Products' Prize in Japan given by Nikkan Kogyo Simbun
- Technology Review's 100 Top Young Innovators Under 35 (TR100),
2004 (More links)
- Mitsubishi Electric Valuable Invention Award, 2004
- MIT Global Indus Technovator Awards (Top
20 Indian Researcher), 2003
- Mitsubishi Electric Information Technology R&D Award,
2003
- Link
Fellowship Award, 1998-99
- The undergraduate degree project in robotics awarded "Best
Project of the University" prize by IEEE chapter of Pune Institute of
Computer Technology , 1991
- Best student paper, "Incremental Curve Generation" at
state level technical paper contest, 1991
- Ranked First among 500,000 students appearing for entrance
exam (HSC Merit List) for the bachelor's program in the state of
Maharashtra, India, 1987
- Selected among TOP 20 to represent India at the
International Mathematics Olympiad, 1987
2007
Prakash: Lighting Aware Motion Capture for Dynamic Virtual Sets,
Emerging
Tech, SIGGRAPH, Aug 2007, San Diego
2006
Instant Replay Air Hockey , Emerging
Tech, SIGGRAPH, Aug 2006, Boston
2005
Contours with Multi-flash , CVPR 2005, June 2005,
San Diego
2004
Camera Non-photo , COLLISION Senses Show,
Nov
2004, Cambridge
Stylized Images with NPR Camera , Emerging
Tech, SIGGRAPH, Aug 2004, Los Angeles
Interacting With Projections Using
iLamp Projectors , Emerging
Tech, SIGGRAPH, Aug 2004, Los Angeles
2001
Contra Inertia , superCOLLISION Show,
Oct 2001, Cambridge