Synthesis of environment maps for mixed reality

David R. Walton, Diego Thomas, Anthony Steed, Akihiro Sugimoto

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Citations (Scopus)

Abstract

When rendering virtual objects in a mixed reality application, it is helpful to have access to an environment map that captures the appearance of the scene from the perspective of the virtual object. It is straightforward to render virtual objects into such maps, but capturing and correctly rendering the real components of the scene into the map is much more challenging. This information is often recovered from physical light probes, such as reflective spheres or fisheye cameras, placed at the location of the virtual object in the scene. For many application areas, however, real light probes would be intrusive or impractical. Ideally, all of the information necessary to produce detailed environment maps could be captured using a single device. We introduce a method using an RGBD camera and a small fisheye camera, contained in a single unit, to create environment maps at any location in an indoor scene. The method combines the output from both cameras to correct for their limited field of view and the displacement from the virtual object, producing complete environment maps suitable for rendering the virtual content in real time. Our method improves on previous probeless approaches by its ability to recover high-frequency environment maps. We demonstrate how this can be used to render virtual objects which shadow, reflect and refract their environment convincingly.

Original languageEnglish
Title of host publicationProceedings of the 2017 IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2017
EditorsWolfgang Broll, Holger Regenbrecht, J Edward Swan
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages72-81
Number of pages10
ISBN (Electronic)9781538629437
DOIs
Publication statusPublished - Nov 20 2017
Event16th IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2017 - Nantes, France
Duration: Oct 9 2017Oct 13 2017

Publication series

NameProceedings of the 2017 IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2017

Other

Other16th IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2017
CountryFrance
CityNantes
Period10/9/1710/13/17

All Science Journal Classification (ASJC) codes

  • Computer Graphics and Computer-Aided Design
  • Media Technology
  • Modelling and Simulation

Fingerprint Dive into the research topics of 'Synthesis of environment maps for mixed reality'. Together they form a unique fingerprint.

Cite this