Scene Decomposition and Relighting from Image Collections in Neural Rendering
Event details
| Date | 23.08.2022 |
| Hour | 14:00 › 16:00 |
| Speaker | Dongqing Wang |
| Location | |
| Category | Conferences - Seminars |
EDIC candidacy exam
Exam president: Prof. Mark Pauly
Thesis advisor: Prof. Sabine Süsstrunk
Co-examiner: Prof. Ronan Boulic
Abstract
The focus of our research is to generate controllable photo-realistic images of real-world scenes from their existing sparse observations, i.e., the inverse rendering problem. The approaches we focus on are those through neural rendering, utilizing neural network to decompose the scene, learn its physical properties and render with novel lighting condition. In this proposal, we discuss three papers and how they relate to our research topic. We first look at NeRF's simple framework representing 3D scenes as volumetric radiance field for view synthesis; Then we look at NeRD which modifies NeRF to allow scene decomposition for illumination, geometry, surface reflectance, etc., for relighting; we lastly present PhySG using signed distance functions for scene geometry addressing drawback of previous methods. Finally, we discuss our proposed solution for the problem as well as possible future research focus.
Background papers
1. NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis.
2. NeRD: Neural Reflectance Decomposition from Image Collections
3. PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relighting.
Exam president: Prof. Mark Pauly
Thesis advisor: Prof. Sabine Süsstrunk
Co-examiner: Prof. Ronan Boulic
Abstract
The focus of our research is to generate controllable photo-realistic images of real-world scenes from their existing sparse observations, i.e., the inverse rendering problem. The approaches we focus on are those through neural rendering, utilizing neural network to decompose the scene, learn its physical properties and render with novel lighting condition. In this proposal, we discuss three papers and how they relate to our research topic. We first look at NeRF's simple framework representing 3D scenes as volumetric radiance field for view synthesis; Then we look at NeRD which modifies NeRF to allow scene decomposition for illumination, geometry, surface reflectance, etc., for relighting; we lastly present PhySG using signed distance functions for scene geometry addressing drawback of previous methods. Finally, we discuss our proposed solution for the problem as well as possible future research focus.
Background papers
1. NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis.
2. NeRD: Neural Reflectance Decomposition from Image Collections
3. PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relighting.
Practical information
- General public
- Free