Multi-view Neural Surface Reconstruction with Learned Regularizations

Thumbnail

Event details

Date 30.06.2023
Hour 09:3011:30
Speaker Aoxiang Fan
Location
Category Conferences - Seminars
EDIC candidacy exam
Exam president: Prof. Sabine Süsstrunk
Thesis advisor: Prof. Pascal Fua
Co-examiner: Prof. Mark Pauly

Abstract
Recent advances in Multi-view Surface Reconstruction show that a 3D scene representation and learning framework built upon volume rendering and Neural Radiance Fields (NeRF) can significantly improve the reconstruction performance. This fact encourages people to extend the approach to include more sophisticated physically-based formulations. On the other aspect, following the conventional Structure-from-Motion (SfM) pipeline, it has also been shown that 3D geometry can be learned with a data-driven approach from 2D image observations regardless of the physically-based formulations.

Following a comprehensive and meticulous examination of the literature, our proposed research focuses on three aspects: 1) extending physically-based models in the context of inverse rendering to facilitate Multi-view Surface Reconstruction 2) studying the role of data-driven priors in physically-based formulations to make the ill-posed problem efficiently solvable; 3) investigating effective architectures and data-efficient training paradigms to learn better image representation for data-driven Multi-view Surface Reconstruction.

Background papers
a. NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction. Peng Wang et al, NeurIPS 2021, https://proceedings.neurips.cc/paper/2021/file/e41e164f7485ec4a28741a2d0ea41c74-Paper.pdf

b. TransMVSNet: Global Context-aware Multi-view Stereo Network with Transformers. Yikang Ding et al, CVPR 2022, http://openaccess.thecvf.com/content/CVPR2022/papers/Ding_TransMVSNet_Global_Context-Aware_Multi-View_Stereo_Network_With_Transformers_CVPR_2022_paper.pdf

c. VolRecon: Volume Rendering of Signed Ray Distance Functions for Generalizable Multi-View Reconstruction. Yufan Ren et al, CVPR 2023, https://arxiv.org/pdf/2212.08067 
 

Practical information

  • General public
  • Free

Contact

  • edic@epfl.ch

Tags

EDIC candidacy exam

Share