Monocular 3D Shape Reconstruction Using Deep Neural Networks

EDIC candidacy exam
Exam president: Prof. Sabine Süsstrunk
Thesis advisor: Prof. Pascal Fua
Thesis co-advisor: Dr. Mathieu Salzmann
Co-examiner: Prof. Mark Pauly

Abstract
Monocular reconstruction of non-rigid surfaces is a long-standing computer vision problem which has so far been addressed by carefully selecting image features and explicitly imposing physical constraints. We attempt to tackle the problem by learning to predict the shape directly from data. To improve on the results obtained from straight-forward regressor we explore the domain of representation learning with focus on adversarial setting which might help produce more realistic shape predictions. Then, we discuss the approaches to estimate the scene lighting which would further help to reason about the shape using shading clues. Finally, we summarize preliminary results on both the synthetic and real dataset and we discuss future directions.

Background papers
Density estimation using real NVP, by L. Dinh et al ICLR, 2016.
On the relationship between radiance and irradiance: determining the illumination from images of a convex lambertian object, by R. Ramamoorthi and P. Hanrahan. JOSA A, 2001.
InfoGAN: Interpretable representation learning by information maximizing generative adversarial nets, by X. Chen et al. NIPS, 2016.