Monocular Depth Estimation via Style Transfer
Event details
| Date | 26.02.2020 |
| Hour | 13:00 › 15:00 |
| Speaker | Deblina Bhattacharjee |
| Location | |
| Category | Conferences - Seminars |
EDIC candidacy exam
Exam president: Prof Robert West
Thesis Advisor: Prof Pascal Fua
Thesis co-advisor: Dr. Mathieu Salzmann
Co-examiner: Prof. Pierre Dillenbourg
Abstract
Estimating depth for comic images is more aptly a monocular depth estimation problem and is subject to the availability of ground truth data while using learning-based approaches. However, most monocular depth estimators either need to rely on large amounts of ground truth depth data, which is extremely expensive and difficult to obtain, or predict disparity as an intermediary step using a secondary supervisory signal leading to blurring and other artefacts. Training a learning-based depth estimation model using synthetic pseudo ground truth data can resolve some of these issues but introduces domain bias. To remedy the problem of domain bias, in this report, we have applied style transfer and adversarial learning via GANs. We also apply a depth estimator simultaneously on the style-transferred images to reduce memory footprint and time overhead. Therefore, our model estimates pixel-level depth from a single comics image based on its training over a large amount of readily available real-world data and its corresponding depth ground truth. The experimental results and qualitative analysis show the achieved accuracy and robustness of this method when compared to pre-existing techniques. Whilst the results have scope for improvement, our preliminary study shows that this approach can lead to an accurate pixel-level depth estimation.
Background papers
Towards Instance level Image-to-ImageTranslation, by Shen, Z, et al.
Cross-Domain Weakly Supervised Object Detection through Progressive Domain Adaptation, by Inoue, N, et al.
Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer, by Ranftl, R., et al.
Exam president: Prof Robert West
Thesis Advisor: Prof Pascal Fua
Thesis co-advisor: Dr. Mathieu Salzmann
Co-examiner: Prof. Pierre Dillenbourg
Abstract
Estimating depth for comic images is more aptly a monocular depth estimation problem and is subject to the availability of ground truth data while using learning-based approaches. However, most monocular depth estimators either need to rely on large amounts of ground truth depth data, which is extremely expensive and difficult to obtain, or predict disparity as an intermediary step using a secondary supervisory signal leading to blurring and other artefacts. Training a learning-based depth estimation model using synthetic pseudo ground truth data can resolve some of these issues but introduces domain bias. To remedy the problem of domain bias, in this report, we have applied style transfer and adversarial learning via GANs. We also apply a depth estimator simultaneously on the style-transferred images to reduce memory footprint and time overhead. Therefore, our model estimates pixel-level depth from a single comics image based on its training over a large amount of readily available real-world data and its corresponding depth ground truth. The experimental results and qualitative analysis show the achieved accuracy and robustness of this method when compared to pre-existing techniques. Whilst the results have scope for improvement, our preliminary study shows that this approach can lead to an accurate pixel-level depth estimation.
Background papers
Towards Instance level Image-to-ImageTranslation, by Shen, Z, et al.
Cross-Domain Weakly Supervised Object Detection through Progressive Domain Adaptation, by Inoue, N, et al.
Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer, by Ranftl, R., et al.
Practical information
- General public
- Free