Aesthetics-oriented video generation and editing
Event details
| Date | 23.08.2022 |
| Hour | 10:00 › 12:00 |
| Speaker | Martin Nicolas EVERAERT |
| Location | |
| Category | Conferences - Seminars |
EDIC candidacy exam
Exam president: Prof. Tanja Käser
Thesis advisor: Prof. Sabine Süsstrunk
Thesis co-advisor: Dr Radhakrishna Achanta
Co-examiner: Dr Mathieu Salzmann
Abstract
The average time spent watching online videos increases every year, across all demographics. Videos are more engaging and are shared twice as much as other types of media. However, making or editing such videos can be expensive and time-consuming. Our research goal is to propose solutions based on machine learning and computational aesthetics to automate steps in the creation and editing of videos that are appealing and of interest for the viewer.
In this research proposal, we discuss three existing works and how they relate to our research. We first examine how generative adversarial networks (GANs) can be used to generate videos and what are their limitations.Then, we take a look at an example of data collection and annotation process, allowing training of models for video aesthetics and message understanding.
Finally, we discuss a framework to navigate GANs' latent space to improve aesthetics.
Background papers
[1] Temporal Shift GAN for Large Scale Video Generation, Munoz et al., WACV 2021, https://arxiv.org/abs/2004.01823
[2] GANalyze: Toward Visual Definitions of Cognitive Image Properties, Goetschalckx et al., ICCV 2019, https://arxiv.org/abs/1906.10112
[3] Automatic Understanding of Image and Video Advertisements, Hussain et al., CVPR 2017, https://arxiv.org/abs/1707.03067
Exam president: Prof. Tanja Käser
Thesis advisor: Prof. Sabine Süsstrunk
Thesis co-advisor: Dr Radhakrishna Achanta
Co-examiner: Dr Mathieu Salzmann
Abstract
The average time spent watching online videos increases every year, across all demographics. Videos are more engaging and are shared twice as much as other types of media. However, making or editing such videos can be expensive and time-consuming. Our research goal is to propose solutions based on machine learning and computational aesthetics to automate steps in the creation and editing of videos that are appealing and of interest for the viewer.
In this research proposal, we discuss three existing works and how they relate to our research. We first examine how generative adversarial networks (GANs) can be used to generate videos and what are their limitations.Then, we take a look at an example of data collection and annotation process, allowing training of models for video aesthetics and message understanding.
Finally, we discuss a framework to navigate GANs' latent space to improve aesthetics.
Background papers
[1] Temporal Shift GAN for Large Scale Video Generation, Munoz et al., WACV 2021, https://arxiv.org/abs/2004.01823
[2] GANalyze: Toward Visual Definitions of Cognitive Image Properties, Goetschalckx et al., ICCV 2019, https://arxiv.org/abs/1906.10112
[3] Automatic Understanding of Image and Video Advertisements, Hussain et al., CVPR 2017, https://arxiv.org/abs/1707.03067
Practical information
- General public
- Free