Research talk - Detecting image forgeries and deepfakes with positional learning - Quentin Bammey
Event details
| Date | 25.07.2024 |
| Hour | 10:15 › 11:15 |
| Speaker | Quentin Bammey |
| Location | |
| Category | Conferences - Seminars |
| Event Language | English |
The IVRL lab is hosting Quentin Bammey (Centre Borelli, École Normale Supérieure Paris-Saclay, Université Paris-Saclay, CNRS), who will give a talk about "Detecting image forgeries and deepfakes with positional learning" on Thursday July 25th at 10.15am in BC420.
Abstract
It is nowadays incredibly easy, even for laymen, to edit images or generate them from scratch in a visually realistic way. However, this ease of manipulation has given rise to the malicious manipulation of images, resulting in the creation and dissemination of realistic but fake content to spread disinformation online, wrongfully incriminate someone, or commit fraud. The detection of such altered images is paramount in exposing those deceitful acts. One approach involves reverse-engineering the image signal processing pipeline or the generation pipeline, to detect and localize inconsistencies. However, many of these steps require analysing information with a strong frequency component: demosaicing, JPEG compression, super-resolution, … Due to their translation invariance, convolutional neural networks struggle to analyse these components. Yet, it is also almost impossible to mathematically model the traces left in an image by the image processing pipeline, due to the sheer variety of processing and post-processing algorithms for each step.
In this context, positional learning emerges as a promising approach to reveal these underlying components in images and detect their inconsistencies. By training a CNN to estimate positional information on each pixel (such as its position modulo n), it implicitly learn to rely on the underlying components of the images that lead to this information. At inference, correct outputs of the CNN thus act as proof that the underlying component is present in an image, while locally incorrect regions are clues that the underlying traces were locally disturbed.
In this talk, we will show how positional learning can be used to reveal forgeries from the image mosaic, or detect generated images from their super-resolution decoding artefacts. We will present how an a contrario layer can be used to make automatic detections of fake images while limiting the tolerated rate of false positives.
Bio
Quentin Bammey (https://bammey.com/) is a researcher at Centre Borelli, École Normale Supérieure Paris-Saclay, Université Paris-Saclay, CNRS, in France. His main interests are image processing, computer vision, machine learning, multimedia forensics, and reproducible research.
He is heavily invested in the Image Processing On Line (IPOL, https://www.ipol.im/) journal and demo system for open science and reproducible research. In particular, he cofounded and now organizes the IPOL MLBriefs workshop (https://mlbriefs.com/) to facilitate reproducible research in algorithms.
He is also involved in the coordination and development of the BrevetAI platform, to offer a learning-by-doing training on artificial intelligence and disseminate knowledge about AI to the public, as part of the SaclAI-School training program of the Université Paris-Saclay and DataIA Institute.
Abstract
It is nowadays incredibly easy, even for laymen, to edit images or generate them from scratch in a visually realistic way. However, this ease of manipulation has given rise to the malicious manipulation of images, resulting in the creation and dissemination of realistic but fake content to spread disinformation online, wrongfully incriminate someone, or commit fraud. The detection of such altered images is paramount in exposing those deceitful acts. One approach involves reverse-engineering the image signal processing pipeline or the generation pipeline, to detect and localize inconsistencies. However, many of these steps require analysing information with a strong frequency component: demosaicing, JPEG compression, super-resolution, … Due to their translation invariance, convolutional neural networks struggle to analyse these components. Yet, it is also almost impossible to mathematically model the traces left in an image by the image processing pipeline, due to the sheer variety of processing and post-processing algorithms for each step.
In this context, positional learning emerges as a promising approach to reveal these underlying components in images and detect their inconsistencies. By training a CNN to estimate positional information on each pixel (such as its position modulo n), it implicitly learn to rely on the underlying components of the images that lead to this information. At inference, correct outputs of the CNN thus act as proof that the underlying component is present in an image, while locally incorrect regions are clues that the underlying traces were locally disturbed.
In this talk, we will show how positional learning can be used to reveal forgeries from the image mosaic, or detect generated images from their super-resolution decoding artefacts. We will present how an a contrario layer can be used to make automatic detections of fake images while limiting the tolerated rate of false positives.
Bio
Quentin Bammey (https://bammey.com/) is a researcher at Centre Borelli, École Normale Supérieure Paris-Saclay, Université Paris-Saclay, CNRS, in France. His main interests are image processing, computer vision, machine learning, multimedia forensics, and reproducible research.
He is heavily invested in the Image Processing On Line (IPOL, https://www.ipol.im/) journal and demo system for open science and reproducible research. In particular, he cofounded and now organizes the IPOL MLBriefs workshop (https://mlbriefs.com/) to facilitate reproducible research in algorithms.
He is also involved in the coordination and development of the BrevetAI platform, to offer a learning-by-doing training on artificial intelligence and disseminate knowledge about AI to the public, as part of the SaclAI-School training program of the Université Paris-Saclay and DataIA Institute.
Practical information
- Informed public
- Free
Organizer
Contact
- Sabine Süsstrunk, Martin Nicolas Everaert