Imaging Seminar: Model-Based Machine Learning for Inverse Problems in Imaging
Event details
| Date | 18.09.2024 |
| Hour | 17:00 › 18:00 |
| Speaker | Prof. Pier Luigi Dragotti, Imperial College London |
| Location | |
| Category | Conferences - Seminars |
| Event Language | English |
<< Registration below >>
Abstract:
Deep Neural Networks are currently able to achieve state-of-the-art performance in many imaging tasks. In this talk, we argue that in inverse imaging problems efficient deep neural networks with more predictable performances can only be achieved by combining model-based solvers with learned models. There is plenty of evidence for this and typical examples where this integration has had an impact include the plug-and-play framework and the network unfolding strategy.
In the first part of the talk, we propose to connect wavelet theory and in particular the lifting scheme to the design of invertible neural networks (INN). We focus on image denoising and present a training strategy and an architecture for the INNs that mimics the multi-resolution property of the wavelet transform as well as the ability of the transform to provide a sparse representation of images. We show that the proposed INN achieves state-of-the-art performance for image denoising and generalizes well to images with unseen noise levels. Next, we introduce INDigo, a novel INN-guided probabilistic diffusion algorithm for arbitrary image restoration tasks. INDigo combines the perfect reconstruction property of INNs with the strong generative capabilities of pre-trained diffusion models. Specifically, we leverage the invertibility of the networks to condition the diffusion process and in this way we generate high quality restored images consistent with the measurements.
In the second part of the talk, we discuss the unfolding techniques which is an approach that allows embedding priors and models in the neural network architecture. In this context we discuss the problem of monitoring the dynamics of large populations of neurons over a large area of the brain. Light-field microscopy (LFM), a type of scanless microscopy, is a particularly attractive candidate for high-speed three-dimensional (3D) imaging which is needed for monitoring neural activity. We review fundamental aspects of LFM and then present computational methods for neuron localization and activity estimation from light-field data. Our approach is based on machine learning and leverages the intrinsic characteristics of neuronal signals and the physics of the acquisition process.
Finally, we discuss the role of sampling theory in inverse problems and discuss the problem of reconstructing shapes from tomographic projections at unknown orientation. This problem arises in many applications and in particular in Cryogenic Electron Microscopy (CryoEM). We show how sampling theory can inspire the design of proper neural networks for this application.
This is joint work with A. Foust, P. Song, C. Howe, H. Verinaz, J. Huang, D. You, Y. Su, V. Leung and D. Bubeck from Imperial College London.
Bio:
Pier Luigi Dragotti is Professor of Signal Processing in the Electrical and Electronic Engineering Department at Imperial College London and a Fellow of the IEEE. He received the Masters Degree (summa cum laude) from the University Federico II, Naples, Italy, in 1997 and PhD degree from the Swiss Federal Institute of Technology of Lausanne (EPFL), Switzerland in 2002. He has held several visiting positions. In particular, he was a visiting student at Stanford University, Stanford, CA in 1996, a summer researcher in the Mathematics of Communications Department at Bell Labs, Lucent Technologies, Murray Hill, NJ in 2000, a visiting scientist at Massachusetts Institute of Technology (MIT) in 2011.
Dragotti was Editor-in-Chief of the IEEE Transactions on Signal Processing (2018-2020), Technical Co-Chair for the European Signal Processing Conference in 2012 and Associate Editor of the IEEE Transactions on Image Processing from 2006 to 2009. He was a SPS Distinguished Lecturer in 2021-22. He was also an Elected Member of the IEEE Image, Video and Multidimensional Signal Processing Technical Committee, IEEE Signal Processing Theory and Methods Technical Committee and of the IEEE Computational Imaging Technical Committee. In 2011 he was awarded the prestigious ERC starting investigator award (consolidator stream).
His research interests include sampling theory and its applications, computational imaging and model-based machine learning.
Registration appreciated
More info here
Abstract:
Deep Neural Networks are currently able to achieve state-of-the-art performance in many imaging tasks. In this talk, we argue that in inverse imaging problems efficient deep neural networks with more predictable performances can only be achieved by combining model-based solvers with learned models. There is plenty of evidence for this and typical examples where this integration has had an impact include the plug-and-play framework and the network unfolding strategy.
In the first part of the talk, we propose to connect wavelet theory and in particular the lifting scheme to the design of invertible neural networks (INN). We focus on image denoising and present a training strategy and an architecture for the INNs that mimics the multi-resolution property of the wavelet transform as well as the ability of the transform to provide a sparse representation of images. We show that the proposed INN achieves state-of-the-art performance for image denoising and generalizes well to images with unseen noise levels. Next, we introduce INDigo, a novel INN-guided probabilistic diffusion algorithm for arbitrary image restoration tasks. INDigo combines the perfect reconstruction property of INNs with the strong generative capabilities of pre-trained diffusion models. Specifically, we leverage the invertibility of the networks to condition the diffusion process and in this way we generate high quality restored images consistent with the measurements.
In the second part of the talk, we discuss the unfolding techniques which is an approach that allows embedding priors and models in the neural network architecture. In this context we discuss the problem of monitoring the dynamics of large populations of neurons over a large area of the brain. Light-field microscopy (LFM), a type of scanless microscopy, is a particularly attractive candidate for high-speed three-dimensional (3D) imaging which is needed for monitoring neural activity. We review fundamental aspects of LFM and then present computational methods for neuron localization and activity estimation from light-field data. Our approach is based on machine learning and leverages the intrinsic characteristics of neuronal signals and the physics of the acquisition process.
Finally, we discuss the role of sampling theory in inverse problems and discuss the problem of reconstructing shapes from tomographic projections at unknown orientation. This problem arises in many applications and in particular in Cryogenic Electron Microscopy (CryoEM). We show how sampling theory can inspire the design of proper neural networks for this application.
This is joint work with A. Foust, P. Song, C. Howe, H. Verinaz, J. Huang, D. You, Y. Su, V. Leung and D. Bubeck from Imperial College London.
Bio:
Pier Luigi Dragotti is Professor of Signal Processing in the Electrical and Electronic Engineering Department at Imperial College London and a Fellow of the IEEE. He received the Masters Degree (summa cum laude) from the University Federico II, Naples, Italy, in 1997 and PhD degree from the Swiss Federal Institute of Technology of Lausanne (EPFL), Switzerland in 2002. He has held several visiting positions. In particular, he was a visiting student at Stanford University, Stanford, CA in 1996, a summer researcher in the Mathematics of Communications Department at Bell Labs, Lucent Technologies, Murray Hill, NJ in 2000, a visiting scientist at Massachusetts Institute of Technology (MIT) in 2011.
Dragotti was Editor-in-Chief of the IEEE Transactions on Signal Processing (2018-2020), Technical Co-Chair for the European Signal Processing Conference in 2012 and Associate Editor of the IEEE Transactions on Image Processing from 2006 to 2009. He was a SPS Distinguished Lecturer in 2021-22. He was also an Elected Member of the IEEE Image, Video and Multidimensional Signal Processing Technical Committee, IEEE Signal Processing Theory and Methods Technical Committee and of the IEEE Computational Imaging Technical Committee. In 2011 he was awarded the prestigious ERC starting investigator award (consolidator stream).
His research interests include sampling theory and its applications, computational imaging and model-based machine learning.
Registration appreciated
More info here
Practical information
- General public
- Free
Organizer
- EPFL Center for Imaging