Imaging Seminar: Normalizing Flows and the Power of Patches in Inverse Problems
Event details
| Date | 28.09.2023 |
| Hour | 17:00 › 18:00 |
| Speaker | Prof. Gabriele Steidl |
| Location | |
| Category | Conferences - Seminars |
| Event Language | English |
<< Registration below >>
Abstract:
Learning neural networks using only a small amount of data is an important ongoing research topic with tremendous potential for applications. We introduce a regularizer for the variational modeling of inverse problems in imaging based on normalizing flows, called patchNR. It involves a normalizing flow learned on patches of very few images. The subsequent reconstruction method is completely unsupervised and the same regularizer can be used for different forward operators acting on the same class of images. By investigating the distribution of patches versus those of the whole image class, we prove that our variational model is indeed a MAP approach. Numerical examples for low-dose CT, limited-angle CT and superresolution of material images demonstrate that our method provides high quality results among unsupervised methods, but requires only very few data. Further, the appoach also works if only the low resolution image is available.
In the second part of the talk I will generalize normalizing flows to stochastic normalizing flows to improve their expressivity. Normalizing flows, diffusion normalizing flows and variational autoencoders are powerful generative models. A unified framework to handle these approaches are Markov chains. We consider stochastic normalizing flows as a pair of Markov chains fulfilling some properties and show how many state-of-the-art models for data generation fit into this framework. Indeed including stochastic layers improves the expressivity of the network and allows for generating multimodal distributions from unimodal ones. The Markov chains point of view enables us to couple both deterministic layers as invertible neural networks and stochastic layers as Metropolis-Hasting layers, Langevin layers, variational autoencoders and diffusion normalizing flows in a mathematically sound way. Our framework establishes a useful mathematical tool to combine the various approaches.
Joint work with F. Altekr¸ger, P. Hagemann, J. Hertrich, P. Maass.
Biography:
Gabriele Steidl received her PhD and habilitation in mathematics from the University of Rostock. After positions as associated professor for Mathematics at the TU Darmstadt and full professor at the University of Mannheim and TU Kaiserslautern, she is currently professor at the TU Berlin. She was a Postdoc, resp. visiting Professor at the Universities of Debrecen, Z¸rich, ENS Cachan/Paris Univ. Paris Est, Sorbonne/IHP Paris and worked as consultant of the Fraunhofer ITWM Kaiserslautern. Gabriele Steidl is Editor-in-Chief of the SIAM Journal of Imaging Sciences and SIAM Fellow. Her research interests include Harmonic Analysis, Optimization, Inverse Problem and Machine Learning with applications in Image Processing.
The talk is followed by an aperitif.
Registration appreciated
More info here
Abstract:
Learning neural networks using only a small amount of data is an important ongoing research topic with tremendous potential for applications. We introduce a regularizer for the variational modeling of inverse problems in imaging based on normalizing flows, called patchNR. It involves a normalizing flow learned on patches of very few images. The subsequent reconstruction method is completely unsupervised and the same regularizer can be used for different forward operators acting on the same class of images. By investigating the distribution of patches versus those of the whole image class, we prove that our variational model is indeed a MAP approach. Numerical examples for low-dose CT, limited-angle CT and superresolution of material images demonstrate that our method provides high quality results among unsupervised methods, but requires only very few data. Further, the appoach also works if only the low resolution image is available.
In the second part of the talk I will generalize normalizing flows to stochastic normalizing flows to improve their expressivity. Normalizing flows, diffusion normalizing flows and variational autoencoders are powerful generative models. A unified framework to handle these approaches are Markov chains. We consider stochastic normalizing flows as a pair of Markov chains fulfilling some properties and show how many state-of-the-art models for data generation fit into this framework. Indeed including stochastic layers improves the expressivity of the network and allows for generating multimodal distributions from unimodal ones. The Markov chains point of view enables us to couple both deterministic layers as invertible neural networks and stochastic layers as Metropolis-Hasting layers, Langevin layers, variational autoencoders and diffusion normalizing flows in a mathematically sound way. Our framework establishes a useful mathematical tool to combine the various approaches.
Joint work with F. Altekr¸ger, P. Hagemann, J. Hertrich, P. Maass.
Biography:
Gabriele Steidl received her PhD and habilitation in mathematics from the University of Rostock. After positions as associated professor for Mathematics at the TU Darmstadt and full professor at the University of Mannheim and TU Kaiserslautern, she is currently professor at the TU Berlin. She was a Postdoc, resp. visiting Professor at the Universities of Debrecen, Z¸rich, ENS Cachan/Paris Univ. Paris Est, Sorbonne/IHP Paris and worked as consultant of the Fraunhofer ITWM Kaiserslautern. Gabriele Steidl is Editor-in-Chief of the SIAM Journal of Imaging Sciences and SIAM Fellow. Her research interests include Harmonic Analysis, Optimization, Inverse Problem and Machine Learning with applications in Image Processing.
The talk is followed by an aperitif.
Registration appreciated
More info here
Practical information
- General public
- Free
Organizer
- EPFL Center for Imaging