Interaction Learning with Neural Cellular Automata
Event details
| Date | 05.07.2024 |
| Hour | 15:00 › 17:00 |
| Speaker | Yitao Xu |
| Location | |
| Category | Conferences - Seminars |
EDIC candidacy exam
Exam president: Prof. Caglar Gulcehre
Thesis advisor: Prof. Sabine Süsstrunk
Co-examiner: Prof. Mathieu Salzmann
Abstract
Neural Cellular Automata (NCA) models the global cell representation via local cell interactions. Cells in NCA are represented by real vectors, allowing various interaction modeling techniques to be adopted. Existing works demonstrate that common operations in deep learning, such as convolution on 2D images, message passing on graphs, and self-attention operation can be applied to model cell interactions in NCA. While promising, the application scenarios of current NCA are mainly on exemplar-based training, discriminative tasks on small datasets, and low-resolution image generation. The limited size of models and datasets hinders further real-world applications of NCA on large-scale problems.
In this thesis, we will first formulate the evolution paradigm of NCA by introducing the synergistic combination of cell Interaction and Adaptation in NCA evolution. The focus will be on the Interaction stage. We will introduce the most commonly used Interaction modeling method, i.e., 2D convolution. Then, we will elaborate on the modification of 2D convolution to allow more flexible cell Interaction learning, which results in message passing on graph data. Finally, a cell-dependent interaction scheme is introduced, where each cell decides its unique interaction strategy via self-attention.
Based on the analysis of NCA paradigm, we will propose the potential practical application scenarios on those three interaction modeling methods, and elaborate on the 2D convolution scheme as the first step of extending NCA into real-world applications.
Background papers
Exam president: Prof. Caglar Gulcehre
Thesis advisor: Prof. Sabine Süsstrunk
Co-examiner: Prof. Mathieu Salzmann
Abstract
Neural Cellular Automata (NCA) models the global cell representation via local cell interactions. Cells in NCA are represented by real vectors, allowing various interaction modeling techniques to be adopted. Existing works demonstrate that common operations in deep learning, such as convolution on 2D images, message passing on graphs, and self-attention operation can be applied to model cell interactions in NCA. While promising, the application scenarios of current NCA are mainly on exemplar-based training, discriminative tasks on small datasets, and low-resolution image generation. The limited size of models and datasets hinders further real-world applications of NCA on large-scale problems.
In this thesis, we will first formulate the evolution paradigm of NCA by introducing the synergistic combination of cell Interaction and Adaptation in NCA evolution. The focus will be on the Interaction stage. We will introduce the most commonly used Interaction modeling method, i.e., 2D convolution. Then, we will elaborate on the modification of 2D convolution to allow more flexible cell Interaction learning, which results in message passing on graph data. Finally, a cell-dependent interaction scheme is introduced, where each cell decides its unique interaction strategy via self-attention.
Based on the analysis of NCA paradigm, we will propose the potential practical application scenarios on those three interaction modeling methods, and elaborate on the 2D convolution scheme as the first step of extending NCA into real-world applications.
Background papers
- Mordvintsev A, Randazzo E, Niklasson E, et al. Growing neural cellular automata[J]. Distill, 2020, 5(2): e23. Link: https://distill.pub/2020/growing-ca/?ref=https://githubhelp.com
- Grattarola D, Livi L, Alippi C. Learning graph cellular automata[J]. Advances in Neural Information Processing Systems, 2021, 34: 20983-20994. Link: https://proceedings.neurips.cc/paper/2021/file/af87f7cdcda223c41c3f3ef05a3aaeea-Paper.pdf
- Tesfaldet M, Nowrouzezahrai D, Pal C. Attention-based neural cellular automata[J]. Advances in Neural Information Processing Systems, 2022, 35: 8174-8186. Link: https://proceedings.neurips.cc/paper_files/paper/2022/file/361e5112d2eca09513bbd266e4b2d2be-Paper-Conference.pdf
Practical information
- General public
- Free