IEM Distinguished Lecturers Seminar: Kernel-driven and Learnable Self-Supervision over Graphs
*** NEW PLACE: BM 5202 ***
Coffee and cookies will be served at 13:00 before the seminar
Abstract
Self-supervision (SeSu) has gained popularity for “data-hungry” training of machine learning models, especially those involving large-scale graphs, where labeled samples are scarce or unavailable. Main learning tasks in such setups are ill-posed, and SeSu renders them well-posed by relying on abundant unlabeled data as input, to yield low-dimensional embeddings of a reference (auxiliary) model output. In this talk, we first outline SeSu approaches, specialized reference models, and their links with (variational) auto-encoders, regularization, semi-supervised, transfer, meta, and multi-view learning; but also their challenges and opportunities when multi-layer graph topologies and multi-view data are present, when nodal features are absent, and when the ad hoc selection of a reference model yields embeddings not optimally designed for the downstream main learning task. Next, we present our novel SeSu approach which selects the reference model to output either a prescribed kernel or a learnable weighted superposition of kernels from a prescribed dictionary. As a result, the learned embeddings offer a novel, reduced dimensionality estimate of the basis kernel, and thus an efficient parametric estimate of the main learning function at hand that belongs to a reproducing kernel Hilbert space. If time allows, we will also cover online variants for dynamic settings, and regret analysis founded on the so-termed neural-tangent-kernel framework to assess how effectively the learned embeddings approximate the underlying optimal kernel(s). We will wrap up with numerical tests using synthetic and real datasets to showcase the merits of kernel-driven and learnable (KeLe) SeSu relative to alternatives. The real data will also compare KeLe-SeSu with auto-encoders and graph neural networks (GNNs), and further test KeLe-SeSu on reference maps with masked inputs and predicted-outputs that are popular in large language models (LLMs).
Bio
Georgios B. GIANNAKIS is a Professor of Electrical and Computer Engineering at the University of Minnesota, where he holds a Presidential Chair. His interests span the areas of statistical learning, communications, and networking - subjects on which he has published over 495 journal papers, 805 conference papers, 26 book chapters, two edited books, and two research monographs. His current research focuses on Data Science with applications to IoT, and power networks with renewables. He is the (co-) inventor of 36 issued patents, and the (co-)recipient of 10 best journal paper awards from the IEEE Signal Processing (SP) and Communications Societies, including the G. Marconi Prize. He received the IEEE-SPS Norbert Wiener Society Award (2019); EURASIP's A. Papoulis Society Award (2020); Technical Achievement Awards from the IEEE-SPS (2000) and from EURASIP (2005); the IEEE ComSoc Education Award (2019); and the IEEE Fourier Technical Field Award (2015). He is a member of the Academia Europaea, Greece's Academy of Athens, and Fellow of the National Academy of Inventors, the European Academy of Sciences, UK's Royal Academy of Engineering, Life Fellow of IEEE, and EURASIP. He has served the IEEE in several posts, including as a Distinguished Lecturer for the IEEE-SPS.
Coffee and cookies will be served at 13:00 before the seminar
Abstract
Self-supervision (SeSu) has gained popularity for “data-hungry” training of machine learning models, especially those involving large-scale graphs, where labeled samples are scarce or unavailable. Main learning tasks in such setups are ill-posed, and SeSu renders them well-posed by relying on abundant unlabeled data as input, to yield low-dimensional embeddings of a reference (auxiliary) model output. In this talk, we first outline SeSu approaches, specialized reference models, and their links with (variational) auto-encoders, regularization, semi-supervised, transfer, meta, and multi-view learning; but also their challenges and opportunities when multi-layer graph topologies and multi-view data are present, when nodal features are absent, and when the ad hoc selection of a reference model yields embeddings not optimally designed for the downstream main learning task. Next, we present our novel SeSu approach which selects the reference model to output either a prescribed kernel or a learnable weighted superposition of kernels from a prescribed dictionary. As a result, the learned embeddings offer a novel, reduced dimensionality estimate of the basis kernel, and thus an efficient parametric estimate of the main learning function at hand that belongs to a reproducing kernel Hilbert space. If time allows, we will also cover online variants for dynamic settings, and regret analysis founded on the so-termed neural-tangent-kernel framework to assess how effectively the learned embeddings approximate the underlying optimal kernel(s). We will wrap up with numerical tests using synthetic and real datasets to showcase the merits of kernel-driven and learnable (KeLe) SeSu relative to alternatives. The real data will also compare KeLe-SeSu with auto-encoders and graph neural networks (GNNs), and further test KeLe-SeSu on reference maps with masked inputs and predicted-outputs that are popular in large language models (LLMs).
Bio
Georgios B. GIANNAKIS is a Professor of Electrical and Computer Engineering at the University of Minnesota, where he holds a Presidential Chair. His interests span the areas of statistical learning, communications, and networking - subjects on which he has published over 495 journal papers, 805 conference papers, 26 book chapters, two edited books, and two research monographs. His current research focuses on Data Science with applications to IoT, and power networks with renewables. He is the (co-) inventor of 36 issued patents, and the (co-)recipient of 10 best journal paper awards from the IEEE Signal Processing (SP) and Communications Societies, including the G. Marconi Prize. He received the IEEE-SPS Norbert Wiener Society Award (2019); EURASIP's A. Papoulis Society Award (2020); Technical Achievement Awards from the IEEE-SPS (2000) and from EURASIP (2005); the IEEE ComSoc Education Award (2019); and the IEEE Fourier Technical Field Award (2015). He is a member of the Academia Europaea, Greece's Academy of Athens, and Fellow of the National Academy of Inventors, the European Academy of Sciences, UK's Royal Academy of Engineering, Life Fellow of IEEE, and EURASIP. He has served the IEEE in several posts, including as a Distinguished Lecturer for the IEEE-SPS.
Practical information
- General public
- Free