IC Colloquium: Beyond General-Purpose Fuzzing: Low-Cost Customization for Testing the AI Systems Stack
Par : Miryung Kim - UCLA
Abstract
The rapid evolution of AI-accelerated hardware and specialized AI/ML compilers has outpaced our ability to check their correctness using traditional software testing. To improve developer productivity and maximize heterogeneous hardware utilization, we must rethink how we discover edge cases within AI compiler stacks. In this talk, I will reflect on my group’s experience designing domain-aware testing engines for compute-intensive systems. I will argue that traditional fuzzing is insufficient for the rapidly evolving requirements of extensible Multi-Level Intermediate Representations (MLIR). Specifically, I will address the high manual effort required to specialize a fuzzer—namely, the labor-intensive process of encoding domain-specific constraints and custom mutation operators.
To lower this barrier to entry, I will discuss techniques to automate this specialization, such as custom mutation synthesis from examples and rule-based repair for bespoke fuzzing. I will conclude by discussing the need to shift fuzzing toward Property-Based Testing (PBT) to bridge the gap between the scale of random fuzzing and the rigor of formal methods, enabling the validation of domain-specific invariants and properties.
Bio
Miryung Kim is a Professor and Vice Chair of Graduate Studies in UCLA’s Computer Science Department. A pioneer in data-intensive software engineering, she led research defining the role of data scientists in software teams. Her current research focuses on developer tools for data and compute-intensive systems, addressing scale and complexity challenges that traditional debugging and testing cannot meet. For her contributions to data-driven software analytics and establishing the significance of code clones in software evolution, she received the IEEE TCSE New Directions Award. Her research demonstrated how recurring patterns could be analyzed to automate bug fixes and refactoring---insights that now inform modern, AI-driven developer tools. Dedicated to mentoring, she was honored with the ACM SIGSOFT Influential Educator Award; eight of her former students and postdocs now hold faculty positions at institutions such as Columbia, Purdue, and Virginia Tech. She served as Program Co-Chair of FSE, delivered keynotes at ASE and ISSTA, and is currently an Amazon Scholar at AWS.
More information
Abstract
The rapid evolution of AI-accelerated hardware and specialized AI/ML compilers has outpaced our ability to check their correctness using traditional software testing. To improve developer productivity and maximize heterogeneous hardware utilization, we must rethink how we discover edge cases within AI compiler stacks. In this talk, I will reflect on my group’s experience designing domain-aware testing engines for compute-intensive systems. I will argue that traditional fuzzing is insufficient for the rapidly evolving requirements of extensible Multi-Level Intermediate Representations (MLIR). Specifically, I will address the high manual effort required to specialize a fuzzer—namely, the labor-intensive process of encoding domain-specific constraints and custom mutation operators.
To lower this barrier to entry, I will discuss techniques to automate this specialization, such as custom mutation synthesis from examples and rule-based repair for bespoke fuzzing. I will conclude by discussing the need to shift fuzzing toward Property-Based Testing (PBT) to bridge the gap between the scale of random fuzzing and the rigor of formal methods, enabling the validation of domain-specific invariants and properties.
Bio
Miryung Kim is a Professor and Vice Chair of Graduate Studies in UCLA’s Computer Science Department. A pioneer in data-intensive software engineering, she led research defining the role of data scientists in software teams. Her current research focuses on developer tools for data and compute-intensive systems, addressing scale and complexity challenges that traditional debugging and testing cannot meet. For her contributions to data-driven software analytics and establishing the significance of code clones in software evolution, she received the IEEE TCSE New Directions Award. Her research demonstrated how recurring patterns could be analyzed to automate bug fixes and refactoring---insights that now inform modern, AI-driven developer tools. Dedicated to mentoring, she was honored with the ACM SIGSOFT Influential Educator Award; eight of her former students and postdocs now hold faculty positions at institutions such as Columbia, Purdue, and Virginia Tech. She served as Program Co-Chair of FSE, delivered keynotes at ASE and ISSTA, and is currently an Amazon Scholar at AWS.
More information
Practical information
- General public
- Free
Contact
- Host Viktor Kuncak