AI Center Impact Talk - Marisa Ferrara Boston, Reins AI & Simthetic AI

The talk is jointly organized by the EPFL Parallel Systems Architecture Lab (PARSA) and the EPFL AI Center.

Hosting professor: Prof. Babak Falsafi

For logistics purposes, please register here (using your EPFL email address): HERE. 

The talk will be followed by a coffee session. 

Title
From Monitoring to Adaptation: Simulation and Metrics for Reliable AI Agents

Abstract
Monitoring AI agents can reveal when systems fail, but it does not tell us how to adapt them. In regulated and data-restricted domains such as finance and healthcare, the challenge is even sharper: failures are consequential, yet the underlying data is often inaccessible for privacy or compliance reasons. This talk introduces simulation and information-theoretic metrics as tools for bridging monitoring and adaptation. By reconstructing failure conditions in simulation, and using measures such as mutual information and entropy, we can generate synthetic data that supports the next cycle of training and evaluation. I will share examples from audit, finance, and healthcare that illustrate how simulation enables adaptation without direct access to sensitive records. The broader aim is to connect foundational theory with applied methods for building reliable AI agents in domains where the value to society is high but the data is constrained.

Bio
Marisa Ferrara Boston is the founder of Reins AI and Simthetic AI, where she develops evaluation and simulation frameworks for generative AI systems in regulated and high-stakes domains. Her work focuses on monitoring, simulation, and information-theoretic metrics to build more reliable AI agents, particularly in data-restricted environments such as audit, finance, and healthcare. She advises global consulting firms and industry leaders on post-deployment monitoring and adaptation strategies, helping them navigate the challenges of building AI systems that can evolve safely under regulatory and privacy constraints. She is also a frequent speaker and teacher on AI evaluation, most recently co-leading the IEEE tutorial on Generative AI Evaluation Essentials.