AI Center Seminar - AI Fundamentals series - Emanuel Tewolde - "Benchmarking Cooperation-Sustaining Mechanisms and LLM Agents in Social Dilemmas"

The talk is jointly organized by the EPFL AI Center and the DLAB as part of the AI fundamentals seminar series.

Hosting professor: Prof. Robert West (DLAB)

Title
Benchmarking Cooperation-Sustaining Mechanisms and LLM Agents in Social Dilemmas

Abstract
It is increasingly important that LLM agents interact effectively and safely with other goal-pursuing agents, yet, recent works report the opposite trend: LLMs with stronger reasoning capabilities behave _less_ cooperatively in mixed-motive games such as the prisoner's dilemma and public goods settings. Indeed, our experiments show that recent models---with or without reasoning enabled---consistently defect in single-shot social dilemmas.
   
To tackle this safety concern, we present the first comparative study of game-theoretic mechanisms designed to enable cooperative outcomes between rational agents _in equilibrium_. Across four social dilemmas testing distinct components of robust cooperation, we evaluate four families of mechanisms: (1) repeating the game for many rounds, (2) reputation systems, (3) third-party mediators to delegate decision making to, and (4) contract agreements for outcome-conditional payments between players. Among our findings, we establish that contracting and mediation are most effective in achieving cooperative outcomes between capable LLM models, and that repetition-induced cooperation deteriorates drastically when co-players vary.  Moreover, we demonstrate that the mechanisms become _more effective_ under evolutionary pressures to maximize individual payoffs.

Bio

Emanuel Tewolde is a fourth-year Computer Science PhD student at Carnegie Mellon University advised by Vincent Conitzer. His research focuses on algorithmic game theory, reinforcement learning, and LLM agents, with an emphasis on the safety, coordination, cooperation, and alignment of AI systems. Ultimately, he strives to understand how to enable artificial intelligence and humans to effectively achieve better social outcomes in strategic interactions.

Emanuel’s work is supported by the Cooperative AI PhD Fellowship, and has previously received the AAAI 2025 Best Poster Award. He has previously worked on AI research agents with Meta FAIR, and researched ML methods for renewable energy systems with the Fraunhofer-Gesellschaft. Prior to CMU, he completed Master’s and Bachelor’s degrees in Mathematics at Imperial College London and the Technical University of Darmstadt respectively.