IEM Distinguished Lecturers Seminar: Artificial Intelligence for Perceptually-Enabled Task Assistants


Event details

Date 08.03.2024
Hour 13:1514:00
Speaker Prof. Ehsan Elhamifar,
Computer Science at Northeastern University, Boston, USA

Location Online
Category Conferences - Seminars
Event Language English
The seminar will take place in ELA 1 and by zoom.
Exceptionally, no broadcast will be available on the Neuchâtel campus.

Coffee and cookies will be served from 13:00.


Humans perform a wide range of complex activities, such as cooking hour-long recipes, assembling and repairing devices and performing surgeries. Many of these activities are procedural: they consist of sequences of steps that must be followed to achieve the desired goals. Learning complex procedures from videos of humans performing them allows us to design intelligent task assistants, robots and coaching platforms that perform or guide people through tasks. In this talk, we present new neural architectures as well as learning and inference frameworks to understand complex activity videos, addressing the following challenges: i) Procedural videos are long, uncurated and contain many task-irrelevant activities, with different videos showing different ways of performing the same task. ii) Gathering framewise video annotation is costly and not scalable to many videos and tasks. ii) At inference time, we must accurately recognize actions as data arrive in real-time, especially with only a few frames.

Ehsan Elhamifar is an Associate Professor in the Khoury College of Computer Sciences, the director of the Mathematical Data Science (MCADS) Lab and the Director of MS in AI at Northeastern University. He has broad research interests in computer vision, machine learning and AI. The overarching goal of his research is to develop AI that learns from and makes inferences about data analogous to humans. He is a recipient of the DARPA Young Faculty Award. Prior to Northeastern, he was a postdoctoral scholar in the EECS department at UC Berkeley. He obtained his PhD in ECE at the Johns Hopkins University (JHU) and received two Masters degrees, one in EE from Sharif University of Technology in Iran and another in Applied Mathematics and Statistics from JHU.