Naiad: a timely dataflow system
Event details
| Date | 08.10.2013 |
| Hour | 15:15 › 16:00 |
| Speaker | Derek Murray, Microsoft Research |
| Location | |
| Category | Conferences - Seminars |
Naiad is a distributed system that combines the high throughput of batch processors, the low latency of stream processors, and the ability to perform iterative and incremental computations. These requirements raise a fundamental challenge: how to coordinate execution efficiently in a computation with complex data dependencies.
To resolve this challenge, we introduce "timely dataflow", which is a computational model that captures opportunities for parallelism across a wide class of algorithms, and gives rise to an efficient, lightweight coordination mechanism. The resulting system is capable of such diverse tasks as streaming data analysis, iterative machine learning, and interactive graph mining. Our evaluation shows that Naiad outperforms existing specialized systems in their target domains.
This presentation is based on a paper to be presented at SOSP 2013, and is joint work with Frank McSherry, Rebecca Isaacs, Michael Isard, Paul Barham and Martin Abadi.
To resolve this challenge, we introduce "timely dataflow", which is a computational model that captures opportunities for parallelism across a wide class of algorithms, and gives rise to an efficient, lightweight coordination mechanism. The resulting system is capable of such diverse tasks as streaming data analysis, iterative machine learning, and interactive graph mining. Our evaluation shows that Naiad outperforms existing specialized systems in their target domains.
This presentation is based on a paper to be presented at SOSP 2013, and is joint work with Frank McSherry, Rebecca Isaacs, Michael Isard, Paul Barham and Martin Abadi.
Practical information
- General public
- Free
- This event is internal
Organizer
- Amitabha Roy
Contact
- Madeleine Robert