CamCube - Rethinking the Data Center Cluster

Event details
Date | 18.06.2012 |
Hour | 11:00 › 12:00 |
Speaker | Dr. Paolo Costa, Imperial College London, UK |
Location | |
Category | Conferences - Seminars |
Abstract:
Since the early days of networks, a basic principle has been that endpoints treat the network as a black box. An endpoint injects a packet with a destination address and the network delivers the packet. This principle has served us well, and has enabled the Internet to scale to billions of devices using networks owned by competing companies and running applications developed by different parties. However, this approach might not be optimal for large-scale Internet data centers, such as those run by Amazon, Google, Microsoft and Facebook, in which all the components are controlled by a single entity. In the CamCube project, we have been looking at a different approach to build data centers, borrowing ideas from the fields of high performance parallel computing, distributed systems and networking. We use a direct-connect topology, similar to those used in HPC, and a novel networking stack, which supports a key-based routing functionality. By providing applications with a more fine-grained control on network resources, CamCube enables increasing performance and reducing development complexity and cluster costs. In this talk, I will provide an overview of the CamCube platform and motivate its peculiar design choices. I will also describe the design and the evaluation of a number of services that we implemented on CamCube. These include a MapReduce service that provides significant higher performance than existing solutions running on traditional clusters.
Bio:
Paolo Costa is a fellow of Imperial College London. Before joining Imperial, he spent 2.5 years in the Systems and Networking Group of the Microsoft Research Lab in Cambridge and, prior to that, he had been a Postdoctoral Researcher in the Computer Systems group at Vrije Universiteit Amsterdam. He holds a M. Sc. and Ph.D. degree in Computer Engineering from the Politecnico di Milano, received, respectively, in 2002 and 2006.
His research interests lie at the intersection of systems and networking with particular focus on large-scale networked systems, ranging from sensor and mobile networks to overlays and, more recently, data centers.
Since the early days of networks, a basic principle has been that endpoints treat the network as a black box. An endpoint injects a packet with a destination address and the network delivers the packet. This principle has served us well, and has enabled the Internet to scale to billions of devices using networks owned by competing companies and running applications developed by different parties. However, this approach might not be optimal for large-scale Internet data centers, such as those run by Amazon, Google, Microsoft and Facebook, in which all the components are controlled by a single entity. In the CamCube project, we have been looking at a different approach to build data centers, borrowing ideas from the fields of high performance parallel computing, distributed systems and networking. We use a direct-connect topology, similar to those used in HPC, and a novel networking stack, which supports a key-based routing functionality. By providing applications with a more fine-grained control on network resources, CamCube enables increasing performance and reducing development complexity and cluster costs. In this talk, I will provide an overview of the CamCube platform and motivate its peculiar design choices. I will also describe the design and the evaluation of a number of services that we implemented on CamCube. These include a MapReduce service that provides significant higher performance than existing solutions running on traditional clusters.
Bio:
Paolo Costa is a fellow of Imperial College London. Before joining Imperial, he spent 2.5 years in the Systems and Networking Group of the Microsoft Research Lab in Cambridge and, prior to that, he had been a Postdoctoral Researcher in the Computer Systems group at Vrije Universiteit Amsterdam. He holds a M. Sc. and Ph.D. degree in Computer Engineering from the Politecnico di Milano, received, respectively, in 2002 and 2006.
His research interests lie at the intersection of systems and networking with particular focus on large-scale networked systems, ranging from sensor and mobile networks to overlays and, more recently, data centers.
Links
Practical information
- General public
- Free
Organizer
- SuRI 2012
Contact
- Simone Muller