Scalable microsecond recovery for microsecond RDMA applications
Event details
| Date | 14.06.2021 |
| Hour | 13:00 › 15:00 |
| Speaker | Antoine Murat |
| Category | Conferences - Seminars |
EDIC candidacy exam
exam president: Prof. Edouard Bugnion
thesis advisor: Prof. Rachid Guerraoui
co-examiner: Prof. Bryan Ford
Abstract
Remote Direct Memory Access (RDMA) is a network technology that allows user space programs to access the memory of a remote machine without involving the distant CPU. Coupled with a high performance fabric such as Infiniband, it allows machines to communicate at the microsecond scale by bypassing the kernel, moving the network stack to the hardware, and directly modifying the remote L3 cache. RDMA gained a lot of traction over the past decade and is becoming prevalent within the data center space.
Recent works have demonstrated how to build moderate-scale systems that leverage RDMA to achieve orders of magnitude improvements in both latency and throughput over systems relying on traditional networking stacks. Nevertheless, while the common failure-free path has been vastly improved, recovery is often overlooked, barely takes advantage of new hardware capabilities.
As RDMA deployments continue to scale both server and client side, failures are expected to become the common case and cannot be neglected anymore.
This thesis will revisit the design of state-of-the-art RDMA systems to bring down recovery time to the microsecond scale, and thus achieve increased availability and shortened tail latency.
To provide fast failover, all components on the recovery path will have to be reworked from ground up for the microsecond scale, ranging from failure detection to re-replication, including leases.
Those rethought abstractions will take full advantage of RDMA features such as the M&M paradigm (i.e., using both message passing and shared memory), permissions, hardware multicast, different levels of reliability, etc.
This work will study the impact of those microsecond scale components on the fast path of existing systems as well as on their overall performance and establish what are the trade-offs a system should make as a function of its availability target.
Hopefully, this research will demonstrate how higher availability can be achieved without compromising performances or weakening abstractions by fully leveraging RDMA hardware.
Background papers
- Design guidelines for high performance RDMA systems https://dl.acm.org/doi/10.5555/3026959.3027000
- Microsecond Consensus for Microsecond Applications https://www.usenix.org/conference/osdi20/presentation/aguilera
- Hermes: A Fast, Fault-Tolerant and Linearizable Replication Protocol https://dl.acm.org/doi/abs/10.1145/3373376.3378496
exam president: Prof. Edouard Bugnion
thesis advisor: Prof. Rachid Guerraoui
co-examiner: Prof. Bryan Ford
Abstract
Remote Direct Memory Access (RDMA) is a network technology that allows user space programs to access the memory of a remote machine without involving the distant CPU. Coupled with a high performance fabric such as Infiniband, it allows machines to communicate at the microsecond scale by bypassing the kernel, moving the network stack to the hardware, and directly modifying the remote L3 cache. RDMA gained a lot of traction over the past decade and is becoming prevalent within the data center space.
Recent works have demonstrated how to build moderate-scale systems that leverage RDMA to achieve orders of magnitude improvements in both latency and throughput over systems relying on traditional networking stacks. Nevertheless, while the common failure-free path has been vastly improved, recovery is often overlooked, barely takes advantage of new hardware capabilities.
As RDMA deployments continue to scale both server and client side, failures are expected to become the common case and cannot be neglected anymore.
This thesis will revisit the design of state-of-the-art RDMA systems to bring down recovery time to the microsecond scale, and thus achieve increased availability and shortened tail latency.
To provide fast failover, all components on the recovery path will have to be reworked from ground up for the microsecond scale, ranging from failure detection to re-replication, including leases.
Those rethought abstractions will take full advantage of RDMA features such as the M&M paradigm (i.e., using both message passing and shared memory), permissions, hardware multicast, different levels of reliability, etc.
This work will study the impact of those microsecond scale components on the fast path of existing systems as well as on their overall performance and establish what are the trade-offs a system should make as a function of its availability target.
Hopefully, this research will demonstrate how higher availability can be achieved without compromising performances or weakening abstractions by fully leveraging RDMA hardware.
Background papers
- Design guidelines for high performance RDMA systems https://dl.acm.org/doi/10.5555/3026959.3027000
- Microsecond Consensus for Microsecond Applications https://www.usenix.org/conference/osdi20/presentation/aguilera
- Hermes: A Fast, Fault-Tolerant and Linearizable Replication Protocol https://dl.acm.org/doi/abs/10.1145/3373376.3378496
Practical information
- General public
- Free
Organizer
- EDIC