On the Scalability of Main Memory OLTP Systems Under High Contention

EDIC candidacy exam
Exam president: Prof. Willy Zwaenepoel
Thesis advisor: Prof. Anastasia Ailamaki
Co-examiner: Prof. James Larus

Abstract
Main memory online transaction processing (OLTP) engines achieve higher throughput than their disk based counterparts. They benefit from not having IO stalls. Main scalability bottleneck in these systems is concurrency control. Current state of the art concurrency control mechanisms are unable to scale to multiple cores under contented workloads. With soon to be arriving 1000-core machines, this lack of scalability will be more visible.

In this report, we discuss the scalability of main memory OLTP engines under contention. First, we explain the architectural changes to database systems with introduction of main memory OLTP systems. Then, we discuss the scalability of these systems on multi-core machines. We use Silo as an example of near-linear scalable engine for low to medium contention. Then, we discuss an extensive study on concurrency control with 1000-core machines. This shows that no known concurrency control scales well under high contention. Therefore, we need to study the bottlenecks of current main memory OLTP engines to make them ready for many-core systems.

Finally, we present our cache hierarchy aware approach to improve scalability of concurrency control methods under high contention.

Background papers
The end of an architectural era:(it's time for a complete rewrite), Stonebraker, M., Madden, S., Abadi, et al.
Staring into the abyss: An evaluation of concurrency control with one thousand cores, Yu, X., Bezerra, G., et al.
Speedy transactions in multicore in-memory databases, Tu, S., Zheng, W., et al.