DrIM Fast In-memory Transaction Processing using RDMA and HTM XINDA WEL, JIAXIN SHL YANZHE CHEN RONG CHEN,HAB○CHEN Institute of Parallel and distributed Systems Shanghai Jiao tong University, China SOSP 15 The 25th ACM Symposium on Operating Systems Principles
XINDA WEI, JIAXIN SHI, YANZHE CHEN, RONG CHEN, HAIBO CHEN Institute of Parallel and Distributed Systems Shanghai Jiao Tong University, China Fast In-memory Transaction Processing using RDMA and HTM DrTM
Transaction: Key Pillar for Many Systems Alibaba. com S9.3 billion/day Demand speedy distributed Transaction Over large data volumes VPaL 9.56 million 6 million 12306] tickets/day payments/day 2
2 Transaction: Key Pillar for Many Systems Demand Speedy Distributed Transaction Over Large Data Volumes $9.3 billion/day 9.56 million tickets/day 11. 6 million payments/day
High COSt for Distributed TX Many scalable systems have low performance a Usually 10s-100s of thousands of TX/second n High COST(config. that outperform single thread o e.g., HStore, Calvin SIGMOD'12 Emerging speedy tx systems not scale-out o Achieve over 100s of thousands TX/second o e.g., Silo SosP'13, DBXEurOSys'14 Dilemma single-node perf. vs. scale-ouf i Salability But at what Cost HotoS 2015
3 High COST for Distributed TX Many scalable systems have low performance □ Usually 10s~100s of thousands of TX/second □ High COST1 (config. that outperform single thread) □ e.g., HStore, CalvinSIGMOD’12 1 Salability! But at what Cost? HotOS 2015 Dilemma: single-node perf. vs. scale-out Emerging speedy TX systems not scale-out □ Achieve over 100s of thousands TX/second □ e.g., SiloSOSP’13, DBXEuroSys’14
Why(Distributed TXs are slow? Only 4% of wall-clock time spent on useful data processing while the rest is occupied with buffer pools, locking, latching, recovery. Michael stone braker Useful Work 4% Buffer Pool Recovery 24% 24% Latching Locking% 24% i"The Traditional RDBMS Wisdom is All Wrong
4 Why (Distributed) TXs are Slow? Only 4% of wall-clock time spent on useful data processing, while the rest is occupied with buffer pools, locking, latching, recovery. 1 -- Michael Stonebraker 1 “The Traditional RDBMS Wisdom is All Wrong
Opportunities: (not so) New HW Features HTM: Hardware Transaction Memory n Allow a group of load& store instructions to execute in an atomic, consistent and isolated (ACl)way RDMA: Remote Direct Memory Access n Provide cross-machine accesses with high speed, low latency and low CPU overhead Rethink the design of low-COST scalable in-memory transaction systems
5 RDMA: Remote Direct Memory Access □ Provide cross-machine accesses with high speed, low latency and low CPU overhead Rethink the design of low-COST scalable in-memory transaction systems Opportunities: (not so) New HW Features HTM: Hardware Transaction Memory □ Allow a group of load & store instructions to execute in an atomic, consistent and isolated (ACI) way