Summary:
In this episode Tobias talks about his work on ScaleStore, a distributed storage engine that exploits DRAM caching, NVMe storage, and RDMA networking to achieve high performance, cost-efficiency, and scalability.
Using low latency RDMA messages, ScaleStore implements a transparent memory abstraction that provides access to the aggregated DRAM memory and NVMe storage of all nodes. In contrast to existing distributed RDMA designs such as NAM-DB or FaRM, ScaleStore stores cold data on NVMe SSDs (flash), lowering the overall hardware cost significantly.
At the heart of ScaleStore is a distributed caching strategy that dynamically decides which data to keep in memory (and which on SSDs) based on the workload. Tobias also talks about how the caching protocol provides strong consistency in the presence of concurrent data modifications.
Questions:
0:56: What is ScaleStore?
2:43: Can you elaborate on how ScaleStore solves the problems you just mentioned? And talk more about its caching protocol?
3:59: How does ScaleStore handle these concurrent updates, where two people want to update the same page?
5:16: Cool, so how does anticipatory chaining work and did you consider any other ways of dealing with concurrent updates to hot pages?
7:13: So over time pages get cached, the workload may change, and the DRAM buffers fill up. How does ScaleStore handle cache eviction?
8:57: As a user, how do I interact with ScaleStore?
10:19: How did you evaluate ScaleStore? What did you compare it against? What were the key results?
12:31: You said that ScaleStore is pretty unique in that there is no other system quite like it, but are there any situations in which it performs poorly or is maybe the wrong choice?
14:09: Where do you see this research having the biggest impact? Who will find ScaleStore useful, who are the results most relevant for?
15:23: What are the most interesting or maybe unexpected lessons that you have learned while building ScaleStore?
16:55: Progress in research is sort of non-linear, so from the conception of the idea to the end, where there things you tried that failed? What were the dead ends you ran into that others could benefit from knowing about so they don’t make the same mistakes?
18:19: What do you have planned for future research?
20:01: What attracted you to this research area? What do you think is the biggest challenge in this area now?
20:21: If the network is no longer the bottleneck, what is the new bottleneck?
22:15: The last word now: what’s the one key thing you want listeners to take away from your research?
Links:
Hosted on Acast. See acast.com/privacy for more information.
Related Episodes
Pat Helland | Scalable OLTP in the Cloud: What’s the BIG DEAL? | #50
In this thought-provoking podcast episode, we dive into the world of scalable OLTP (OnLine Transaction Processing) systems with the insightful Pat Helland. As a seasoned expert in the field, Pat shares his insights on the critical role of isolation semantics in the scalability of OLTP systems, emph...
Rui Liu | Towards Resource-adaptive Query Execution in Cloud Native Databases | #49
In this episode, we talk to Rui Liu and explore the transformative potential of Ratchet, a groundbreaking resource-adaptive query execution framework. We delve into the challenges posed by ephemeral resources in modern cloud environments and the innovative solutions offered by Ratchet. Rui guides u...
Yifei Yang | Predicate Transfer: Efficient Pre-Filtering on Multi-Join Queries | #48
In this episode, Yifei Yang introduces predicate transfer, a revolutionary method for optimizing join performance in databases. Predicate transfer builds on Bloom joins, extending its benefits to multi-table joins. Inspired by Yannakakis's theoretical insights, predicate transfer leverages Bloom fi...