Learning to Optimize LSM-trees: Towards A Reinforcement Learning based Key-Value Store for Dynamic Workloads

  • 2023-09-17 09:55:09
  • Dingheng Mo, Fanchao Chen, Siqiang Luo, Caihua Shan
  • 0

Abstract

LSM-trees are widely adopted as the storage backend of key-value stores.However, optimizing the system performance under dynamic workloads has not beensufficiently studied or evaluated in previous work. To fill the gap, we presentRusKey, a key-value store with the following new features: (1) RusKey is afirst attempt to orchestrate LSM-tree structures online to enable robustperformance under the context of dynamic workloads; (2) RusKey is the firststudy to use Reinforcement Learning (RL) to guide LSM-tree transformations; (3)RusKey includes a new LSM-tree design, named FLSM-tree, for an efficienttransition between different compaction policies -- the bottleneck of dynamickey-value stores. We justify the superiority of the new design with theoreticalanalysis; (4) RusKey requires no prior workload knowledge for systemadjustment, in contrast to state-of-the-art techniques. Experiments show thatRusKey exhibits strong performance robustness in diverse workloads, achievingup to 4x better end-to-end performance than the RocksDB system under varioussettings.

 

Quick Read (beta)

loading the full paper ...