ADOC: Automatically Harmonizing Dataflow Between Components in Log-Structured Key-Value Stores for Improved Performance

Simple item page
dc.contributor.authorYu, Jinghuanen
dc.contributor.authorNoh, Sam H.en
dc.contributor.authorChoi, Young-ri R.en
dc.contributor.authorXue, Chun Jasonen
dc.date.accessioned2024-02-19T20:07:07Zen
dc.date.available2024-02-19T20:07:07Zen
dc.date.issued2023en
dc.description.abstractLog-Structure Merge-tree (LSM) based Key-Value (KV) systems are widely deployed. A widely acknowledged problem with LSM-KVs is write stalls, which refers to sudden performance drops under heavy write pressure. Prior studies have attributed write stalls to a particular cause such as a resource shortage or a scheduling issue. In this paper, we conduct a systematic study on the causes of write stalls by evaluating RocksDB with a variety of storage devices and show that the conclusions that focus on the individual aspects, though valid, are not generally applicable. Through a thorough review and further experiments with RocksDB, we show that data overflow, which refers to the rapid expansion of one or more components in an LSM-KV system due to a surge in data flow into one of the components, is able to explain the formation of write stalls. We contend that by balancing and harmonizing data flow among components, we will be able to reduce data overflow and thus, write stalls. As evidence, we propose a tuning framework called ADOC (Automatic Data Overflow Control) that automatically adjusts the system configurations, specifically, the number of threads and the batch size, to minimize data overflow in RocksDB. Our extensive experimental evaluations with RocksDB show that ADOC reduces the duration of write stalls by as much as 87.9% and improves performance by as much as 322.8% compared with the auto-tuned RocksDB. Compared to the manually optimized state-of-the-art SILK, ADOC achieves up to 66% higher throughput for the synthetic write-intensive workload that we used, while achieving comparable performance for the real-world YCSB workloads. However, SILK has to use over 20% more DRAM on average.en
dc.description.versionPublished versionen
dc.format.extentPages 65-80en
dc.format.extent16 page(s)en
dc.format.mimetypeapplication/pdfen
dc.identifier.isbn9781939133328en
dc.identifier.orcidNoh, Sam Hyuk [0000-0002-9152-0321]en
dc.identifier.urihttps://hdl.handle.net/10919/118051en
dc.language.isoenen
dc.publisherUsenix Associationen
dc.rightsIn Copyrighten
dc.rights.urihttp://rightsstatements.org/vocab/InC/1.0/en
dc.titleADOC: Automatically Harmonizing Dataflow Between Components in Log-Structured Key-Value Stores for Improved Performanceen
dc.title.serialPROCEEDINGS of THE 21ST USENIX CONFERENCE ON FILE AND STORAGE TECHNOLOGIES, FAST 2023en
dc.typeConference proceedingen
dc.type.dcmitypeTexten
dc.type.otherProceedings Paperen
dc.type.otherBooken
pubs.finish-date2023-02-23en
pubs.organisational-group/Virginia Techen
pubs.organisational-group/Virginia Tech/Engineeringen
pubs.organisational-group/Virginia Tech/Engineering/Computer Scienceen
pubs.organisational-group/Virginia Tech/All T&R Facultyen
pubs.organisational-group/Virginia Tech/Engineering/COE T&R Facultyen
pubs.start-date2023-02-21en

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
fast23-yu.pdf
Size:
1.77 MB
Format:
Adobe Portable Document Format
Description:
Published version
Download
License bundle
Now showing 1 - 1 of 1
Name:
license.txt
Size:
1.5 KB
Format:
Plain Text
Description:
Download

Collections

All Faculty Deposits
Scholarly Works, Computer Science