PCP-27: level compaction skip some sst to reduce write amplification
zhangjinpeng87 opened this issue · comments
Description
Ln sst1[a...b e...f]
Ln+1 sst2[a...b] sst3[c...d] sst4[e...f]
Assume a level compaction picks sst1, sst2, sst3 and sst4 as source ssts, we can see sst3's key range is a gap in sst1, but their ranges overlaps. And sst3's content will be unnecessarily re-written in level n+1, this will cause write amplification.
In TiDB the gap between index keys and row keys for the same row is broad, and will cause extra write amplification.
Difficulty
- Hard
Score
- 3600
Mentor(s)
Recommended Skills
- Algorithm
@huachaohuang Seems you have finish this issue facebook/rocksdb#6021, Do you have interest to participate our performance challenge match? https://pingcap.com/community-cn/tidb-performance-challenge/
Well, I'm not sure, I just do that for fun. Thanks for letting me know.
How's the progress of this issue, are you still working on facebook/rocksdb#5201?
I'm interested on using this feature now.
How's the progress of this issue, are you still working on facebook/rocksdb#5201?
I'm interested on using this feature now.
I'm restarting the work.