Compaction evicts entries from Block Cache

General
1

Rocksdb version used: 6.11.4
Does any one know if when background compaction triggers on rocksdb, does it have impact on block cache.
I keep seeing issue on my system when there is compaction running, we see a lot of miss from block cache for filter.
Any idea what could be the reason for this ?
I am attaching a graph which show the hit rate for (index/filter) cache.
image

2

Compaction does not flood the cache . If you like please send us your rocksdb LOG files and we will see if we can help you with the issue of filters falling out of cache,

1 Like
3

the effect a bg compaction has on the block cache is replacing the table readers which are compacted, so filters and indexes of files which were compacted are now deleted and new ones are created instead. the new filters and indexes are not automatically inserted to the block cache and only after a cache miss will they be added.

1 Like
4

Are you saying that during or after the compaction job you experience the eviction? Compaction takes the keys in one set of SST files and “compacts” them into another set of files. Since the keys in the block cache are based on the SST file, any file that is part of compaction will no longer be accessed and those keys will expire from the cache. The same applies to filters and indices in the block cache – they are based on the SST file.

Because compaction does not explicitly remove the entries from the cache when the SST file is compacted away (they are left to expire with their TTL), it is possible that some “live” items could be evicted as the cache is rehydrated after compaction. I do not have a sense of how much of an issue this is in practice, and is likely to be very workload dependent.

1 Like
5

From Mark C.:
"The prepopulate_block_cache option can be set to prepopulate the block cache with data that is written to the L0 on memtable flushes. There is no option yet to do that for compaction writes (L1, L2, …). "