Cache capacity is twice of the expected size #10707

Expected behavior
I want the max shared cache capacity to 1gb including 150mb mem tables.
To do that, I use this config (jvm) that is coming from kafka streams, that is the same as lot of other libs using rocksdb, even not in Java :

class BoundedMemoryRocksDBConfigSetter : RocksDBConfigSetter {
override fun setConfig(storeName: String, options: Options, configs: Map<String, Any>) {
options.updateTableFormatConfig {

    options.setMaxWriteBufferNumber(3) // kafka streams default
    options.setCompactionStyle(CompactionStyle.UNIVERSAL) // kafka streams default
    options.setStatsPersistPeriodSec(0) // disable dumping stats, unable to have them locally btw

private fun Options.updateTableFormatConfig(block: BlockBasedTableConfig.() -> Unit) {
    setTableFormatConfig((this.tableFormatConfig() as BlockBasedTableConfig).apply(block))

override fun close(storeName: String, options: Options) {
    // nothing to clean

companion object {
    val cache = LRUCache(
        /* capacity = */ 1 * 1024 * 1024 * 1024,
        /* numShardBits = */ -1,
        /* strictCapacityLimit = */ false,
        /* highPriPoolRatio = */ 0.1,
    val writeBufferManager = WriteBufferManager(
        /* bufferSizeBytes = */ 150 * 1024 * 1024,
        /* cache = */ cache,
        /* allowStall = */ true,

It’s deployed inside a container in kubernetes.

But after that, the rocksdb cache capacity indicates the twice (2gb). And the cache usage is near to 2gb.

Actual behavior
Get capacity returns exactly twice of the configured size, e.g. Returns 2gb when 1gb configured, 600mb when 300mb configured, 400 when 200 configured…

Steps to reproduce the behavior
Just use this config and it should be the same ?

Q: “Can you set strictCapacityLimit true?”
A: Yes, I’ve put strict limit to true and false, stall writes to true and false. Tried to allow a capacity of 1gb on 2.5gb available, and 2gb on 2.5gb available, it’s always the same : twice of the configured capacity.

from what i can tell, kafka streams creates a new block cache for each RocksDB state store. so could it be that you’re not passing the same cache object as specified here - Kafka Streams Memory Management | Confluent Documentation

1 Like

From: Chuckame
My code is already coming from your link Kafka Streams Memory Management
That’s exactly why I have a shared cache