Rocksdb iterator does not release memory ? #10762

General
1

Expected behavior

When the rocksdb iterator is closed the memory used should be cleaned up i.e RSS of my app should not jump up every time an iteration scan is done

Profiler shows that heap memory is not growing

Actual behavior

Ever scan via an iterator jumps RSS of the app up by roughly the DB size [100mb]- and it never goes down - eventually getting OOM

Steps to reproduce the behavior

Basic try-with-resources iteration scan from Java

try(RocksIterator iterator = rocksdb.newIterator()) {
  iterator.seekToFirst();
  while(iterator.isValid() {
    iterator.next()
  }
}

Even though the iterator is closed no memory is being free’d up - and the memory is not being used in the subsequent scans - it is always allocating roughly the same amount each scan

Using Rocksdb v6.22.1 using Linux Java Jars

Can you share your full code for repro?

This is literally it - get a rocksdb object, create an iterator, and scan through in a try with resources. We were debugging this and narrowed it down to this code snippet where the issue was 100% reproducible

try(RocksIterator iterator = rocksdb.newIterator()) {
  iterator.seekToFirst();
  while(iterator.isValid() {
    iterator.next()
  }
}

We have an http server that takes in a request, gets a rocksdb instance from our singleton, triggers this full DB scan and returns an empty response

We start at 2g of RSS and then after around 20 requests it goes to 3gb at which point it gets oom - DB size is 105mb and the rss jumps up by ~100mb each request

but they should be freed when the iterator is destructed.

Yeah this is what i thought too - but rss keeps going up even when the iterator is closed
we are closing the iterator [try with resources does it - also did it explicitly ]

Are you using snapshot?

not using a snapshot - even tried release the implicitly created snapshot before leaving the try - still no luck