Hi all,
I am using rocks Iterator to search for some prefix keys. It takes more pinned immutable memtables and continuously increasing memory size at one point in time my server will be down during out of memory. How to control the pinned immutable memtable size and use the iterators effectively.
Mem Table readers (Index and Filters) :86535080
Live version :1
Unflushed mem tables size :1050632
All mem tables Size:201602032
Block cache usgae :8203312
Block cache pinned usage for iterators :202904
The number of immutable memtables that have not yet been flushed:0
The approximate size of the active memtable in bytes:1050632
The approximate size of active and unflushed immutable memtables in bytes:1050632
The approximate size of active, unflushed immutable, and pinned immutable memtables in bytes:201602032
my pinned immutable memtable size is increasing high how to control it
Reply by Hilik, Speedb’s co-founder and chief scientist:
in the current implementation if you hold an iterator the memtables are held until the itterator is released or renewed. What I would suggest in the case of long itterator is : take a snapshot, run the iterator on the snapshot and renew the iterator after flush (or release the itterator after flush and start it again)
Reply by Hilik, Speedb’s co-founder and chief scientist:
Sure . Instead of running an iterator for a long time, you can get the same functionality by taking a snapshot , starting an iterator (on the snapshot) and periodically (or else after every flush) delete the iterator and start it again, when you are done delete the snapshot . If this feature of a long lasting iterator is commonly needed, please open a PR and we can implement internally a long lived iterator …