Hello, we are experimenting with RocksDB (v. 7.9.0) to replace an internal data structure and are experiencing a high memory usage of several GB when using transactions.
Here’s the way we intend to use RocksDB. Our system has a two-phase commit mechanism where multiple threads can read the committed data and never the uncommitted data. There is a single thread that can write the data, and does so in a transaction to prevent the reading threads from accessing the data until the different mechanisms (there is more than just RocksDB involved here) are in sync. When updating the data in RocksDB, there will never be a conflict, since this is handled by the writing thread - we only update a single key once, at most, during a transaction.
There is only a single transaction at once and 99% of the time it is committed. The only way it would not be committed is on system shutdown/crash where the transaction will be rolled back if not PreCommitted/Prepared and committed otherwise.
We created a small program that simulates the code we will have in our main system. Doing so, we noticed that writing directly in the database leads to memory usage of around 400MB in the test, which is really good, considering we write 10B values in the database.
We then converted the test to use transactions, trying with TransactionDB, OptimisticTransactionDB as well as PessimisticTransactionDB. All of these used way much more ram, hitting 10GB before reaching the 100M values.
We read the documentation on WritePrepared and WriteUnprepared. It looked like WriteUnprepared was what we needed and we played with it along with some other options such as max_open_files. There was some hope with WRITE_UNPREPARED as we saw the data being written to disk while the updates were happening, but it did not affect the ram usage by much, if it did.
Any pointers on things to try would be useful.