Large transactions in RocksDB uses a lot of memory

Hello, we are experimenting with RocksDB (v. 7.9.0) to replace an internal data structure and are experiencing a high memory usage of several GB when using transactions.

Here’s the way we intend to use RocksDB. Our system has a two-phase commit mechanism where multiple threads can read the committed data and never the uncommitted data. There is a single thread that can write the data, and does so in a transaction to prevent the reading threads from accessing the data until the different mechanisms (there is more than just RocksDB involved here) are in sync. When updating the data in RocksDB, there will never be a conflict, since this is handled by the writing thread - we only update a single key once, at most, during a transaction.

There is only a single transaction at once and 99% of the time it is committed. The only way it would not be committed is on system shutdown/crash where the transaction will be rolled back if not PreCommitted/Prepared and committed otherwise.

We created a small program that simulates the code we will have in our main system. Doing so, we noticed that writing directly in the database leads to memory usage of around 400MB in the test, which is really good, considering we write 10B values in the database.

We then converted the test to use transactions, trying with TransactionDB, OptimisticTransactionDB as well as PessimisticTransactionDB. All of these used way much more ram, hitting 10GB before reaching the 100M values.

We read the documentation on WritePrepared and WriteUnprepared. It looked like WriteUnprepared was what we needed and we played with it along with some other options such as max_open_files. There was some hope with WRITE_UNPREPARED as we saw the data being written to disk while the updates were happening, but it did not affect the ram usage by much, if it did.

Any pointers on things to try would be useful.

1 Like

Can you explain your requirement regarding the transactions? Is there a requirement that data in uncommitted transactions be persisted/restored on failures or is the requirement more of an “atomic commit” of a series of updates?

I am wondering if WriteBatch makes more sense for your use case than a full transactional system. Would WriteBatch meet your use case or is something missing?

1 Like