Switching a database to read-only

Question by Marc Sanfacon:

Hello! I am experimenting with RocksDB to embed it within a larger system. In the system, there are multiple classes that use different types of storage. One aspect that the system has is a way to switch the system in read-only mode so that the files are flushed to disk and not more modifications are done to them until the system is switched back to read-write mode.

This is used when performing a copy or a backup operation, for example.

In the case of RocksDB, I saw that there is a way to open a database in RO, but that would mean adding a locking mechanism to the system when closing/re-opening the database, which I would prefer not do.

Since we are using transactions, I’m wondering if once the commit is done and calling CancelAllBackgroundWork, would that make the database be in read-only with no modifications done to it if we only read data from it?

Thanks!

Reply by Hilik, Speedb’s co-founder and chief scientist:
Hi Marc, the simplest way to do a backup is to ask for a checkpoint of the database . The current checkpoint implementation is flushing all the memtables of all the CF and creates a hard link from which the data can be copied. During the time of the checkpoint no new writes can come in. The rocksdb will not become read only but as far as I understand your upper layer will prevent new writes (so while the database structure may change the database content is kept ) . If you want also to disable compaction this can be done as well.

Reply by Mark R. from the speedb engineering team :
I am a little confused as to what exactly your use case is. Are you trying to do this inside of a single process or have the backup/copy job done in a separate process? In a single process, there are backup tools (see include/rocksdb/utilities/backup_engine.h). There are also ReadOnly and Secondary databases you can open. Checkpoints are another way of creating a stable copy of a database, which you could use your own tools to back up. With a better understanding of your use case, we may be able to provide more insight into possible solutions

Reply by Marc Sanfacon:
Thanks for the pointer, I’ll look into this and come back with other questions if needed.

To answer @mrambacher question - it is integrated into an application that is a single process.