Does Rocksdb perform sth like compaction when only read requests occurs?
While running YCSB workloadc (read 100%) with rocksdb, new sst files are generated even when there is no write request. (It seems like when the read performance is very slow, rocksdb creates new sstables.)
Using blktrace, I checked bulk writes were issued by low level threads of rocksdb.
Do Read queries also contribute to a reconstruction of SSTables?
If then, is there a way to prevent this?
Rocksdb may end up the bulk load with “pending compaction bytes” and when invoked again it is cleaning those pending compaction. Based on my knowledge (at least until version 7.2.2 of Rocksdb) Rocksdb does not initiate compaction in order to optimise the read time …
1 Like
Reply was:
How was the data loaded? I can imagine if there was data left in WAL, a DB reopen would rewrite these data.
is there a way to prevent this?
You can set the option disable_auto_compactions to true.
From Mark C.:
RocksDB inherited read-triggered compaction from LevelDB long ago, but that feature has since been disabled.
But as others mention above if you do:
- start RocksDB process, load a lot of data, shutdown RocksDB process
- start RocksDB process, run queries, …
Then there can be compaction debt at the end of step 1, and thus step 2 might require RocksDB to do some compaction.