I’m trying to implement WAL replication in RocksDB. My code for generating the logs on the primary node and reading the logs at secondary node looks like below:
Primary
public List<byte> getTransactionLogs(long sequenceNumber) throws Exception { List<byte> logs = new ArrayList<>(); TransactionLogIterator iterator = this.rocksDB.getUpdatesSince(sequenceNumber); while (iterator.isValid()) { TransactionLogIterator.BatchResult batchResult = iterator.getBatch(); logs.add(batchResult.writeBatch().getWriteBatch().data()); iterator.next(); } return logs; }
Secondary
public void applyTransactionLog(String transactionLog) throws RocksDBException { WriteBatch writeBatch = new WriteBatch(transactionLog.getBytes(StandardCharsets.UTF_8)); this.rocksDB.write(new WriteOptions(), writeBatch); }
Surprisingly, this works fine for small number of entries. But, if the number of entries are big (say even 100), then I get the below error while writing the logs in the secondary node:
WriteBatch has wrong count
Wham am I doing wrong here?