RocksDB is an embedded KV database designed with write-ahead-log and log-structured-merge-tree. This project is a java version implementing persistent queue on RocksDB. It's very suitable for:
- Applications want to persistent data on local disk for preventing data loss in memory if a crash.
- Client and server speed do not match, but usually in memory queue are bounded,
rocks-queueprovides unlimited storage capacity.
RocksQueueconsists ofqueue_nameand_queue_nametwo column families.queue_namecolumn family for storing data,_queue_namefor storing queue'sheadandtailpointer.RocksStoreis a rocks queue factory and maintaining the<queue_name, RocksQueue>relationship.
StoreOptions storeOptions = StoreOptions.builder().database("rocks_db").build();
rocksStore = new RocksStore(storeOptions);
queue = rocksStore.createQueue(generateQueueName());byte[] something = "something".getBytes();
long id = queue.enqueue(something);
QueueItem dequeue = queue.dequeue();
assertArrayEquals(dequeue.getValue(), something);You can also consume out the head of the queue and process it, and then invoke the removeHead method to delete it.
QueueItem head = queue.consume();
log.info("Processing queue head {}", head);
queue.removeHead()This project can expose JMX metrics for monitoring.
| Metric Name | Description |
|---|---|
| DatabaseName | RocksStore database name |
| RocksdbLocation | RocksStore location |
| RocksDBDiskUsageInBytes | The current size for RocksStore in bytes |
| NumberOfQueueCreated | How many queues have been created in store |
| IsOpen | Is RocksStore been open |
| IsClosed | Is RocksStore been closed |
| Metric Name | Description |
|---|---|
| QueueName | The queue name |
| QueueSize | Queue size |
| AccumulateBytes | The current size of the queue in bytes,enqueue will increase and dequeue decrease |
| HeadIndex | The head of the queue |
| TailIndex | The tail oft the queue |
| IsCreated | Has the queue been created |
| IsClosed | Has the queue been closed |
| SecondsSinceLastEnqueue | Seconds since the last enqueue in ms |
| SecondsSinceLastConsume | Seconds since the last consume in ms |
| SecondsSinceLastDequeue | Seconds since the last dequeue in ms |
| Benchmark | Mode | Cnt | Score | Error | Units |
|---|---|---|---|---|---|
| RocksQueueBenchmark.consume | avgt | 50 | 12576.618 | ± 17929.697 | ns/op |
| RocksQueueBenchmark.dequeue | avgt | 50 | 2168917.940 | ± 1063197.522 | ns/op |
| RocksQueueBenchmark.enqueue | avgt | 50 | 1762257.820 | ± 232716.449 | ns/op |
| RocksQueueBenchmark.removeHead | avgt | 50 | 1558168.420 | ± 276410.130 | ns/op |