WAL Design: Crash-Resistant Off-Heap Memory Persistence
An in-memory database must provide durability. If the host system crashes, the memory contents must be recoverable.
Spector achieves this by integrating an off-heap Write-Ahead Log (WAL).
Write-Ahead Logging Flow
Before any memory node is written or updated off-heap:
- The update is serialized to a sequential WAL file on disk.
- Once the write is flushed to disk, the off-heap memory segment is updated.
- During startup, the engine replays the WAL to reconstruct the off-heap buffers.
By keeping the log sequential and mapping updates off-heap, Spector ensures data safety without blocking search queries.