WAL Design: Crash-Resistant Off-Heap Memory Persistence

An in-memory database must provide durability. If the host system crashes, the memory contents must be recoverable.

Spector achieves this by integrating an off-heap Write-Ahead Log (WAL).


Write-Ahead Logging Flow

Before any memory node is written or updated off-heap:

  1. The update is serialized to a sequential WAL file on disk.
  2. Once the write is flushed to disk, the off-heap memory segment is updated.
  3. During startup, the engine replays the WAL to reconstruct the off-heap buffers.

By keeping the log sequential and mapping updates off-heap, Spector ensures data safety without blocking search queries.