Pluggable Memory Scoring: How We Built a Service Provider Interface for AI Importance
Every time an AI agent stores a memory, the system must answer one critical question: how important is this? A casual comment about the weather deserves far less cognitive weight than a security vulnerability the agent just discovered. Getting this wrong means either forgetting critical knowledge or drowning in noise.
In Spector Memory, this decision was baked deep into the ingestion pipeline — a hardwired sequence of novelty detection, multi-signal fusion, personality boosts, and flashbulb triggers. It worked well, but it had a fundamental architectural problem: you couldn't change the algorithm without surgery on the core engine.
Today we shipped the ImportanceProvider SPI — a clean service provider interface that makes the entire scoring pipeline pluggable.
The Problem: Hardwired Scoring in Two Places
Spector's importance scoring lived inline inside the ingestion pipeline. But the same logic was needed in two distinct contexts:
graph LR
A["Agent calls remember()"] --> B["Ingestion Pipeline"]
B --> C["Inline Scoring Logic
~40 lines"] C --> D["Store Memory"] E["Agent calls estimateImportance()"] --> F["Read-Only Estimator"] F --> G["Duplicate Scoring Logic
~35 lines"] G --> H["Return Score"] style C fill:#ff6b6b,stroke:#c0392b,color:#fff style G fill:#ff6b6b,stroke:#c0392b,color:#fff
~40 lines"] C --> D["Store Memory"] E["Agent calls estimateImportance()"] --> F["Read-Only Estimator"] F --> G["Duplicate Scoring Logic
~35 lines"] G --> H["Return Score"] style C fill:#ff6b6b,stroke:#c0392b,color:#fff style G fill:#ff6b6b,stroke:#c0392b,color:#fff
This created three concrete problems:
| Problem | Impact |
|---|---|
| Duplication | Same novelty → ICNU → salience → flashbulb logic in two places |
| Rigidity | No way to swap algorithms without modifying engine internals |
| Testability | Scoring math entangled with storage infrastructure |
For a deeper look at what the ingestion pipeline does end-to-end, see our earlier post on The Mathematics of AI Memory Ingestion.
The Solution: Extract → Interface → Wire
The SPI pattern we adopted follows the same architecture Spector already uses for Salience Profiles — a single-method functional interface with a context-in, result-out contract:
graph TD
subgraph "Before: Hardwired"
A1["Ingestion Target"] --> B1["40 lines inline"]
A2["Estimator"] --> B2["35 lines inline"]
end
subgraph "After: SPI"
C1["Ingestion Target"] --> D["ImportanceProvider
.score(context)"] C2["Estimator"] --> D D --> E["ImportanceResult"] end style B1 fill:#ff6b6b,stroke:#c0392b,color:#fff style B2 fill:#ff6b6b,stroke:#c0392b,color:#fff style D fill:#2ecc71,stroke:#27ae60,color:#fff style E fill:#3498db,stroke:#2980b9,color:#fff
.score(context)"] C2["Estimator"] --> D D --> E["ImportanceResult"] end style B1 fill:#ff6b6b,stroke:#c0392b,color:#fff style B2 fill:#ff6b6b,stroke:#c0392b,color:#fff style D fill:#2ecc71,stroke:#27ae60,color:#fff style E fill:#3498db,stroke:#2980b9,color:#fff
The contract is simple:
- Input: A context object containing the memory text, embedding vector, ICNU hints, salience profile, nearest-neighbor distance, and novelty z-score
- Output: A result containing the final importance score, a flashbulb flag, and a full explainability breakdown
The Default Scoring Pipeline
The default provider preserves all existing scoring behaviors, unified into a single pipeline:
flowchart TD
A["Memory Candidate"] --> B["① Novelty Detection"]
B --> C{"Welford Stats
≥ 20 samples?"} C -->|Yes| D["Compute z-score
z = (d - μ) / σ"] C -->|No| E["Baseline: 1.0"] D --> F["② Sigmoid Mapping
σ(1.2 · (z − 1.0))"] F --> G["Scale to 0.05 – 10.0"] E --> G G --> H["③ ICNU Fusion"] H --> I{"Hints
provided?"} I -->|Yes| J["Sigmoid-Gated Fusion
with I×N interaction"] I -->|No| K["Novelty-only score"] J --> L["④ Salience Boosts"] K --> L L --> M["Topic Boost × Self Boost × Agent Boost"] M --> N["⑤ Flashbulb Check"] N --> O{"z-score > threshold?"} O -->|Yes| P["🔴 FLASHBULB
PIN memory"] O -->|No| Q["Normal importance"] P --> R["ImportanceResult"] Q --> R style A fill:#1a1a2e,stroke:#16213e,color:#e94560 style F fill:#0f3460,stroke:#16213e,color:#e94560 style J fill:#0f3460,stroke:#16213e,color:#e94560 style P fill:#e94560,stroke:#c0392b,color:#fff style R fill:#2ecc71,stroke:#27ae60,color:#fff
≥ 20 samples?"} C -->|Yes| D["Compute z-score
z = (d - μ) / σ"] C -->|No| E["Baseline: 1.0"] D --> F["② Sigmoid Mapping
σ(1.2 · (z − 1.0))"] F --> G["Scale to 0.05 – 10.0"] E --> G G --> H["③ ICNU Fusion"] H --> I{"Hints
provided?"} I -->|Yes| J["Sigmoid-Gated Fusion
with I×N interaction"] I -->|No| K["Novelty-only score"] J --> L["④ Salience Boosts"] K --> L L --> M["Topic Boost × Self Boost × Agent Boost"] M --> N["⑤ Flashbulb Check"] N --> O{"z-score > threshold?"} O -->|Yes| P["🔴 FLASHBULB
PIN memory"] O -->|No| Q["Normal importance"] P --> R["ImportanceResult"] Q --> R style A fill:#1a1a2e,stroke:#16213e,color:#e94560 style F fill:#0f3460,stroke:#16213e,color:#e94560 style J fill:#0f3460,stroke:#16213e,color:#e94560 style P fill:#e94560,stroke:#c0392b,color:#fff style R fill:#2ecc71,stroke:#27ae60,color:#fff
The Mathematics
Stage 1 — Novelty Detection uses Welford's online algorithm to maintain running mean μ and variance σ² over observed L2 distances to the nearest existing memory:
$$z = \frac{d_{\text{nearest}} - \mu}{\sigma}$$
A z-score of 0 means "I've seen something just like this." A z-score of 3+ means "this is unlike anything I've stored." For the full mathematics of how this feeds into the SIMD-accelerated scoring pipeline, see our deep dive.
Stage 2 — Sigmoid Mapping converts the raw z-score into a 0.05–10.0 importance range via a shifted sigmoid:
$$I_{\text{novelty}} = 0.05 + \frac{9.95}{1 + e^{-1.2 \cdot (z - 1.0)}}$$
| z-score | Importance | Meaning |
|---|---|---|
| ≤ 0 | ~0.05 | Redundant — nearly identical to existing memory |
| 1.0 | ~5.0 | Moderate novelty — midpoint |
| 2.0 | ~8.1 | High novelty — dopamine spike |
| ≥ 3.0 | ~9.8 | Extreme outlier — flashbulb candidate |
Stage 3 — ICNU Fusion blends four cognitive signals using a sigmoid-gated formula with an Interest × Novelty multiplicative interaction (dopaminergic gating):
$$\text{stimulus} = w_I \cdot (I \times N) + w_C \cdot C + w_U \cdot U$$
$$I_{\text{fused}} = 0.05 + \frac{9.95}{1 + e^{-k \cdot (\text{stimulus} - \theta)}}$$
Where $w_I, w_C, w_U$ are configurable weights, $k$ is steepness, and $\theta$ is the gating threshold. The multiplicative $I \times N$ term ensures high interest without novelty (or vice versa) doesn't inflate importance — both must fire together, mimicking biological dopaminergic reward pathways.
Stage 4 — Salience Boosts applies up to three multiplicative adjustments from the agent's Cognitive Profile:
$$I_{\text{boosted}} = \text{clamp}\big(I_{\text{fused}} \times B_{\text{topic}} \times B_{\text{self}} \times B_{\text{agent}},\ 0.05,\ 10.0\big)$$
Stage 5 — Flashbulb evaluates whether the z-score crosses the flashbulb threshold, triggering emotional valence pinning that prevents the memory from ever being forgotten.
Why Pluggability Matters
The SPI unlocks scenarios that were previously impossible:
graph TB
subgraph "Default Provider"
D1["Welford Novelty
+ ICNU + Salience
+ Flashbulb"] end subgraph "Custom Providers" C1["LLM-as-Judge
Provider"] C2["Domain-Specific
Medical/Legal"] C3["Reinforcement
Learning Provider"] C4["A/B Testing
Provider"] end E["ImportanceProvider SPI"] --> D1 E --> C1 E --> C2 E --> C3 E --> C4 style E fill:#9b59b6,stroke:#8e44ad,color:#fff style D1 fill:#2ecc71,stroke:#27ae60,color:#fff style C1 fill:#3498db,stroke:#2980b9,color:#fff style C2 fill:#3498db,stroke:#2980b9,color:#fff style C3 fill:#3498db,stroke:#2980b9,color:#fff style C4 fill:#3498db,stroke:#2980b9,color:#fff
+ ICNU + Salience
+ Flashbulb"] end subgraph "Custom Providers" C1["LLM-as-Judge
Provider"] C2["Domain-Specific
Medical/Legal"] C3["Reinforcement
Learning Provider"] C4["A/B Testing
Provider"] end E["ImportanceProvider SPI"] --> D1 E --> C1 E --> C2 E --> C3 E --> C4 style E fill:#9b59b6,stroke:#8e44ad,color:#fff style D1 fill:#2ecc71,stroke:#27ae60,color:#fff style C1 fill:#3498db,stroke:#2980b9,color:#fff style C2 fill:#3498db,stroke:#2980b9,color:#fff style C3 fill:#3498db,stroke:#2980b9,color:#fff style C4 fill:#3498db,stroke:#2980b9,color:#fff
| Scenario | How |
|---|---|
| LLM-as-Judge | Ask a language model to evaluate importance — useful for domains where statistical novelty isn't enough |
| Domain-Specific | Medical agents weight patient safety signals; legal agents weight precedent relevance |
| RL Feedback | Agent learns which memories actually helped downstream tasks and adjusts scoring |
| A/B Testing | Run two scoring algorithms side-by-side to measure recall quality differences |
The Broader Architecture Pattern
This is part of an ongoing effort to make Spector's cognitive pipeline fully composable. The SPI pattern has now been applied to three critical extension points:
| Extension Point | SPI | Controls |
|---|---|---|
| Salience | SalienceProfileProvider |
What the agent cares about |
| Importance | ImportanceProvider |
How much a new memory matters |
| Embedding | EmbeddingProvider |
How memories are vectorized |
Each follows the same pattern: context-in, result-out, with a baseline factory for zero-config defaults.
What's Next
The ImportanceProvider SPI opens the door for longitudinal agent evaluation — measuring not just whether the scoring looks correct at ingestion time, but whether it actually improves downstream agent task performance over multi-day sessions. We're currently designing a benchmark framework around MemoryArena (ICML 2026) to validate that importance scoring translates to real-world agent effectiveness.
For a deep dive into how every importance score is broken down into an auditable 10-field explanation, see our companion post: Explainable AI Memory: Why Every Importance Score Deserves a Breakdown.
The ImportanceProvider SPI ships in Spector Memory v1.1.0. The full source is available at github.com/spectrayan/spector.