Memory Interference: Solving Proactive and Retroactive Overwrite
In cognitive psychology, interference theory explains why memories become harder to retrieve. It exists in two forms:
- Proactive Interference: Older memories interfere with the retrieval of newer ones.
- Retroactive Interference: Newly acquired information interferes with the recall of older memories.
In LLMs, retroactive interference is a major issue: a long, chatty conversation causes the model to "forget" the initial instructions or parameters set at the beginning. Spector implements interference equations to balance stability and plasticity.
The Interference Equation
Spector calculates an interference factor \(IF_{ij}\) between a candidate memory \(m_i\) and a newly incoming memory \(m_j\):
\[IF_{ij} = \text{Overlap}(m_i, m_j) \times e^{-\delta d(m_i, m_j)}\]
Where:
- \(\text{Overlap}\) represents semantic tag intersection.
- \(d(m_i, m_j)\) is the vector distance.
- \(\delta\) is the protection coefficient.
Safeguarding Core Rules
If a new memory covers the exact same semantic space as an older one, the older memory's retrieval weight is degraded slightly unless its importance vector indicates it is a core system prompt or user rule.
Casual chat node -> overlapping vector -> older casual node decayed (retroactive interference).
Core system prompt node -> overlapping vector -> protected by high importance vector -> immune.
This balance ensures that while casual conversational context decays naturally, fundamental parameters remain protected from retroactive overwrite.