Dopaminergic Feedback Loops: Online Synaptic Weight Adjustments
Dopamine is the primary neurotransmitter involved in reward-motivated behavior. When you perform an action that results in a positive outcome, a splash of dopamine reinforces the neural pathways that led to that action, making you more likely to repeat it.
Spector translates this pathway into a Dopaminergic Reinforcement Loop for AI agents.
Reinforcement API Mechanics
When an agent executes a plan or provides an answer using recalled memories, the user or system provides a feedback score \(R \in [-1.0, 1.0]\) (Reward).
Spector intercepts this reward signal and adjusts the synaptic weights of all memories involved in that specific decision path:
\[W_{ij} \leftarrow W_{ij} + \eta \cdot R \cdot (1.0 - W_{ij})\]
Where:
- \(W_{ij}\) is the Hebbian connection strength between memory \(m_i\) and \(m_j\).
- \(\eta\) is the learning rate (dopamine sensitivity).
- \(R\) is the reward rating.
Reinforcing Cognitive Success
If the outcome was successful (positive reward), the associative links between the query and the used memories are strengthened:
Feedback +1.0 -> Hebbian connection between (Query, Memory Node) increases.
Feedback -1.0 -> Connection decreases, causing the agent to search for other nodes next time.
This reinforcement mechanism allows agents to adapt to user preferences and refine their search paths over time.