Engineering
2026-07-25 • By Spectrayan Team 7 min read

Building the First SIMD-Native Hyperdimensional Computing Library for Java

What if text similarity didn't need neural networks? What if you could compare documents, extract meaning, and map relationships using simple bitwise operations—and run the entire pipeline at hardware speed?

At Spectrayan, we are obsessed with performance and simplicity. Modern AI often defaults to heavy transformer embeddings for everything, including basic lexical similarity tasks. But loading a gigabyte-sized neural network model just to compare a few strings feels like overkill. We wanted a lightweight, blisteringly fast alternative.

Enter Hyperdimensional Computing (HDC). And more specifically, enter spector-hdc: our new, pure Java 25 library that brings 10,000-dimensional binary vector operations to the JVM, powered directly by AVX-512 SIMD instructions.

What is Hyperdimensional Computing?

Hyperdimensional Computing (HDC), also known as Vector Symbolic Architectures (VSA), is a computing paradigm inspired by how the human brain might process information. Instead of using dense floating-point vectors (like traditional embeddings), HDC relies on massive, pseudo-random binary vectors, typically with 10,000 or more dimensions.

The magic of HDC stems from the mathematical properties of high-dimensional spaces. In a 10,000-dimensional binary space, any two randomly generated vectors are nearly orthogonal (i.e., their Hamming distance is roughly 5,000). This property allows us to encode and combine information with remarkable robustness.

HDC relies on three fundamental operations:

  1. Bind (XOR): Binds two vectors together to represent an association (like a key-value pair). In binary HDC, this is a simple element-wise XOR operation.
  2. Bundle (Majority Vote): Combines multiple vectors into a single vector that represents a set. It's calculated by taking the bitwise majority across all vectors.
  3. Permute (Shift): Cyclically shifts the bits of a vector to represent sequences or order.

To measure the similarity between two hypervectors, we simply calculate their Hamming distance (the number of bits that differ). The closer the distance is to 0, the more similar the concepts; a distance around 5,000 means they are completely unrelated.

For tasks like lexical similarity, language identification, and cognitive memory, HDC provides a fascinating, highly efficient alternative to transformer embeddings. It's deterministic, transparent, and computationally cheap.

Why Java? Why SIMD?

When we started exploring HDC for the Spectrayan ecosystem, we noticed a glaring gap in the tooling. Most existing HDC libraries are built for Python and rely heavily on wrapping C++ or CUDA libraries. For example, torchhd is a phenomenal library, but it wraps PyTorch. hdlib relies on NumPy.

If you are building a high-throughput backend service in Java, dragging in a Python runtime or managing JNI bindings for a simple vector similarity task is an operational nightmare. We couldn't find a single native, hardware-accelerated HDC library built for the JVM.

Java 25 has matured significantly. With the latest iterations of Project Panama, we now have access to the Foreign Function & Memory (FFM) API and the Vector API. The Vector API allows developers to write platform-agnostic Java code that compiles down to highly optimized SIMD (Single Instruction, Multiple Data) instructions, such as AVX-2 and AVX-512 on x86, or NEON on ARM.

We realized we could build an HDC library in pure Java that runs at C-like speeds without any native dependencies.

The Key Discovery: Bit Counting at the Speed of Light

In binary HDC, the most expensive operation during inference or search is computing the Hamming distance between two 10,000-dimensional vectors. A naïve loop iterating over 10,000 bits (or even 157 long primitives) is simply too slow for millions of comparisons per second.

While diving deep into the JDK source code, we made a crucial discovery. The Java Vector API has supported VectorOperators.BIT_COUNT since JDK 19. On modern x86 hardware, this operator directly maps to the VPOPCNTDQ AVX-512 instruction, which counts the number of set bits in a 512-bit register in a single clock cycle.

This realization unlocked our ability to compute the Hamming distance with unprecedented speed. Using the Vector API, the core logic for comparing two hypervectors boils down to just three lines of Java:

// Perform an element-wise XOR between two vectors (find the differing bits)
LongVector xor = va.lanewise(VectorOperators.XOR, vb);

// Count the number of set bits (1s) in the resulting vector
LongVector counts = xor.lanewise(VectorOperators.BIT_COUNT);

// Sum up the counts across all SIMD lanes to get the final Hamming distance
long distance = counts.reduceLanes(VectorOperators.ADD);

By unrolling this loop and processing 512 bits (eight 64-bit longs) per iteration, a complete 10,000-dimensional Hamming distance calculation executes in roughly ~20 AVX-512 instructions. The performance is staggering.

Architecture: Built for the Modern JVM

We designed spector-hdc to be as lean and unopinionated as possible.

Zero Dependencies: The library relies entirely on the standard Java 25 library. No external logging frameworks, no third-party math libraries. Just pure JDK capabilities.

To achieve maximum throughput and minimize garbage collection pauses, we leveraged the latest JVM features:

  • Off-Heap Panama FFM: Large collections of hypervectors (like an in-memory document index) are stored entirely off-heap using the Foreign Function & Memory API (MemorySegment and Arena). This keeps the vectors out of the Garbage Collector's view, allowing for predictable latencies even with millions of stored vectors.
  • Masked Tail SIMD: Vector dimensions don't always align perfectly with hardware register sizes. We use the Vector API's masking capabilities to elegantly handle the "tail" of the vector arrays without falling back to slow scalar loops.
  • Immutable Records: The API heavily utilizes Java Records to provide immutable, thread-safe representations of hypervectors and similarity results.
  • Minimal Surface Area: The entire library consists of just 8 core classes. It is designed to do one thing and do it exceptionally well.

Getting Started

Integrating spector-hdc into your project is designed to be frictionless. Here is how you can use the library to compute the lexical similarity between two phrases:

import dev.spectrayan.hdc.HdcSimilarity;

public class HdcExample {
    public static void main(String[] args) {
        // Initialize the similarity engine with default settings (10,000 dimensions)
        var hdc = new HdcSimilarity();
        
        String text1 = "the quick brown fox";
        String text2 = "the fast brown fox";
        
        // Compute the similarity score (0.0 to 1.0)
        double score = hdc.similarity(text1, text2);
        
        System.out.printf("Similarity score: %.4f%n", score);
        // Output might look like: Similarity score: 0.8245
    }
}

Behind the scenes, the library tokenizes the strings, generates random hypervectors for the n-grams, binds and bundles them into a single 10,000-dimensional document vector, and computes the normalized Hamming distance—all in a fraction of a millisecond.

What's Next

This initial release of spector-hdc proves that the JVM is a formidable platform for high-performance vector operations. But we are just getting started. Our immediate roadmap includes:

  • Binary HNSW Indexing: To support large-scale similarity search, we are implementing a specialized Hierarchical Navigable Small World (HNSW) index optimized specifically for binary vectors and Hamming distance.
  • JMH Benchmarks: We are finalizing a comprehensive suite of Java Microbenchmark Harness (JMH) benchmarks to continuously track performance across different CPU architectures (x86 AVX-512 vs. ARM NEON).
  • Cognitive Memory Integration: We plan to integrate spector-hdc into the core Spectrayan agent framework to serve as a fast, localized cognitive memory layer, allowing agents to recall past interactions without relying on external vector databases.

Call to Action

We believe HDC has a massive role to play in the future of efficient AI and cognitive architectures. If you're as excited about Java performance and alternative AI paradigms as we are, we'd love your help!

Check out the source code, give the repository a star, and try using it in your next project. We are actively looking for feedback on the API design.

Specifically, we are looking for contributors to help with the HNSW implementation. Head over to Issue #359 to join the discussion.

Let's build the fastest HDC library in the world, together.

Related Dispatches