JVector Advanced Usage

This section covers advanced usage patterns for production deployments.

On-Disk Index with Compression

The on-disk format has two independent settings. approximateScoring controls how traversal scores candidates and trades disk space or heap, and some recall, for fewer reads on a graph larger than the page cache; vectorStorage controls how each vector is stored and trades a much smaller file for a little candidate recall, and only when approximateScoring is NONE. This section covers the first; see NVQ: Quantized Vectors in the Graph for the second.

For on-disk graphs larger than the page cache, where reads dominate search latency, combine on-disk storage with Product Quantization (PQ) compression. While the graph fits in the page cache, keep NONE: it is both faster and more accurate there.

VectorIndexConfiguration config = VectorIndexConfiguration.builder()
    .dimension(768)
    .similarityFunction(VectorSimilarityFunction.COSINE)
    .maxDegree(32)
    .beamWidth(200)
    // On-disk storage
    .onDisk(true)
    .indexDirectory(Path.of("/data/vectors"))
    // PQ compression (fewer reads per hop on a graph larger than the page cache)
    .approximateScoring(ApproximateScoring.FUSED_PQ)
    .pqSubspaces(48)  // Must divide dimension evenly
    .build();
PQ compression makes the on-disk index larger, not smaller, and it costs recall. It is an I/O optimisation for graphs larger than the page cache. See What PQ Compression Costs and Buys before enabling it. To make the index smaller, set vectorStorage to NVQ — see NVQ: Quantized Vectors in the Graph.
enablePqCompression(true) is the deprecated spelling of approximateScoring(ApproximateScoring.FUSED_PQ). Both write the same setting.

How the On-Disk Index Works

The vector index uses the HNSW (Hierarchical Navigable Small World) algorithm, which constructs a multi-layer proximity graph for efficient approximate nearest neighbor search. When on-disk mode is enabled, this graph is persisted to disk as immutable segments and accessed via memory-mapped files rather than being held entirely on the Java heap.

The Vector File

Independently of the graph, every index keeps its vectors in a memory-mapped vector file ({indexName}.vec plus chunks {indexName}.vec-{n}), off the heap. Mutations write it directly; graph construction, search and reranking read it. It is a cache of what the vectorizer derives from the entities, not part of what the storage commits:

  • A checkpoint every vectorCheckpointIntervalMs forces the chunks written since the previous one to disk. After a crash, only the chunks written since the last checkpoint are recovered, by re-vectorizing their entities.

  • A missing vector file, or one that does not match the stored index, is replaced: every entity is re-vectorized.

That is why Vectorizer.vectorize must reproduce the committed vector of a persisted entity; see Where the Vectors Live.

Segments and the Persist Lifecycle

New entities are always inserted into an in-memory graph first, each by finding its nearest neighbors through a beam search and connecting it to them via bidirectional edges. A persist freezes that graph in an instant, writes it as one more immutable segment while a fresh in-memory graph takes new mutations, and commits the segments with a new manifest. Nothing is rebuilt, and searches and mutations go on while the segment is written. A search visits every segment and the in-memory graphs and merges their results, so each entity is answered by exactly one of them.

  • persistToDisk() and the persist on shutdown write whatever changed.

  • A background persist (persistenceIntervalMs) writes a segment once the in-memory graph holds flushThreshold nodes. Keep that threshold large: small segments add search cost without making anything more durable, since the vectors are in the vector file anyway.

  • Background merges combine segmentMergeFactor segments of similar size into one, rebuilding them from the vector file in parallel while searches and mutations go on, so the segment count grows only with the logarithm of the index size. Only if the merges fall behind and a persist would exceed maxSegments does it consolidate all segments into one, which blocks mutations and searches for the duration of the rebuild.

The configured indexDirectory then holds:

  • {indexName}.manifest — The commit point: the segment list with each segment’s format and deletions, and the state of the stored index the segments describe (vector count, highest entity id, structural modification count), protected by a CRC32 checksum.

  • {indexName}.seg-{n}.graph — The HNSW graph of segment n: node connectivity (edges), one vector block per node — full-precision inline vectors or NVQ-quantized ones, depending on vectorStorage — and, when approximateScoring is FUSED_PQ, the PQ codebook and the fused neighbour codes.

  • {indexName}.seg-{n}.l2g — The entity id of each node of segment n.

  • {indexName}.seg-{n}.pqv — Only with PQ_IN_MEMORY: the PQ codes of segment n and their codebook.

  • {indexName}.vec and {indexName}.vec-{n} — The vector file.

Segment files are written and synced before the manifest that lists them, and the manifest is replaced atomically, so an interrupted persist leaves the previous manifest with all of its segments. Files no manifest lists are deleted at the next load.

Loading from Disk

On startup, the index reads the manifest. If its checksum holds and the state it records matches the stored index, the segments are opened with memory-mapped I/O and searched as they are, without rebuilding the graph.

If the manifest does not match — for example because the process ended after a store of the GigaMap but before the next persist of the index — it is rejected, and so is a load in which a segment fails to open. The graph is then rebuilt from the vector file, and the next persist replaces the rejected files; it is scheduled even if the index is only searched. The same rebuild happens when the directory holds no manifest. A named location bound to a new, empty directory (see Index Locations) also has no vector file, so the vector file is rebuilt there first, one vectorizer call per entity.

A segment written for another format or codebook than the configured one — after a change to vectorStorage, approximateScoring, pqSubspaces or nvqSubvectors — is still used, but searched exactly and rewritten first by the next background merge. An explicit persistToDisk() rewrites such segments itself before it returns.

The graph files of 4.x ({indexName}.graph, .meta and .pqv) are not read. They are logged once at the first load and can be removed; the graph is rebuilt from the vector file and persisted as segments at the next persist. See Upgrading from 4.x.

Memory-Mapped Access

The on-disk graph is accessed through memory mapping the segment files. The operating system’s virtual memory subsystem pages graph data in and out as needed, which means:

  • The JVM heap is not burdened with the full graph structure.

  • The OS page cache automatically keeps frequently accessed portions of the graph in physical memory.

  • Datasets much larger than available RAM can still be searched, with a trade-off of slightly higher query latency when pages need to be fetched from disk.

Product Quantization (PQ) Compression

Product Quantization is a lossy vector compression technique. Here it is used to make graph traversal cheaper, not to shrink the index.

The Core Idea

A raw float vector of dimension D requires D * 4 bytes. For a 768-dimensional embedding, that is 3,072 bytes per vector. Scoring a candidate during traversal therefore means reading 3 KB and computing a 768-element similarity.

PQ shrinks the scoring representation by splitting each vector into M subspaces (contiguous slices) and independently quantizing each subspace using a codebook of 256 centroids (cluster centers). Since 256 values fit in a single byte (2^8 = 256), each subspace is encoded as one byte, so a vector is represented by M bytes for scoring purposes.

Example: A 768-dimensional vector with pqSubspaces=48:

  • Each subspace covers 768 / 48 = 16 dimensions.

  • Each subspace is encoded as 1 byte (index into its 256-entry codebook).

  • Scoring representation: 48 bytes per vector, vs. 3,072 bytes raw.

What PQ Compression Costs and Buys

The compressed codes are stored in addition to the full-precision vectors, not instead of them — the exact vectors are still needed for reranking. Worse, FusedPQ stores the codes of every one of a node’s neighbours inside that node’s own block, so each code is duplicated maxDegree times.

A node therefore costs dimension * 4 bytes (inline vector) + pqSubspaces * maxDegree bytes (fused codes), against dimension * 4 with PQ disabled. At dimension=768, maxDegree=32:

Configuration Bytes per node 1M vectors

PQ disabled

~3,208

~3.2 GB

PQ, pqSubspaces=48

~4,744

~4.7 GB

PQ, pqSubspaces=192 (the dimension/4 default)

~9,352

~9.4 GB

What you get in return is far less I/O per query. Without PQ, every hop scores its candidates by issuing one random read of a full vector each. With FusedPQ, a single sequential read of the node’s fused block scores all of its neighbours at once, using SIMD table lookups instead of full-precision arithmetic. On a memory-mapped index whose working set exceeds RAM, that is the difference between one page fault per candidate and one per node.

Enable PQ when query latency on an on-disk graph larger than the page cache matters, and accept some recall loss for it. Do not enable it to save disk space — it does the opposite.

Memory, not just disk

The disk growth above is also RAM growth for whatever part of the file is resident, so it is worth being explicit about where PQ helps and where it costs.

It reduces the resident working set during search. Without PQ, each traversal hop scores its candidates by issuing maxDegree scattered reads of a full dimension * 4 byte vector — at dimension 768 and maxDegree=32, around 98 KB touched across many pages per hop. With FusedPQ it is a single contiguous read of the node’s fused block: 6 KB at pqSubspaces=192, 1.5 KB at 48. Reranking is also cheaper on the heap, since it reads the inline vectors straight out of the mapping into one reused buffer rather than materialising each candidate.

It costs transient heap at persist time. Encoding a segment allocates pqSubspaces bytes per node it writes. Training takes a sample of at most 128,000 vectors; it happens once, after which the codebook is recovered from the segments instead. Steady-state heap is unaffected beyond the codebook and a small cache of the top graph layer’s codes. A persist reads the vectors it encodes and writes from the vector file; it does not call the vectorizer.

It costs speed and recall while the graph fits in the page cache. PQ’s only win is avoiding reads. Scoring from codes saves reads, not arithmetic: at the automatic pqSubspaces a hop costs about as much as the exact comparison it replaces, and a neighbour that scores badly from its code is never visited, so traversal finds fewer of the true nearest neighbours. While the graph is cached, PQ is therefore slower and less accurate than NONE, and the extra bytes are simply resident: at 1M vectors and dimension 768, roughly 3.2 GB without PQ against 5.3 GB at pqSubspaces=48 or 9.9 GB at the default. Treat PQ as a way to scale past available memory, not as a latency tweak.

These figures are extrapolated from a small synthetic benchmark. Per-node costs scale cleanly with dimension and maxDegree, but measure at your own size before sizing hardware.

Training the Codebook

Before vectors can be compressed, PQ must learn the codebooks by clustering the training data. This happens automatically when the index is persisted to disk, on the first persist at which at least 256 vectors exist. Below that threshold segments are written uncompressed, and training is reattempted on the next persist after the data changed — an unchanged index is deliberately not retried. The training process:

  1. Collects a training sample: entities without an embedding are skipped, and at most 128,000 vectors are taken, drawn as a uniform reservoir sample when the index holds more than that. The cap bounds the memory the persist needs and matches the size JVector subsamples to internally, so a larger sample would not improve the codebook.

  2. Splits each vector into M subspaces.

  3. Runs k-means clustering (k=256) independently on each subspace, producing 256 centroids per subspace.

  4. Keeps the resulting codebook for encoding the segments.

The codebook is written into each segment (the .graph header for FUSED_PQ, the .pqv file for PQ_IN_MEMORY) and recovered on every subsequent load, so it is trained once and shared by all segments of the index. A segment without the codebook, or with another one, is searched exactly and rewritten by the next merge.

Approximate Distance Computation

Once vectors are compressed, distances between a query vector and a stored (compressed) vector can be computed without decompressing. For each subspace, the distance between the query’s subspace and the centroid (looked up by the stored byte index) is precomputed into a lookup table. The total distance is the sum of these per-subspace distances.

This is much faster than computing the full exact distance, because it replaces D floating-point multiplications with M table lookups and additions.

FusedPQ: Compression Fused into the Graph

When PQ compression is enabled for an on-disk index, the implementation uses FusedPQ — a technique where the compressed PQ codes for each node’s neighbors are stored directly alongside the graph edges in the on-disk file.

During graph traversal (search), the HNSW algorithm must evaluate the distance from the query to each candidate neighbor. With FusedPQ, these approximate distances can be computed directly from the fused PQ codes embedded in the graph structure, without a separate random-access read of each candidate’s vector. This reduces I/O and cache misses during search.

Each segment graph file thus contains, per node:

  • Graph edges — The connections to neighboring nodes (the HNSW structure).

  • One vector-storage block — either full-precision inline vectors or NVQ quantized vectors, which are about a third of the size. Whether a search reads it at all depends on both settings; the table under Two-Phase Search with Reranking gives the four cases. See NVQ: Quantized Vectors in the Graph.

  • FusedPQ codes, when approximateScoring is FUSED_PQ — The PQ-compressed representations of the node’s neighbors, used for fast approximate scoring during traversal.

FusedPQ places no constraint on maxDegree. The degree is recorded in the graph header, so the reader sizes each fused block from the file itself. Because a node’s fused block holds maxDegree codes, however, maxDegree is what multiplies the storage and per-hop read cost of PQ.

NVQ: Quantized Vectors in the Graph

vectorStorage and approximateScoring are two independent dimensions of the on-disk format, and they pull in opposite directions on size. FUSED_PQ adds the neighbour codes and makes the file larger; NVQ replaces the full-precision vectors with 8-bit quantized ones and makes it smaller. Any combination of the two is valid.

VectorIndexConfiguration config = VectorIndexConfiguration.builder()
    .dimension(768)
    .similarityFunction(VectorSimilarityFunction.COSINE)
    .maxDegree(32)
    .onDisk(true)
    .indexDirectory(Path.of("/data/vectors"))
    // Quantized vectors instead of full precision: about 3.4x smaller
    .vectorStorage(VectorStorage.NVQ)
    // The encode cost is almost entirely absorbed by the parallel writer
    .parallelOnDiskWrite(true)
    .build();

Or use the forCompactLargeDataset(dimension, indexDirectory) preset, which sets NVQ together with NONE and turns on parallelOnDiskWrite for the reason given below. That pairing is the smallest on-disk footprint available, because it drops the fused codes as well as shrinking the vectors.

What NVQ Costs and Buys

Each vector is recentred on a global mean and then scaled per subvector through a learned nonlinearity, which is what lets one byte per dimension carry as much as it does. Per node the block is 4 + dimension + 28 * nvqSubvectors bytes against dimension * 4 for full precision.

Measured at dimension 768, maxDegree 32, 20,000 vectors:

vectorStorage Graph file Bytes per node

INLINE

64.2 MB

3,208

NVQ

18.8 MB

936

That is a 3.4x reduction, and it is the only setting on either dimension that shrinks the file. The ratio depends on the dimension, because the fixed per-node costs weigh less as it grows, and on nvqSubvectors: eight subvectors add 28 bytes each and bring it down to about 2.8x, which is why the default is one.

It never costs score accuracy, whatever approximateScoring is set to: reranking compares against the full-precision vectors of the vector file, so the scores and the ordering a search returns are exact. What it costs beyond that depends on the scoring mode, because that decides whether the quantized vectors are read during traversal at all.

With NONE they are, so quantization changes which candidates traversal finds — and a candidate it never reaches is one reranking cannot recover. What recovers those is a wider search beam, the same lever that answers a low pqSubspaces.

With FUSED_PQ the fused codes drive traversal and the quantized block is not consulted, so candidate selection is exactly what it would be with INLINE storage. What changes there is the footprint, and that the rerank now reads the vector file rather than the graph’s own mapping — one vector-file read per reranked candidate where there was none.

The suite’s five-seed recall check, at dimension 64, puts all four storage/scoring combinations within noise of each other. For a sense of what the traversal-only loss can look like where it does bite, a raw jvector probe that reranked from the quantized vectors — which this integration does not do — measured recall@10 of 0.921 against an exact baseline of 0.923 at dimension 256. Both figures come from synthetic data, so measure it on yours if the last fraction of a percent matters.

Persist Cost, and Why to Pair NVQ with Parallel Writes

Quantizing is not free: each vector’s nonlinearity is fitted by a grid search at write time, which is real work per node. It is worth knowing how it lands, because it falls on the persist path, every time a segment is written or merged.

Sequentially NVQ costs about 3x the write time of an INLINE segment. Under the parallel writer that falls to about 1.2x, because the encode runs per node on the writer’s worker threads rather than in one pass up front. Set parallelOnDiskWrite(true) when using NVQ, unless you have a specific reason not to.

PQ_IN_MEMORY: The Same Codes, Kept Out of the Graph

FUSED_PQ and PQ_IN_MEMORY buy the same approximate traversal and differ only in where the codes live. Fusing writes each node’s code into every neighbour that references it, so the cost is pqSubspaces * maxDegree bytes per node on disk. Keeping them out of the graph costs pqSubspaces bytes per ordinal once, in heap.

This section describes PQ_IN_MEMORY on an on-disk index, where the codes are persisted in a {indexName}.seg-{n}.pqv file per segment and read back into heap on load. An in-memory index has no persist and therefore no such file at all; it builds the codes at optimization instead. See Compressed Scoring for an In-Memory Index.

As with FUSED_PQ, this pays only for a graph larger than the page cache; while the graph fits, NONE is faster and more accurate.

At dimension=768, maxDegree=32 and the automatic pqSubspaces of 192, that is 6,144 bytes per node of graph file against 192 bytes per ordinal of heap. Per ordinal rather than per vector because the codes are indexed by entity id, so a deletion hole still costs its bytes — the IMPORTANT note below puts numbers on that.

VectorIndexConfiguration config = VectorIndexConfiguration.builder()
    .dimension(768)
    .similarityFunction(VectorSimilarityFunction.COSINE)
    .maxDegree(32)
    .onDisk(true)
    .indexDirectory(Path.of("/data/vectors"))
    .vectorStorage(VectorStorage.NVQ)                        // smallest graph
    .approximateScoring(ApproximateScoring.PQ_IN_MEMORY)     // codes out of the graph entirely
    .parallelOnDiskWrite(true)
    .build();
The heap cost is linear and has no ceiling. It is pqSubspaces bytes for every ordinal up to the highest in use — deletion holes included, because the codes are indexed by entity id — and it stays resident for as long as the index is open. Roughly 192 MB per million vectors at the settings above, and roughly 19 GB at a hundred million. Size it before choosing this over FUSED_PQ, which pays in disk and page cache instead.

The .pqv File

On an on-disk index, PQ_IN_MEMORY adds {indexName}.seg-{n}.pqv beside each segment’s graph. Like every segment file it is written and synced before the manifest that lists it, so a crash before the commit leaves the previous manifest in effect.

A missing or damaged .pqv file (it carries a CRC32 checksum) rejects the load. The index is not served without it: the configuration says this index traverses on PQ codes, and quietly scoring it exactly instead would make the configuration untrue. The graph is rebuilt from the vector file, and the next persist writes the segments and their .pqv files anew. Include the .pqv files in any backup or copy of the directory, and do not delete them separately from the graph.

The codebook lives in the .pqv file too. A fused graph carries it in the graph header, but this mode writes no fused feature, so the .pqv file is the only copy.

Compressed Scoring for an In-Memory Index

PQ_IN_MEMORY is the one PQ-based scoring mode an in-memory index can use - NONE, the default, stays available and is what an index uses until it switches. Everything above about the .pqv file applies to on-disk indices only: there is no persist here, so there is no file, and the codes live in heap for as long as the index is open. An in-memory index scores from its memory-mapped vector file, so the trade-off is the same as on disk: compressed scoring replaces vector-file reads with in-heap code lookups, which pays only when the vector file does not fit in the page cache, and it costs recall. While it fits, keep NONE.

VectorIndexConfiguration config = VectorIndexConfiguration.builder()
    .dimension(768)
    .similarityFunction(VectorSimilarityFunction.COSINE)
    .approximateScoring(ApproximateScoring.PQ_IN_MEMORY)
    // Required here: the switch to compressed scoring happens at optimization
    .optimizationIntervalMs(600_000)
    // And the optimization has to actually run: the background manager skips one until this many
    // changes have accumulated, and the default of 1000 would strand an index of a few hundred
    // vectors on exact scoring for good.
    .minChangesBetweenOptimizations(100)
    .build();

An index cannot start compressed. The graph builder takes its scoring strategy at construction and a codebook needs at least 256 vectors, so the switch is necessarily a rebuild: train, encode every live ordinal, construct a fresh builder against the codes, and replay the nodes into it. That is O(n), so it happens at optimization — which already pays an O(n) cost — and nowhere else. Triggering it from a mutation would put a full retrain and rebuild inside whichever add happened to cross the threshold.

That is why optimizationIntervalMs must be set for this mode. Without a scheduled optimization there is no point at which the switch could happen, so the configuration would be accepted and then silently never take effect; it is rejected at build time instead. isPqCompressionActive() reports which side of the switch the index is on.

The switch holds the GigaMap’s lock for its entire duration, and that is a rebuild, not an instant. Everything the map does waits for the whole of it — writes and reads alike, on any index on that map, and searches against this one too. A search entering during the switch blocks before it reaches the index at all, because the entry point takes that same lock to settle a deferred graph rebuild before it starts. Measured at 20,000 vectors: a switch of 8.6 s, during which a search that would take 5 ms waits 7.9 s, as do get, iterate and add. Measured at dimension=64, the cost is close to linear in the number of vectors:

+

Vectors Switch duration

5,000

~2 s

10,000

~4 s

20,000

~6-9 s

25,000

~10 s

50,000

~14-19 s

1,000,000

several minutes

+ It happens at most once per index generation, and it is the same kind of exclusion a removeAll() takes — but those are usually deliberate, and this one is triggered by the background optimization, so it can arrive at a time nobody chose. Treat it as a full stop on that GigaMap for the duration, not as a write pause. On a large index, plan for it: call optimize() explicitly at a quiet moment so the pause happens when you decide rather than on the scheduler’s tick, and size optimizationIntervalMs and minChangesBetweenOptimizations with the table above in mind.

An index restored from storage is a special case: its entities are already in the store, so nothing bumps the change count and a scheduled optimization would be skipped forever. Its first one is therefore armed automatically, regardless of minChangesBetweenOptimizations, if it holds enough vectors to train a codebook. Since an in-memory index rebuilds its graph at every start, the switch is repeated after every restart. A newly created index is not armed — its entities arrive afterwards and each one counts — so the threshold applies there in full.

Construction under compressed scoring does no reranking, so graph quality can drop relative to an exactly built graph. Measured recall stays above 0.90 on clustered data at the default beam width, but raise beamWidth for this mode if your own measurements come in lower.

When PQ compression is active, searches use a two-phase approach to balance speed and accuracy:

Phase 1 — Approximate candidate retrieval: The HNSW graph of each segment is traversed using the PQ codes for fast approximate distance computation. This phase fetches max(k, minSearchBeamWidth) candidates — never less than the configured search beam width (see minSearchBeamWidth) so the top-k stays stable across different k values. The per-query search(query, k, searchBeamWidth) overload raises this floor for a single call when higher recall is needed.

Phase 2 — Exact reranking: The approximate candidates are then re-scored against full-precision vectors, and the exact scores are sorted so the best k results are returned.

The two phases above describe the PQ modes, which are what they exist for. The other combinations are shorter: NVQ + NONE traverses on the graph’s quantized vectors and then reranks, and INLINE + NONE scores exactly during traversal and has no second phase to run.

Reranking is exact under every combination of the two settings, but it does not always read the same vectors:

vectorStorage approximateScoring Traversal scores from Reranking reads

INLINE

NONE

the vector file

the vector file — traversal is already exact, so nothing is re-scored

INLINE

FUSED_PQ

the fused neighbour codes

the graph’s inline vectors, at no extra read per candidate

NVQ

NONE

the quantized vectors in the graph

the vector file

NVQ

FUSED_PQ

the fused neighbour codes

the vector file

any

PQ_IN_MEMORY

the in-heap PQ codes

the vector file

The vector file is the source wherever the graph holds no full-precision copy to read, and for the default INLINE + NONE, where the vectors in the graph are deliberately left unused. See NVQ: Quantized Vectors in the Graph.

An on-disk index searches every segment this way, plus the in-memory graph that holds the changes since the last persist, which is always scored exactly. The results are merged by score, so each entity is answered once.

This two-phase approach achieves nearly the same recall as an uncompressed search while making each traversal hop much cheaper.

Choosing PQ Subspaces

The pqSubspaces parameter (often called M in PQ literature) controls the accuracy/cost trade-off. Under FusedPQ each node stores maxDegree codes, so the per-node cost is pqSubspaces * maxDegree bytes — the figures below assume maxDegree=32:

Subspaces Fused bytes per node Characteristics

dimension / 2 (e.g., 384 for dim=768)

12,288 bytes

Highest approximate accuracy, but each hop reads more than the full vectors it replaces — rarely worth it

dimension / 4 (e.g., 192 for dim=768)

6,144 bytes

The auto default. Still twice the size of an inline vector per hop; prefer a smaller M unless recall demands it

dimension / 8 (e.g., 96 for dim=768)

3,072 bytes

Break-even against the inline vector, with mild accuracy loss

dimension / 16 (e.g., 48 for dim=768)

1,536 bytes

Cheapest hops, but needs a wider minSearchBeamWidth to hold recall

An explicitly configured pqSubspaces must divide the dimension evenly. If pqSubspaces is left at 0 it defaults to dimension / 4, which is not subject to that rule: JVector distributes any remainder across the subvectors and only requires the count to be at most the dimension.

Subspaces, Beam Width and Recall

Lowering M does not come for free, and the exact rerank does not compensate for it: reranking only reorders the candidates traversal already found, so candidates that lossy scoring never reached are lost for good. What recovers them is a wider search beam.

The figures below come from a one-off benchmark, not from the test suite: 2,000 synthetic clustered vectors at dimension=256, maxDegree=32, 50 queries scored against brute-force ground truth, with exact search reaching 0.976 for reference. The suite’s own testPqCompressionRecallStaysHigh pins a floor at dimension=64 with pqSubspaces=dimension/4; it is a regression guard, not the source of this table.

Subspaces beam 100 beam 200 beam 400 beam 800

dimension / 16 (16)

0.74

0.89

0.98

1.00

dimension / 8 (32)

0.94

0.99

1.00

1.00

dimension / 4 (64, the default)

0.97

1.00

1.00

1.00

At the default minSearchBeamWidth of 100, recall tracks M closely, and the dimension / 4 default is the safe choice — it is within noise of exact search. Going lower is worthwhile, but only together with a larger minSearchBeamWidth: dimension / 16 at beam 400 reaches 0.98 recall while its fused block is a quarter the size of dimension / 4, so it is both smaller and more accurate than the default at beam 100. Spending PQ’s cheaper hops on a wider beam is the trade the feature is built for.

Tune with a measured recall figure for your own data; these numbers come from synthetic clustered vectors and are indicative, not a guarantee.

Parallel On-Disk Writes

For large indices, writing a segment to disk can be slow. Enabling parallelOnDiskWrite(true) uses the OnDiskParallelGraphIndexWriter, which allocates parallel direct buffers and uses multiple worker threads (one per available processor) to write the segment graph concurrently. This can significantly speed up persists and merges of segments with millions of nodes.

VectorIndexConfiguration config = VectorIndexConfiguration.builder()
    .dimension(768)
    .similarityFunction(VectorSimilarityFunction.COSINE)
    .onDisk(true)
    .indexDirectory(Path.of("/data/vectors"))
    .approximateScoring(ApproximateScoring.FUSED_PQ)
    .pqSubspaces(48)
    .parallelOnDiskWrite(true) // Multi-threaded disk writes
    .build();

Sequential (single-threaded) writing is the default, which is preferable in resource-constrained environments or for smaller indices.

Production Configuration

For production systems with continuous updates, enable background persistence for an on-disk index, and background optimization for an in-memory one. An on-disk index needs no optimization: every persist cleans up the graph it writes, and the segments on disk are immutable.

// On-disk index
VectorIndexConfiguration onDiskConfig = VectorIndexConfiguration.builder()
    .dimension(768)
    .similarityFunction(VectorSimilarityFunction.COSINE)
    // On-disk storage
    .onDisk(true)
    .indexDirectory(Path.of("/data/vectors"))
    // Background persistence (async, non-blocking, enabled by setting interval > 0)
    .persistenceIntervalMs(30_000)        // Enable, check every 30 seconds
    .minChangesBetweenPersists(100)       // Only persist if >= 100 changes
    .flushThreshold(100_000)              // Write a segment once the in-memory graph holds this many nodes
    .persistOnShutdown(true)              // Persist on close()
    .build();

// In-memory index
VectorIndexConfiguration inMemoryConfig = VectorIndexConfiguration.builder()
    .dimension(768)
    .similarityFunction(VectorSimilarityFunction.COSINE)
    // Background optimization (periodic cleanup, enabled by setting interval > 0)
    .optimizationIntervalMs(60_000)       // Enable, check every 60 seconds
    .minChangesBetweenOptimizations(1000) // Only optimize if >= 1000 changes
    .optimizeOnShutdown(false)            // Skip for faster shutdown
    .build();

Manual Optimization and Persistence

For fine-grained control, you can manually trigger optimization and persistence.

// Optimize graph (removes excess neighbors, improves query latency; in-memory indices only)
index.optimize();

// Persist to disk (writes the in-memory graph as a new segment; on-disk indices only)
index.persistToDisk();

// Close index (runs shutdown hooks based on config)
index.close();
  • optimize() returns at once for an on-disk index (after applying pending eventual-indexing operations), and persistToDisk() does nothing for an in-memory index.

  • optimize(), persistToDisk() on an on-disk index, and close() must not be called while holding the GigaMap’s monitor, i.e. inside synchronized(map) or update/apply logic. They wait for work whose holders may need that monitor, and throw an IllegalStateException instead.

  • Shutting the storage down closes every index of the map, persisting it if persistOnShutdown is set; calling close() yourself is not required.

Multiple Vector Indices

You can create multiple vector indices for different embedding types on the same entity.

GigaMap<Document> gigaMap = GigaMap.New();
VectorIndices<Document> vectorIndices = gigaMap.index().register(VectorIndices.Category());

// Title embeddings (smaller dimension)
VectorIndexConfiguration titleConfig = VectorIndexConfiguration.builder()
    .dimension(384)
    .similarityFunction(VectorSimilarityFunction.COSINE)
    .build();
VectorIndex<Document> titleIndex = vectorIndices.add("title", titleConfig, new TitleVectorizer());

// Content embeddings (larger dimension)
VectorIndexConfiguration contentConfig = VectorIndexConfiguration.builder()
    .dimension(768)
    .similarityFunction(VectorSimilarityFunction.COSINE)
    .build();
VectorIndex<Document> contentIndex = vectorIndices.add("content", contentConfig, new ContentVectorizer());

Combine vector similarity search with traditional bitmap index filtering. VectorIndex.search(…​) returns a VectorSearchResult that is itself a GigaMap.SubQuery, so the two can be intersected directly — no manual id filtering required.

VectorSearchResult<Document> hits = vectorIndex.search(queryVector, 10);

// From the bitmap side — scores dropped
List<Document> hybridResults = gigaMap.query(categoryIndexer.is("technology"))
    .and(hits)
    .toList();

// From the scored side — keep iterating with similarity scores
ScoredSearchResult<Document> ranked = hits.and(
    gigaMap.query(categoryIndexer.is("technology"))
);

Vectorizer Implementations

isEmbedded() documents where the vector comes from; the index keeps a copy of every vector in its vector file either way (see Where the Vectors Live). The flag matters in two places: only an in-memory index with a computed vectorizer keeps its vector file in a configured directory (an embedded one re-reads the fields at each start), and reindex() holds the computed vectors on the heap while it rebuilds, where it re-reads embedded ones from the entities. In both cases vectorize() must reproduce the committed vector of a persisted entity, because the index calls it again to repair its vector file after a crash or when the file is missing.

Embedded Vectors

When the vector is a field of the entity, set isEmbedded() to true. Re-vectorizing is then cheap, so an in-memory index simply re-reads every vector from the entities at each start.

public class DocumentVectorizer extends Vectorizer<Document>
{
    @Override
    public float[] vectorize(Document entity)
    {
        return entity.embedding();
    }

    @Override
    public boolean isEmbedded()
    {
        return true;
    }
}

Computed Vectors

When vectors are computed on-the-fly or fetched from an external service, isEmbedded() returns false (the default). The computed vector is not persisted with the entity: the vector file is the only copy outside the vectorizer, so the service must return the same vector for the same persisted entity.

public class TextVectorizer extends Vectorizer<Document>
{
    private final EmbeddingService embeddingService;

    public TextVectorizer(EmbeddingService embeddingService)
    {
        this.embeddingService = embeddingService;
    }

    @Override
    public float[] vectorize(Document entity)
    {
        return embeddingService.embed(entity.text());
    }

    @Override
    public boolean isEmbedded()
    {
        return false; // Computed from the entity (the default)
    }
}
When using computed vectors with an external service, be aware of potential latency during indexing operations. Consider pre-computing embeddings and storing them in the entity for better performance. An in-memory index with a computed vectorizer and no indexDirectory/indexLocation uses a temporary vector file and calls the service once per entity at every start; configure a directory to keep the vector file across restarts.

Vector Keys

An update re-vectorizes the entity to find out whether its vector changed. For an expensive vectorizer, override vectorKey(entity) to return a cheap version key of the vector, for example a record version. An update whose entity has the same key as the indexed one is taken to leave the vector unchanged, and vectorize() is not called at all.

public class TextVectorizer extends Vectorizer<Document>
{
    // ...

    @Override
    public long vectorKey(Document entity)
    {
        return entity.textVersion(); // must change whenever the text changes
    }
}

The key must change whenever the vector changes, and it must be cheap, since it is asked for on every add and update. The default, Vectorizer.NO_KEY, compares vectors by content.

Changing an Entity’s Vector

Change what an entity’s vector derives from only through the GigaMap: update or apply for an in-place change, or set/replace with a new instance. A field mutated in place without them is not seen by the index, which keeps serving the old vector until a recovery or reindex() re-vectorizes the entity.

gigaMap.update(document, d -> d.setText("new text")); // the index re-vectorizes the document
By default the vectorize() method must never return null. If it does, an IllegalStateException is thrown when the entity is added, updated, or re-indexed — this fail-fast behavior catches accidental bugs in vectorizer implementations.

Entities Without an Embedding

If some entities legitimately have no embedding, override Vectorizer.allowsNullVectors() to return true. A null vector then means "this entity has no embedding": the entity remains in the GigaMap and all of its other indices, but is excluded from the vector index’s HNSW graph and never appears in vector search results. Its slot in the vector file stays empty, and getVector(entityId) returns null.

public class ProductVectorizer extends Vectorizer<Product>
{
    @Override
    public float[] vectorize(Product product)
    {
        return product.embedding(); // may be null for products not yet embedded
    }

    @Override
    public boolean allowsNullVectors()
    {
        return true;
    }
}

Vector transitions on update behave as follows:

  • non-null → null: the entity is removed from the vector index.

  • null → non-null: the entity is added to the vector index.

  • null → null: no-op.

An entity whose vector is null cannot be used as a similarity query (search(entity, k)); doing so throws IllegalArgumentException.

When eventual indexing is enabled, an entity whose vector was just set to null may still appear in search results for a short window until the background worker applies the graph mutation — the same consistency window that already applies to removals.

Batch Vectorization

When adding multiple entities via GigaMap.addAll(), the vectorizer’s vectorizeAll() method is called instead of vectorize() for each entity individually. The default implementation delegates to vectorize() one by one, but you can override it to use a more efficient batch API — for example, sending all texts to an embedding service in a single request.

The list returned by vectorizeAll() must have exactly the same size as the input, with positional correspondence (element i is the vector of entity i). When allowsNullVectors() returns true, individual elements may be null; otherwise every element must be non-null.

This is especially useful with external embedding APIs like LangChain4j, where a single batch call is significantly faster than many individual calls due to reduced network overhead and server-side batching.

public class ProductVectorizer extends Vectorizer<Product>
{
    private final EmbeddingModel embeddingModel;

    public ProductVectorizer(EmbeddingModel embeddingModel)
    {
        this.embeddingModel = embeddingModel;
    }

    @Override
    public float[] vectorize(Product product)
    {
        return embeddingModel.embed(product.name()).content().vector();
    }

    @Override
    public List<float[]> vectorizeAll(List<? extends Product> products)
    {
        List<TextSegment> segments = products.stream()
            .map(p -> TextSegment.from(p.name()))
            .toList();

        return embeddingModel.embedAll(segments).content().stream()
            .map(Embedding::vector)
            .toList();
    }
}

Using addAll() with a batch vectorizer:

EmbeddingModel embeddingModel = OllamaEmbeddingModel.builder()
    .baseUrl("http://localhost:11434")
    .modelName("all-minilm")
    .build();

GigaMap<Product> gigaMap = GigaMap.New();
VectorIndices<Product> vectorIndices = gigaMap.index().register(VectorIndices.Category());

VectorIndexConfiguration config = VectorIndexConfiguration.builder()
    .dimension(384)  // all-minilm dimension
    .similarityFunction(VectorSimilarityFunction.COSINE)
    .build();

vectorIndices.add("embeddings", config, new ProductVectorizer(embeddingModel));

// Batch add - calls vectorizeAll() once instead of vectorize() 100 times
gigaMap.addAll(products);
Batch vectorization works with any EmbeddingModel implementation from LangChain4j, including OpenAI, Cohere, Ollama, and in-process models. Just swap the model builder.

Benchmarking

The library includes benchmark tests following ANN-Benchmarks methodology.

# Run benchmark tests (disabled by default)
mvn test -Dtest=VectorIndexBenchmarkTest \
    -Djunit.jupiter.conditions.deactivate=org.junit.*DisabledCondition

Benchmark Results (10K vectors, 128 dimensions)

Metric Result

Recall@10 (clustered data)

94.3%

Recall@50 (clustered data)

100%

QPS (queries/second)

~10,000+

Average latency

< 0.1ms

p99 latency

< 0.2ms