Haystack integration

July 31, 2026 · View on GitHub

turbovec.haystack.TurboQuantDocumentStore is a Haystack DocumentStore backed by an IdMapIndex. It implements the same public surface as haystack.document_stores.in_memory.InMemoryDocumentStore and can be used as a drop-in replacement wherever the in-memory store is used.

Install

pip install turbovec[haystack]

Basic usage

from haystack import Document
from turbovec.haystack import TurboQuantDocumentStore

store = TurboQuantDocumentStore()
store.write_documents([
    Document(content="...", embedding=[...], meta={"source": "a"}),
    Document(content="...", embedding=[...], meta={"source": "b"}),
])

results = store.embedding_retrieval(query_embedding=[...], top_k=5)

Documents must have pre-computed embeddings — TurboQuantDocumentStore doesn't invoke an embedder. Pipe a Haystack embedder component upstream if your documents arrive without embeddings.

Constructor

TurboQuantDocumentStore(
    dim: Optional[int] = None,
    bit_width: int = 4,
    *,
    embedding_similarity_function: Literal["dot_product", "cosine"] = "cosine",
    async_executor: Optional[ThreadPoolExecutor] = None,
    return_embedding: bool = False,
)

Parameter	Notes
`dim`	Optional. When omitted the vector dimensionality is inferred from the first `write_documents` call.
`bit_width`	Quantization width per coordinate; one of `{2, 3, 4}`.
`embedding_similarity_function`	The store's similarity mode — see Similarity modes. Selects both how vectors are stored (`"cosine"`, the default, normalizes; `"dot_product"` keeps them raw) and the `scale_score=True` formula on retrieval. Any other value raises `ValueError`.
`async_executor`	Optional `ThreadPoolExecutor` for the `*_async` methods. If omitted, a single-threaded executor is created and cleaned up with the store.
`return_embedding`	Accepted for API parity with `InMemoryDocumentStore`. The full-precision embedding is never available (quantized away), so `Document.embedding` on retrieved docs is always `None` regardless of the flag.

Similarity modes

embedding_similarity_function selects how scores are computed. It is fixed for the lifetime of the store:

"cosine" (default). Document embeddings are L2-normalized at write time and query embeddings at retrieval time, so raw scores are cosine similarity in [-1, 1] for embeddings of any magnitude, ranking matches InMemoryDocumentStore's cosine branch, and scale_score=True maps scores into [0, 1] via (s + 1) / 2 preserving order. Zero vectors are kept as-is and score 0 against everything (matching the reference, which substitutes a norm of 1 for zero-norm vectors).
"dot_product". Vectors are stored and queried raw: scores are raw inner products and ranking is magnitude-aware — matching InMemoryDocumentStore's dot-product branch. scale_score=True applies the reference's expit(s / 100) sigmoid.

`DuplicatePolicy`

write_documents takes a policy argument controlling how id collisions are handled:

from haystack.document_stores.types import DuplicatePolicy

store.write_documents(docs, policy=DuplicatePolicy.FAIL)      # raise if any id collides
store.write_documents(docs, policy=DuplicatePolicy.SKIP)      # silently skip colliding ids
store.write_documents(docs, policy=DuplicatePolicy.OVERWRITE) # remove-then-re-add colliding ids
# DuplicatePolicy.NONE is treated as FAIL.

Returns the number of documents actually written (so SKIP may return less than len(docs)).

Under FAIL (and NONE), documents are committed one at a time in batch order and the DuplicateDocumentError is raised on the first colliding id — every non-duplicate document before the collision stays persisted, matching InMemoryDocumentStore's post-exception state exactly.

Delete

store.delete_documents(["id-1", "id-2"])     # by id; missing ids are silently ignored
store.delete_by_filter(filters)               # by filter; returns count
store.delete_all_documents()                  # clear everything

delete_documents and delete_by_filter are O(1) per matching document via the inner IdMapIndex.

Filters

filter_documents(filters), embedding_retrieval(..., filters=...), and the other filter-aware helpers accept the full Haystack filter DSL:

filters = {
    "operator": "AND",
    "conditions": [
        {"field": "meta.source", "operator": "==", "value": "manual"},
        {"field": "meta.version", "operator": ">=", "value": 2},
    ],
}

# All docs matching the filter (no vector search):
docs = store.filter_documents(filters=filters)

# Top-k nearest to a query, filtered:
results = store.embedding_retrieval(
    query_embedding=[...],
    top_k=5,
    filters=filters,
)

Filter evaluation is delegated to haystack.utils.filters.document_matches_filter — anything Haystack's own stores support, we support.

embedding_retrieval validates query_embedding up front and raises ValueError("query_embedding should be a non-empty list of floats.") for an empty or non-numeric vector, matching InMemoryDocumentStore. A negative top_k also raises, where the reference returns n - 1 documents.

For embedding_retrieval, filters are resolved to an allowlist before scoring rather than via post-filtering. Selective filters return up to top_k matches from the filtered set; you never get fewer than top_k results just because the filter happened to exclude the top-scoring candidates.

Metadata helpers

store.count_documents_by_filter(filters)                          # int
store.count_unique_metadata_by_filter(filters, ["source", "tag"]) # dict[str, int]
store.update_by_filter(filters, {"reviewed": True})               # bulk metadata update; returns count

store.get_metadata_fields_info()
# {"source": {"type": "keyword"}, "version": {"type": "int"}, ...}

store.get_metadata_field_min_max("version")     # {"min": 1, "max": 5}
store.get_metadata_field_unique_values("source")
# (["a", "b", "c"], 3)

update_by_filter updates metadata only — embeddings are quantized at write time and not re-encoded.

Async

Every public method has an *_async variant:

await store.write_documents_async(docs)
results = await store.embedding_retrieval_async(query_embedding=q, top_k=5)
await store.delete_documents_async(["id-1"])

By default they run on a single-threaded executor owned by the store. Pass an async_executor= to the constructor to share an executor across stores (or to use more workers).

Save / load

store.save_to_disk("./my-store")
# ... later ...
store = TurboQuantDocumentStore.load_from_disk("./my-store")

Writes two files under the given folder path:

index.tvim — the IdMapIndex payload (quantized vectors + id maps).
docstore.json — JSON-encoded document text, metadata, and id maps.

Document metadata must be JSON-serializable — the same constraint InMemoryDocumentStore.save_to_disk imposes. If the docstore.json side-car is out of sync with its index.tvim (a partial copy, a stale backup, tampering), load_from_disk raises a ValueError immediately rather than failing later with a KeyError at query time.

save_to_disk is atomic with respect to the destination: both files are written to sibling temp files and moved into place, so a failed save (e.g. non-JSON-serializable metadata) leaves a store previously saved at the same path intact.

The store also supports pickle (e.g. for multiprocessing workers; the restored store owns a fresh async executor) and copy.copy / copy.deepcopy — both copies return a fully independent store (there is no shallow copy that shares the underlying index).

Using in a Haystack Pipeline

TurboQuantDocumentStore implements to_dict / from_dict so it can be serialized as part of a Haystack Pipeline. to_dict captures the component config (dim, bit_width, embedding_similarity_function, return_embedding); persisting the stored documents is the job of save_to_disk / load_from_disk.

Plug into a standard RAG pipeline the same way you'd use InMemoryDocumentStore. The sentence-transformers embedders live in their own integration package (pip install sentence-transformers-haystack, which requires haystack-ai 2.24 or newer):

from haystack import Pipeline
from haystack.components.writers import DocumentWriter
from haystack_integrations.components.embedders.sentence_transformers import (
    SentenceTransformersDocumentEmbedder,
)

store = TurboQuantDocumentStore()                 # dim inferred from first batch
indexing = Pipeline()
indexing.add_component("embedder", SentenceTransformersDocumentEmbedder(
    model="sentence-transformers/all-MiniLM-L6-v2",
))
indexing.add_component("writer", DocumentWriter(document_store=store))
indexing.connect("embedder.documents", "writer.documents")

indexing.run({"embedder": {"documents": my_docs}})

Thread safety

The store is safe for concurrent multi-threaded use:

Reads run concurrently and scale. embedding_retrieval, filter_documents, the count and metadata helpers, and storage take no lock; the underlying index releases the GIL during scoring, so independent retrievals from multiple threads overlap and scale.
Writes serialize. write_documents, delete_documents / delete_all_documents / delete_by_filter, update_by_filter, and save_to_disk serialize on a per-store lock. The *_async variants delegate to the same locked bodies.
A read overlapping a write sees pre- or post-write state — never a torn one. Under heavy concurrent churn a retrieval may transiently return fewer than top_k documents (hits deleted mid-retrieval are skipped).

What the contract does not cover:

No cross-call atomicity. A caller-side check-then-act sequence (count_documents then filter_documents) can interleave with other writers. Batch writes are not atomic with respect to readers: a retrieval overlapping an OVERWRITE write can briefly see a document id under both its old and new entry.
save_to_disk serializes with writes (so it always snapshots a consistent store); reads may proceed during a save.
to_dict / from_dict and the executor lifecycle are assumed single-threaded.
Two stores writing to the same path is safe. Concurrent save_to_disk calls to one destination from several threads each publish atomically and the last writer wins; a caller never sees a torn file, and never an error caused only by the other writer. Which writer wins is not defined.
Multi-process access is not supported.

Known limitations

Embeddings are not retained. embedding_retrieval(..., return_embedding=True) is accepted for signature compatibility but Document.embedding is always None on retrieved docs — turbovec discards the full-precision vector after quantization.
JSON-serializable metadata only. Document metadata is stored as JSON in the side-car. Non-JSON-serializable values (custom objects, sets, etc.) fail at save time — the same constraint InMemoryDocumentStore.save_to_disk imposes.
dim is locked on the first add. Subsequent calls with a different shape raise ValueError. If you need to change dim, construct a fresh store.