Vector DB, Part 2: HNSW Index Tuning and Hybrid Search

The Real Reason Vector DBs Get Slow

In Part 1 we covered embedding generation and basic search. At production scale, there is a point where accuracy and speed both fall apart. Index strategy and hybrid search are the solution.

Index Algorithm Comparison

Vector search performance is 90% determined by the choice of index algorithm.

Algorithm	Search speed	Accuracy	Memory	Suitable scale
Flat (brute force)	Slow	100%	Low	~100K items
IVF	Fast	95–98%	Medium	100K–10M
HNSW	Very fast	97–99%	High	All scales
ScaNN	Very fast	97–99%	Low	100M+ items

HNSW Parameter Tuning

Python

import hnswlib
import numpy as np

def build_hnsw_index(vectors: np.ndarray, dim: int) -> hnswlib.Index:
    index = hnswlib.Index(space="cosine", dim=dim)

    index.init_index(
        max_elements=len(vectors),
 ef_construction=200, # , (: 100~400)
 M=16, # . , (: 8~48)
    )
    index.add_items(vectors)

 # when ef: , (: ef_construction )
    index.set_ef(50)
    return index

# vs measure
def benchmark(index, queries, ground_truth, ef_values):
    for ef in ef_values:
        index.set_ef(ef)
        labels, _ = index.knn_query(queries, k=10)
        recall = compute_recall(labels, ground_truth)
        print(f"ef={ef:3d} → recall={recall:.3f}")

Hybrid Search: Semantics + Keywords

Pure vector search is weak on exact keywords, code, and proper nouns. Combining it with BM25 makes them complementary.

Python

from rank_bm25 import BM25Okapi
import numpy as np

class HybridSearcher:
    def __init__(self, docs: list[str], embeddings: np.ndarray):
        tokenized = [d.split() for d in docs]
        self.bm25 = BM25Okapi(tokenized)
        self.embeddings = embeddings
        self.docs = docs

    def search(self, query: str, query_emb: np.ndarray,
               alpha: float = 0.5, top_k: int = 10) -> list:
 # BM25
        bm25_scores = self.bm25.get_scores(query.split())
        bm25_norm = bm25_scores / (bm25_scores.max() + 1e-9)

 #
        cosine_scores = self.embeddings @ query_emb
        cosine_norm = (cosine_scores + 1) / 2  # [-1,1] → [0,1]

 # (RRF method )
        final = alpha * cosine_norm + (1 - alpha) * bm25_norm
        top_idx = np.argsort(final)[::-1][:top_k]
        return [(self.docs[i], float(final[i])) for i in top_idx]

How to Choose alpha

alpha 0.7–0.9: Semantics-first (general Q&A, conceptual search)
alpha 0.3–0.5: Keyword-first (code search, product names, dates)
alpha 0.5: Balanced (most RAG use cases)

Shrink the Search Space with Metadata Filtering

Python

# pgvector example — after
async def filtered_search(
    query_emb: list[float],
    department: str,
    date_from: str,
    limit: int = 10
) -> list:
    return await db.fetch("""
        SELECT id, title, content,
               1 - (embedding <=> $1::vector) AS similarity
        FROM documents
        WHERE department = $2
          AND created_at >= $3
          AND status = 'published'
        ORDER BY embedding <=> $1::vector
        LIMIT $4
    """, query_emb, department, date_from, limit)

Applying filters first reduces the candidate set and typically yields a 2–5× speedup.

Embedding Caching

Eliminates the cost of re-embedding the same query over and over.

Python

import hashlib
import redis

class EmbeddingCache:
    def __init__(self, ttl: int = 3600):
        self.redis = redis.Redis()
        self.ttl = ttl

    def _key(self, text: str, model: str) -> str:
        h = hashlib.sha256(f"{model}:{text}".encode()).hexdigest()[:16]
        return f"emb:{h}"

    async def get_or_embed(self, text: str, model: str, embed_fn) -> list[float]:
        key = self._key(text, model)
        cached = self.redis.get(key)
        if cached:
            return json.loads(cached)
        emb = await embed_fn(text)
        self.redis.setex(key, self.ttl, json.dumps(emb))
        return emb

In Part 3 we complete Vector DB operations with multi-tenant isolation, incremental index updates, and production monitoring.

Symptom-Based Tuning Decision Table

When you hit a performance issue in production, start from the symptom in the table below.

Symptom	Likely cause	First action
Low accuracy (recall)	ef_search too low / hybrid not applied	Double ef_search stepwise while measuring the recall–latency curve → if you hit a ceiling, tune alpha
p99 latency spikes	ef_search too high / filters applied after the index	Check whether metadata pre-filtering is supported; reset the ef_search cap
Memory keeps growing	HNSW graph (M) too large / deleted vectors not cleaned up	Consider reindexing with a lower M; periodic compaction/rebuild
Indexing takes too long	ef_construction too high	Lower it against your accuracy requirements — you can compensate with ef_search after indexing
Exact keyword matches are missed	Vector search only	Introduce hybrid (BM25 + vectors); tune alpha from search logs

Don't tune without measuring. The only trustworthy method is to record recall@k and p95 latency on the same golden query set (50–100 representative queries) before and after every parameter change.

#Vector DB #HNSW #BM25

✦ ✦ ✦

Sources and verification

Sources and checks vary by article. Only articles with a recorded check date, scope and sources below were checked, and only within that scope. For other articles, verify against the current docs for your version before applying anything, and email us about errors. Editorial policy · email us

This article has no recorded human check (date and scope) yet.

Published · May 23, 2026

Topic-related official docs (auto-linked · not per-claim sources)pgvector repository ↗

How we write and review →Editorial policy →

Was this helpful?

Comments

Be the first to comment.