Overview
A vector database stores high-dimensional float arrays (embeddings) and finds the closest ones to a query vector using Approximate Nearest Neighbor (ANN) search. It's the retrieval engine in every RAG system.
Why not just use PostgreSQL with a LIKE query? Semantic similarity lives in embedding space, not string space. "car" and "automobile" have different characters but near-identical embedding vectors. ANN search in a vector DB finds semantically related content; full-text search finds lexically similar content.
Key concepts:
- Embedding dimension: typically 768–3072 floats per vector
- Distance metric: cosine similarity (semantic), dot product (ranking), Euclidean (geometric)
- ANN index: HNSW or IVFFlat — trade recall for speed at billion-vector scale
- Metadata filtering: filter by tenant, date, category before ANN search
Architecture
┌─────────────────────────────────────────────────────────────┐
│ Vector DB Internals │
│ │
│ Incoming Vectors │
│ │ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ HNSW Index (Hierarchical │ │
│ │ Navigable Small World Graph) │ │
│ │ │ │
│ │ Layer 2 •─────────────────────────• │ │
│ │ \ / │ │
│ │ Layer 1 •──•─────────────•──────•──• │ │
│ │ | \ / | \ │ │
│ │ Layer 0 •────•──•──•───•──•──•───•────•────• │ │
│ │ (all nodes, dense connections) │ │
│ │ │ │
│ │ Search: start at top layer, greedily descend │ │
│ │ O(log N) hops to reach nearest neighbors │ │
│ └─────────────────────────────────────────────────────┘ │
│ │
│ Metadata Store (separate from vectors) │
│ ┌────────────────────────────────────────────────────┐ │
│ │ id │ tenant_id │ source │ created_at │ text │ │
│ │ 001 │ acme │ docs/ │ 2026-01-01 │ "The..." │ │
│ └────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────┘
ANN Query Flow:
┌─────────────────────────────────────────────────────────────┐
│ │
│ query_vector ──▶ Pre-filter metadata ──▶ ANN search │
│ [0.2, 0.8,...] (tenant = "acme" AND top-k results │
│ date > 2025-01-01) ranked by │
│ cosine sim │
└─────────────────────────────────────────────────────────────┘
Technical Implementation
Pinecone — Managed Vector DB
// src/vector-store/pinecone.client.ts
import { Pinecone, Index } from '@pinecone-database/pinecone';
export class PineconeVectorStore {
private client: Pinecone;
private index: Index;
constructor() {
this.client = new Pinecone({ apiKey: process.env.PINECONE_API_KEY! });
this.index = this.client.index(process.env.PINECONE_INDEX_NAME!);
}
async upsert(vectors: VectorRecord[]): Promise<void> {
// Pinecone recommends batches of 100
const batchSize = 100;
for (let i = 0; i < vectors.length; i += batchSize) {
await this.index.upsert(
vectors.slice(i, i + batchSize).map((v) => ({
id: v.id,
values: v.embedding,
metadata: {
text: v.text, // store raw text — no re-fetch needed
source: v.source,
tenantId: v.tenantId,
updatedAt: v.updatedAt,
},
}))
);
}
}
async query(embedding: number[], options: QueryOptions): Promise<SearchResult[]> {
const results = await this.index.query({
vector: embedding,
topK: options.topK ?? 8,
filter: {
tenantId: { $eq: options.tenantId },
...(options.sourceFilter && { source: { $in: options.sourceFilter } }),
},
includeMetadata: true,
});
return results.matches.map((m) => ({
id: m.id,
score: m.score ?? 0,
text: m.metadata!.text as string,
source: m.metadata!.source as string,
}));
}
async delete(ids: string[]): Promise<void> {
await this.index.deleteMany(ids);
}
}
pgvector — Postgres-Native (No Extra Infrastructure)
-- Enable pgvector extension
CREATE EXTENSION IF NOT EXISTS vector;
-- Table for embeddings
CREATE TABLE embeddings (
id TEXT PRIMARY KEY,
tenant_id TEXT NOT NULL,
source TEXT NOT NULL,
text TEXT NOT NULL,
embedding VECTOR(1536) NOT NULL, -- OpenAI text-embedding-3-small dimension
updated_at TIMESTAMPTZ DEFAULT NOW()
);
-- HNSW index for fast ANN search (pgvector 0.5+)
-- m=16 (graph connections per node), ef_construction=64 (build quality)
CREATE INDEX ON embeddings USING hnsw (embedding vector_cosine_ops)
WITH (m = 16, ef_construction = 64);
-- Partial index per tenant for better performance
CREATE INDEX ON embeddings USING hnsw (embedding vector_cosine_ops)
WHERE tenant_id = 'acme';
-- IVFFlat alternative (less memory, slightly lower recall)
-- CREATE INDEX ON embeddings USING ivfflat (embedding vector_cosine_ops)
-- WITH (lists = 100); -- lists ≈ sqrt(row_count)
// VectorSearchRepository.java — Spring JDBC with pgvector
@Repository
@RequiredArgsConstructor
public class VectorSearchRepository {
private final JdbcTemplate jdbc;
public void upsert(String id, String tenantId, String text, String source, float[] embedding) {
jdbc.update("""
INSERT INTO embeddings (id, tenant_id, text, source, embedding)
VALUES (?, ?, ?, ?, ?::vector)
ON CONFLICT (id) DO UPDATE
SET text = EXCLUDED.text, embedding = EXCLUDED.embedding, updated_at = NOW()
""",
id, tenantId, text, source, floatArrayToPgVector(embedding)
);
}
public List<SearchResult> findSimilar(String tenantId, float[] queryEmbedding, int topK) {
return jdbc.query("""
SELECT id, text, source,
1 - (embedding <=> ?::vector) AS similarity
FROM embeddings
WHERE tenant_id = ?
ORDER BY embedding <=> ?::vector
LIMIT ?
""",
(rs, i) -> new SearchResult(
rs.getString("id"),
rs.getString("text"),
rs.getString("source"),
rs.getDouble("similarity")
),
floatArrayToPgVector(queryEmbedding), tenantId,
floatArrayToPgVector(queryEmbedding), topK
);
}
private String floatArrayToPgVector(float[] arr) {
// pgvector expects: '[0.1,0.2,0.3]' string format
StringBuilder sb = new StringBuilder("[");
for (int i = 0; i < arr.length; i++) {
sb.append(arr[i]);
if (i < arr.length - 1) sb.append(",");
}
return sb.append("]").toString();
}
}
Choosing the Right Vector DB
Decision Tree:
Already using PostgreSQL?
YES ──▶ pgvector. Zero extra infra.
Good to ~10M vectors with HNSW.
NO ──▶ Need managed, serverless?
YES ──▶ Pinecone
- Fully managed, scales to
billions of vectors
- Strong metadata filtering
- $70+/mo for production pod
NO ──▶ Need hybrid (vector + BM25)?
YES ──▶ Weaviate or OpenSearch
- BM25 + dense vector
- Good for search UX
- Self-hosted or cloud
NO ──▶ On AWS, need VPC?
YES ──▶ OpenSearch
(k-NN plugin)
NO ──▶ Weaviate or
Qdrant
Hybrid Search — Dense + BM25
// Combine semantic search (embeddings) with keyword search (BM25)
// Better recall for queries with specific terms ("trail running shoe price")
export class HybridSearchService {
async search(query: string, tenantId: string): Promise<SearchResult[]> {
// Run both in parallel
const [denseResults, keywordResults] = await Promise.all([
this.vectorStore.query(await this.embed(query), { topK: 20, tenantId }),
this.bm25Index.search(query, { topK: 20, tenantId }),
]);
// Reciprocal Rank Fusion (RRF) — combine rankings
return this.rrf([denseResults, keywordResults], { k: 60 });
}
private rrf(resultSets: SearchResult[][], { k }: { k: number }): SearchResult[] {
const scores = new Map<string, number>();
for (const results of resultSets) {
results.forEach((result, rank) => {
const current = scores.get(result.id) ?? 0;
scores.set(result.id, current + 1 / (k + rank + 1));
});
}
// Merge and sort by RRF score
const all = [...new Map([...resultSets.flat()].map((r) => [r.id, r])).values()];
return all.sort((a, b) => (scores.get(b.id) ?? 0) - (scores.get(a.id) ?? 0));
}
}
Interview Preparation
Q: How does HNSW work?
HNSW (Hierarchical Navigable Small World) builds a multi-layer graph. Top layers are sparse — long-range connections for coarse navigation. Bottom layer is dense — all nodes, short connections for fine search. Query starts at the top layer's entry point, greedily moves to the nearest neighbor, descends to the next layer, repeats. Result: O(log N) hops to find approximate nearest neighbors. Trade recall (exact nearest) for speed (approximate).
Q: HNSW vs IVFFlat — when to use each?
HNSW: higher recall, faster queries, more memory (stores the graph). Best for < 100M vectors in RAM. IVFFlat: lower memory, works well for very large datasets when sharded. IVFFlat partitions the space into N clusters (Voronoi cells), searches only the most relevant clusters. Lower recall unless you probe many clusters.
Q: How do you handle multi-tenancy in a vector DB?
Option 1: Namespace per tenant (Pinecone namespaces, Weaviate classes) — total isolation, no cross-tenant bleed. Best for < 1000 tenants. Option 2: Metadata filter — one shared index, every query includes tenantId = X. Works at scale but requires the DB to support efficient pre-filtering before ANN (otherwise it scans everything). Option 3: Separate index per tenant — maximum isolation, painful to manage at scale.
Q: How do you keep the vector store in sync with source documents?
Event-driven: on document create/update, emit an event (Kafka/SQS) → indexing worker consumes it → re-embed → upsert. Track contentHash — only re-embed if the hash changed. On delete, delete the vectors by document ID. Never batch-rebuild on a schedule — that creates consistency windows.
Learning Resources
- pgvector GitHub — Postgres vector extension
- Pinecone Docs — managed vector DB
- Weaviate Docs — hybrid search
- HNSW Paper — original algorithm paper
- ANN Benchmarks — compare recall/QPS across indexes
- OpenSearch k-NN — AWS-native option