AI series · Part 2: Retrieval (RAG)
Vector Databases, Top-K and Hybrid Search
A vector database finds the chunks closest in meaning to a question. Keyword search finds exact terms. Hybrid search uses both, and usually beats either one alone.
A vector database stores embeddings and quickly finds the ones closest to the question. Top-K is how many results come back: top-5 means the 5 closest.
Three kinds of search
| Type | Works like | Weak spot |
|---|---|---|
| Keyword search (BM25) | Ctrl+F: exact words only | Searching "vacation" won't find "annual leave" |
| Semantic search (vectors) | A smart librarian who understands meaning | Can miss exact codes and IDs |
| Hybrid search | Both together, results merged | Slightly more moving parts |
Hybrid usually wins. Keywords catch exact terms like a policy ID ("POL-TRV-07"), and vectors catch paraphrases ("dad" vs "paternity").
Common vector databases
Pinecone, Qdrant, Weaviate, Milvus, pgvector (inside Postgres) and Chroma. Our reference build uses Chroma for vectors and adds a keyword index alongside it for hybrid search.
I use hybrid search. Keywords catch exact terms like policy IDs, and vectors catch paraphrases.
- The short version
Next in the GenAI series: Re-ranking: The Second Opinion That Picks the Best Chunks
Start from the top: GenAI Quick Answers and Series Guide
We deploy these patterns across production engagements in fintech, healthcare, and eCommerce. Talk to a senior engineer about how they apply to your platform.