AI series · Part 2: Retrieval (RAG)
Choosing an Embedding Model: Dimensions, Storage and Accuracy
More embedding dimensions mean better accuracy but more storage and slower search. Here's a comparison of popular models, what they cost at a million chunks, and three rules for choosing.
An embedding model is the AI that converts text into numbers. Dimensions is how many numbers each vector has.
It's like describing a person: 3 details (height, age, city) is quick but vague, while 100 details is precise but takes more space. More dimensions = more accurate, but more storage and slower search.
Popular models compared
| Model | Dimensions | Best for |
|---|---|---|
| all-MiniLM-L6-v2 | 384 | Small, fast, free (used in our reference build) |
| bge-base | 768 | Good free option |
| bge-large / Cohere | 1024 | Higher quality |
| OpenAI text-embedding-3-small | 1536 | Popular paid default |
| OpenAI text-embedding-3-large | 3072 | Best quality, most storage |
What it costs at 1 million chunks
| Dimensions | Storage |
|---|---|
| 384 | 1.5 GB |
| 1536 | 6.1 GB |
| 3072 | 12.3 GB |
Each dimension is a 4-byte number, so storage grows in a straight line with dimensions. Search speed and memory follow.
Three rules
- 1Use the same model for documents and questions. Two different models are like two different map systems: the coordinates don't line up.
- 2If you change the model, re-embed everything.
- 3Test, don't guess. Compare models on your golden eval set and keep the smallest one that hits your target.
In our reference build, swapping models is a one-line change, and the eval run tells us whether it was worth it:
python -m evals.run_evals --model BAAI/bge-base-en-v1.5I pick the smallest embedding dimension that meets my accuracy target on the golden eval, balancing accuracy against storage cost and speed.
- The short version
Next in the GenAI series: Vector Databases, Top-K and Hybrid Search
Start from the top: GenAI Quick Answers and Series Guide
We deploy these patterns across production engagements in fintech, healthcare, and eCommerce. Talk to a senior engineer about how they apply to your platform.