AI series · Part 3: Evals and Accuracy
Which Model for Which Job: Picking LLMs by Task, Cost and Evals
Use the smallest, cheapest model that still passes your eval. A practical map of which model types fit which tasks, plus model routing.
Choose the smallest, cheapest model that still passes your eval. It's like choosing a vehicle: a bicycle to buy milk, a car for a family trip, a truck to move house.
Task → model
| Task | Model |
|---|---|
| Complex reasoning, coding, agents | Big model (Claude Opus, GPT-5, Gemini Pro) |
| Simple high-volume tasks (sorting emails, extracting fields) | Small model (Claude Haiku, GPT-mini, Gemini Flash) |
| Data can't leave the company | Open-source on your own servers (Llama, Mistral) |
| Search | Embedding model |
| Re-ranking | Cross-encoder |
Model routing
A support team uses a small model to sort 10,000 tickets a day, which is cheap, and sends only the hard complaints to a big model, which is smart. This is called model routing, and it's often the biggest single cost saving in a GenAI system.
I pick models by task: small models for high-volume simple work, big models for reasoning and agents, open-source when data must stay in-house. The final choice is based on eval results, cost and speed.
- The short version
Next in the GenAI series: AI Agents: From Chatbots That Talk to Assistants That Act
Start from the top: GenAI Quick Answers and Series Guide
We deploy these patterns across production engagements in fintech, healthcare, and eCommerce. Talk to a senior engineer about how they apply to your platform.