⚖️ Fine-Tuning vs RAG
The pricing page sells fine-tuning as a training bill: a few dollars per million training tokens, paid once. The invoice tells a different story. A fine-tuned model is billed at roughly 1.5× to 2× the base rate on every input and output token, forever — and the premium lands on output, which already costs several times what input costs. This calculator separates the one-time cost from the permanent one, then finds the retrieval length at which RAG overtakes it.
📐 Workload
Enter rates from your own contract. The comparison is between three paths on the same traffic: what you pay today, the same model plus retrieved context, and a fine-tuned model.
If you are fine-tuning a smaller model to replace a larger one, set the base prices to the small model's rates and keep today's prices as the large model's. That is the only configuration where fine-tuning usually wins on cost — and this calculator will show it.
The Cost the Pricing Page Buries
Hosted fine-tuning has two separate meters. The first is training: dollars per million training tokens, charged once per run, and genuinely small — a typical run on a few thousand examples lands in the tens of dollars. The second is inference on the tuned model, and it is charged at a premium over the base model, commonly 1.5× to 2×, applied to both input and output for the entire life of the product.
Marketing copy tends to quote the first meter and imply that is the cost of fine-tuning. The second meter is the one that scales with your business. Work the arithmetic on a support-desk workload at 20,000 tasks a month: the training bill might be $10, while the inference premium adds hundreds of dollars monthly, and it does not amortise, because it is a rate rather than a fee.
The premium also lands unevenly. Output tokens cost several times what input tokens cost, so a workload with long answers pays the fine-tuning tax on its most expensive component. A terse classifier pays comparatively little. Two teams can run the same number of tasks, on the same base model, with the same fine-tuning job, and see premiums that differ by an order of magnitude — purely because one of them writes paragraphs.
Why Retrieval Usually Wins on Cost
RAG's cost structure is the mirror image. Instead of raising the price of every token, it adds more input tokens: retrieved passages get prepended to each prompt. Input is the cheap side of the ledger, and it is also the side that prompt caching discounts, so a well-designed retrieval layer can reuse a large stable prefix at a fraction of list price.
This is why the two approaches diverge so sharply at scale. Adding 2,000 retrieved tokens to a prompt at an input rate of $0.40 per million costs $0.0008 per task. Applying a 2× premium to an output of 300 tokens at $1.60 per million costs $0.00096 per task — more, on a workload that produces almost nothing. As output length grows, the gap widens further, because the premium follows the output and retrieval does not.
There is a crossover, and it is computable. Fine-tuning becomes cheaper per task than retrieval only when the retrieved context is long enough that the input tokens it adds cost more than the premium applies. The threshold is:
Below that threshold, retrieval is cheaper per task. Above it, fine-tuning is. Note what the formula says about output length: the longer your answers, the higher the threshold climbs, because the premium is being paid on more expensive tokens. A workload with 1,500-token answers needs several times more retrieved context before fine-tuning breaks even.
The Only Case Where Fine-Tuning Reliably Saves Money
Fine-tuning to save tokens on the same model is almost always a losing trade, because the premium pushes every token in the wrong direction. The configuration that does work is different: fine-tuning a smaller model until it matches a larger one on your narrow task, then serving the small model.
The economics are straightforward once you see them. A small model fine-tuned to handle a narrow, well-specified task can land at 50–80% below the large model's base rate, and the fine-tuning premium is applied to that lower base. The saving comes from the model swap, not from the fine-tuning — fine-tuning is what makes the swap viable. That distinction matters, because it tells you what to test: whether the tuned small model clears your quality bar, not whether fine-tuning is cheaper in the abstract.
Set the base prices to the small model's rates and today's prices to the large model's, and the calculator will show whether your volume makes the swap worthwhile. Set both to the same model and it will show the premium eating the budget, which is the more common outcome and the one marketing pages tend not to illustrate.
Cost Is the Second Question: Fine-Tuning Is Not a Memory
The decisive argument against fine-tuning for knowledge is not financial. FineTuneBench, a study that tested commercial fine-tuning APIs directly rather than open models in a lab, measured roughly 37% accuracy when injecting new facts and 19% when updating facts the model already had. A separate evaluation of retrieval on comparable tasks scored far higher. Fine-tuning changes how a model responds; it does not reliably change what it knows.
This matters for cost modelling because the failure mode is silent. A retrieval system that cannot find a document fails visibly. A fine-tuned model that half-learned a fact fails by producing a confident, plausible, wrong answer — and you find out from a customer. Teams that fine-tune for knowledge often end up adding retrieval on top to fix the accuracy, at which point they are paying for both paths.
The defensible split is well established: retrieval for facts and freshness, fine-tuning for form. Format adherence, tone, output structure, domain-specific reasoning patterns — those are behaviour changes, they are what fine-tuning is genuinely good at, and they are the cases where the premium is buying something real.
Reading the Result
Per-token premium is the fine-tuned cost per task divided by today's cost per task. Above 1.0 means fine-tuning made every request more expensive; below 1.0 means the model swap is doing the work, and the fine-tuning premium is small enough not to undo it.
Training as % of fine-tuned spend is the reality check on the one-time cost. If this is below about 5%, the training bill is noise and the decision is entirely about the inference premium. That is the usual case, and it is why the training price gets quoted in marketing: it is the small number.
Crossover retrieval length is the number of retrieved tokens at which RAG and fine-tuning cost the same per task. If your actual retrieval is shorter, RAG is cheaper per task and the only remaining question is fixed costs. If it reports "fine-tuning cheaper at any length", the tuned model's per-task cost is already below retrieval's, and RAG cannot catch up on variable cost.
What This Model Leaves Out
Engineering time. This is usually the largest real cost and it is excluded on purpose, because it depends on your team's rates. A retrieval pipeline with permissions and evaluation typically runs 6–12 engineer-weeks; fine-tuning with data curation, evaluation and deployment runs 8–16. At fully loaded rates, those numbers frequently exceed the entire token bill for a year.
Data curation. Fine-tuning needs labelled input-output pairs — commonly 50 to 10,000 examples — and the labelling is done by people who understand the domain. That cost does not appear on any pricing page.
Retraining cycles. Modelled here as a count per year. In practice a retrain also requires a fresh evaluation pass and often a fresh labelling round, so the true cost per cycle is higher than the training bill alone.
Prompt caching on the retrieval side. Excluded, which makes RAG look more expensive than a well-built pipeline actually is. A stable retrieved prefix that hits the cache can be billed at a small fraction of list input price.
Latency. Retrieval adds a lookup step, typically tens to hundreds of milliseconds. Fine-tuning removes it. If your product is latency-sensitive, that is a real advantage for fine-tuning that no cost model captures.
Pricing Basis, Sources & Assumptions
This page does not assert vendor prices. Every rate is an input you control; the defaults are placeholders chosen to make the arithmetic legible, not a quote. Enter the rates on your invoice, including any negotiated or committed-use discount.
| Pricing input | Basis used on this page |
|---|---|
| Source | All per-token rates, training costs, embedding costs and vector-database fees are user-supplied and every result depends on what you enter. The two structural facts this model rests on — training is billed once per run, and hosted fine-tuned inference carries a per-token premium — follow the OpenAI API pricing documentation and the Anthropic pricing page, both re-read on . Default values are illustrative placeholders, not a price claim. |
| Region | Not applicable — API pricing is global per provider. Hosted vector databases do vary by region and by tier; use your plan's actual monthly bill. |
| Currency | USD, as entered by the user |
| Last checked | — the structural claims (training is one-off, tuned-model inference carries a per-token premium, retrieved context is input-priced) were re-confirmed against provider documentation on this date. No rate is asserted. |
Modelling assumptions
- Premium applies to both sides. The fine-tuned premium multiplies both input and output base rates. Providers that discount one side only would produce a lower figure than shown.
- Annualised at twelve months. Monthly figures are multiplied by twelve rather than by a per-month calendar. One-off costs (training, embeddings) are counted once, plus retrains.
- Retrieval adds only tokens. RAG is modelled purely as extra input tokens per task plus a fixed vector-database fee and a one-off embedding bill. Reranking, query expansion and hybrid search are excluded.
- Same task volume across paths. All three paths are priced on identical traffic. In practice a retrieval system may answer a query the fine-tuned model would have failed, which changes volume and is not modelled.
- Excluded. Engineering time, data labelling, evaluation harnesses, latency differences, prompt caching, batch discounts, hosting for self-managed models, free tiers, and any committed-use discount.