Is self-hosting an LLM actually cheaper than using an API?

Usually not at typical traffic levels. A 70B model in INT4 needs about 39 GiB of VRAM, so one A100 80GB at $1.79/hour costs roughly $1,307/month before MLOps time. Serving the same traffic through a hosted API for the same 70B-class model (DeepSeek-V3 at $0.27/M input, $1.10/M output) costs about $176/month at 432 million tokens. The GPU line alone is 7x the API bill. Self-hosting only wins once monthly token volume passes roughly 3 billion — the calculator computes that exact crossover for your configuration.

How much VRAM does a self-hosted LLM need?

VRAM scales with parameters and quantization: weights = parameters x bits / 8, plus scale overhead for quantized formats, plus about 15% for runtime and fragmentation. A 70B model is roughly 150 GiB at FP16, 76 GiB at INT8, and 39 GiB at INT4 with group size 128 — that is the difference between a two-GPU A100 node and a single card. VRAM decides the floor on GPU count before any throughput calculation starts.

What costs do people forget when self-hosting?

Three: GPU idle time (you rent 730 hours but serve bursts), MLOps labor (model updates, quantization, autoscaling, monitoring — typically 4-10 engineer-hours a month), and peak provisioning, because throughput capacity must cover the busiest hour, not the monthly average. The calculator includes an explicit MLOps line and a peak-throughput check so the comparison is not just raw GPU rental.

When does self-hosting make sense?

Four cases: (1) token volume high enough to pass the crossover, typically billions of tokens per month; (2) data cannot leave your infrastructure for compliance or privacy reasons; (3) you run a fine-tuned model no provider hosts; (4) latency or availability requirements a shared API cannot meet. Cost alone rarely justifies it — the calculator shows the volume threshold for your exact configuration.

Does the comparison use equal-quality models?

Pick an API row in the same class as the model you would self-host, otherwise the comparison is meaningless. Comparing a self-hosted 70B against a frontier reasoning model overstates API cost, and comparing it against a small distilled model understates it. The calculator defaults to a 70B-class hosted API for a 70B self-hosted model, and the API table can be switched to the frontier model you actually use.

Self-Hosted LLM vs API Cost — Breakeven Calculator

Self-Hosted LLM vs API

One A100 80GB rents for $1,307/month whether or not a single request arrives. The same 70B-class traffic through a hosted API costs $176/month — and you still have to operate the GPU. Self-hosting is not a cost decision at this scale; it is a control decision. This calculator sizes the VRAM, counts the GPUs against peak load, prices the MLOps time, and finds the monthly token volume where self-hosting actually wins.

Model & Serving

Pick the open-weight model you would host and its quantization. VRAM drives the GPU count: weights plus quantization scale overhead plus ~15% runtime headroom.

Model size

Quantization

GPU type (rental)

GPU hours per month

Traffic

Requests per day

Input tokens per request

Output tokens per request

MLOps hours per month

Engineer cost ($/hour)

Compare against API

Peak multiplier (busiest hour vs average)

📊 Cost Comparison

VRAM needed

GPUs (VRAM)

GPUs (peak I/O)

Tokens / month

Self-hosted

Hosted API

Why Self-Hosting Rarely Wins on Cost

The pitch for self-hosting sounds reasonable: renting a GPU costs less per hour than paying per token, so at volume you must come out ahead. The arithmetic says otherwise, because you pay for the GPU whether or not tokens flow through it. An A100 80GB at $1.79/hour for 730 hours is $1,307/month before a single request. A hosted 70B-class API serving the same traffic at $0.27 per million input tokens and $1.10 per million output tokens costs about $176/month at 432 million tokens. The GPU is not cheaper — it is 7x more expensive, because the API provider spreads that same hardware across thousands of customers while you amortize it over one workload.

What Actually Drives the Number

Model size	FP16 VRAM	INT4 VRAM	Cheapest viable GPU	Monthly rental
8B	17 GiB	4.4 GiB	RTX 4090 24GB	$321
32B	68 GiB	17.7 GiB	L40S 48GB	$796
70B	150 GiB	39 GiB	A100 80GB	$1,307
405B	868 GiB	224 GiB	2× MI300X 192GB	$3,343

Quantization is what makes self-hosting possible at all: INT4 cuts a 70B model from 150 GiB to 39 GiB, the difference between a two-GPU node and a single A100. VRAM is only the entry ticket though. Throughput decides the real fleet size, because capacity must cover your busiest hour, not your monthly average — a 3x peak multiplier triples the concurrency you have to provision for.

The Costs That Do Not Appear on the GPU Invoice

Three items flip marginal self-hosting cases back to the API. Idle capacity: you rent 730 hours and serve bursts; utilization below 50% doubles your effective cost per token. MLOps labor: model updates, re-quantization, autoscaling policy, evaluation, and incident response typically run 4-10 engineer-hours a month — at $120/hour that is $480-1,200, often more than the GPU line itself at small scale. Evaluation drift: a quantized local model is not the API model, so quality regressions become your problem to detect. The calculator prices the first two explicitly rather than pretending they are free.

When Self-Hosting Is the Right Answer

Four situations justify it, and only one is about cost. Data residency: regulated data cannot leave your infrastructure, and no discount on an API fixes a compliance finding. Custom weights: you fine-tuned a model no provider hosts, so there is no API price to compare against. Latency and control: you need guarantees a shared API cannot offer. Scale: monthly token volume past the crossover, typically in the billions — where the fixed GPU cost finally amortizes across enough tokens that the per-token economics invert. Run your own numbers above; the answer for most teams is that the API is cheaper and the decision should be made on control, not cost.