Is self-hosting an LLM actually cheaper than using an API?

Usually not at typical traffic levels. A 70B model in INT4 needs about 41.5 GB of VRAM, so one A100 80GB at $1.79/hour costs roughly $1,307/month before MLOps time. Serving the same traffic through a hosted API for the same 70B-class model (DeepSeek-V4.1-Flash at $0.30/M input, $1.20/M output) costs about $194/month at 432 million tokens. The GPU line alone is 7x the API bill. Self-hosting only wins once monthly token volume passes roughly 3 billion — the calculator computes that exact crossover for your configuration.

How much VRAM does a self-hosted LLM need?

VRAM scales with parameters and quantization: weights = parameters x bits / 8, plus scale overhead for quantized formats, plus about 15% for runtime and fragmentation. A 70B model is roughly 161 GB at FP16, 82 GB at INT8, and 41.5 GB at INT4 with group size 128 — that is the difference between a two-GPU A100 node and a single card. VRAM decides the floor on GPU count before any throughput calculation starts.

What costs do people forget when self-hosting?

Three: GPU idle time (you rent 730 hours but serve bursts), MLOps labor (model updates, quantization, autoscaling, monitoring — typically 4-10 engineer-hours a month), and peak provisioning, because throughput capacity must cover the busiest hour, not the monthly average. The calculator includes an explicit MLOps line and a peak-throughput check so the comparison is not just raw GPU rental.

When does self-hosting make sense?

Four cases: (1) token volume high enough to pass the crossover, typically billions of tokens per month; (2) data cannot leave your infrastructure for compliance or privacy reasons; (3) you run a fine-tuned model no provider hosts; (4) latency or availability requirements a shared API cannot meet. Cost alone rarely justifies it — the calculator shows the volume threshold for your exact configuration.

Does the comparison use equal-quality models?

Pick an API row in the same class as the model you would self-host, otherwise the comparison is meaningless. Comparing a self-hosted 70B against a frontier reasoning model overstates API cost, and comparing it against a small distilled model understates it. The calculator defaults to a 70B-class hosted API for a 70B self-hosted model, and the API table can be switched to the frontier model you actually use.

Tool Hub

📰📡

Home · JSON · Calculators · Articles · About

Self-Hosted LLM vs API

One A100 80GB rents for $1,307/month whether or not a single request arrives. The same 70B-class traffic through a hosted API costs $194/month — and you still have to operate the GPU. Self-hosting is not a cost decision at this scale; it is a control decision. This calculator sizes the VRAM, counts the GPUs against peak load, prices the MLOps time, and finds the monthly token volume where self-hosting actually wins.

Model & Serving

Pick the open-weight model you would host and its quantization. VRAM drives the GPU count: weights plus quantization scale overhead plus ~15% runtime headroom.

Model size

Quantization

GPU type (rental)

GPU hours per month

Traffic

Requests per day

Input tokens per request

Output tokens per request

MLOps hours per month

Engineer cost ($/hour)

Compare against API

Peak multiplier (busiest hour vs average)

📊 Cost Comparison

VRAM needed

GPUs (VRAM)

GPUs (peak I/O)

Tokens / month

Self-hosted

Hosted API

Why Self-Hosting Rarely Wins on Cost

The pitch for self-hosting sounds reasonable: renting a GPU costs less per hour than paying per token, so at volume you must come out ahead. The arithmetic says otherwise, because you pay for the GPU whether or not tokens flow through it. An A100 80GB at $1.79/hour for 730 hours is $1,307/month before a single request. A hosted 70B-class API serving the same traffic at $0.30 per million input tokens and $1.20 per million output tokens costs about $194/month at 432 million tokens. The GPU is not cheaper — it is roughly 7× more expensive, because the API provider spreads that same hardware across thousands of customers while you amortize it over one workload.

What Actually Drives the Number

Model size	FP16 VRAM	INT4 VRAM	Cheapest viable GPU	Monthly rental
8B	18.4 GB	4.7 GB	RTX 4090 24GB	$321
32B	73.6 GB	19 GB	L40S 48GB	$796
70B	161 GB	41.5 GB	A100 80GB	$1,307
405B	931.5 GB	240.2 GB	2× MI300X 192GB	$3,343

Quantization is what makes self-hosting possible at all: INT4 cuts a 70B model from 161 GB to 41.5 GB, the difference between a two-GPU node and a single A100. VRAM is only the entry ticket though. Throughput decides the real fleet size, because capacity must cover your busiest hour, not your monthly average — a 3x peak multiplier triples the concurrency you have to provision for.

The Costs That Do Not Appear on the GPU Invoice

Three items flip marginal self-hosting cases back to the API. Idle capacity: you rent 730 hours and serve bursts; utilization below 50% doubles your effective cost per token. MLOps labor: model updates, re-quantization, autoscaling policy, evaluation, and incident response typically run 4-10 engineer-hours a month — at $120/hour that is $480-1,200, often more than the GPU line itself at small scale. Evaluation drift: a quantized local model is not the API model, so quality regressions become your problem to detect. The calculator prices the first two explicitly rather than pretending they are free.

When Self-Hosting Is the Right Answer

Four situations justify it, and only one is about cost. Data residency: regulated data cannot leave your infrastructure, and no discount on an API fixes a compliance finding. Custom weights: you fine-tuned a model no provider hosts, so there is no API price to compare against. Latency and control: you need guarantees a shared API cannot offer. Scale: monthly token volume past the crossover, typically in the billions — where the fixed GPU cost finally amortizes across enough tokens that the per-token economics invert. Run your own numbers above; the answer for most teams is that the API is cheaper and the decision should be made on control, not cost.

Pricing Basis, Sources & Assumptions

Every rate on this page is a published vendor list price — no negotiated discounts, private pricing, or credit offsets. Totals are in USD and exclude tax. Rates were last checked against the sources below on 2026-09-23.

Pricing input	Basis used on this page
Region	GPU rental rates are the publicly listed hourly prices of the named rental marketplaces (US regions, on-demand, no committed contract). API rates are each vendor's global USD list price.
Currency	USD — on-demand list price, tax excluded
Last checked	2026-09-23 — The rental ladder — H100 SXM $2.99/hr, H100 PCIe $2.49/hr, A100 80GB $1.79/hr, A100 40GB $1.19/hr, L40S $1.09/hr, MI300X $2.29/hr, RTX 4090 $0.44/hr — was reviewed against the linked rental marketplaces on this date, as were the API comparison rates.

Modelling assumptions

Self-hosting is priced per GPU-hour against the marketplace's listed on-demand rate. The rate applies whether or not a request arrives, which is the entire shape of the break-even question.
Utilisation is the variable that decides the answer. A 730-hour always-on month and a 300-hour business-hours month cost very different amounts for identical hardware.
An open-weight model is not the same product as a frontier API model. The comparison is only meaningful against an API model in the same capability class, and the page labels which is which.
Quantisation changes both the hardware requirement and the output quality. INT4 fits larger models on the same card but is a quality trade, not a free saving.
MLOps hours are priced at the engineer rate entered, because operating inference is recurring labour, not a one-off setup task.

What this model excludes

Cluster networking between GPUs: the interconnect and fabric that tensor-parallel serving requires is not in the per-GPU rate.
Storage for weights, checkpoints, and logs, and the egress to serve them.
Redundancy, failover capacity, and the idle replica that keeps the endpoint up.
Model licensing terms for commercial use of open weights, which are not priced in the hourly rate.
Observability, evaluation, and guardrail infrastructure around the served model.

Sources

RunPod (2026). "RunPod GPU Pricing." Listed hourly rates for the GPU classes in the rental ladder. runpod.io
Vast.ai (2026). "Vast.ai Marketplace." Second rental marketplace used to bracket the hourly rates. vast.ai
Lambda (2026). "Lambda GPU Cloud." Third reference point for on-demand GPU-hour pricing. lambdalabs.com
DeepSeek (2026). "DeepSeek API Pricing." Per-million-token API rates used in the comparison. api-docs.deepseek.com
OpenAI (2026). "OpenAI API Pricing." Frontier API rates used as the upper comparison point. developers.openai.com
Anthropic (2026). "Claude API Pricing." Second frontier API rate card used in the comparison. platform.claude.com

Vendor list prices change without notice — re-check the linked pages before committing spend. jslet takes no vendor sponsorship and carries no affiliate links; see about.