Self-hosted model or vendor API: the five-line calculation
Running a model on your own hardware costs little per request and a lot in human time. The five lines of the comparison, and the three situations where self-hosting genuinely wins.
By Laurent TulpanPublished 9 min read
The question arrives early in automation projects and it is usually asked backwards: should we host a model ourselves or call a service?
Framed that way it invites a debate about sovereignty and cost. Framed properly it invites a calculation, and the calculation depends on what you are trying to do.
The five lines of the full comparison
One. Cost per request. The only line the usual comparison keeps. An API bills per use. A self-hosted model, once the machine is paid for, costs electricity. On this line, self-hosting wins as soon as volume is meaningful.
Two. The machine. A mid-sized model needs a graphics card with enough video memory, and that constraint bites harder than raw compute. It is bought, amortized, replaced, and it generates heat. It is also occupied by the model, which means it is not doing anything else.
Three. Human time. The decisive line, and the one that never appears. Installing, updating, monitoring, diagnosing why generation became slow for no visible reason, testing a new version of the model. In an organization without technical staff this time does not exist, which does not mean it is free: it means the maintenance will not be done. And where it does exist, it usually rests on one person, which is an exposure of its own.
Four. The cost of the quality gap. If a local model is wrong more often, somebody catches the errors. On a low-stakes task the gap is absorbable. On a task where an error is expensive, the additional verification time can cancel the entire saving.
Five. The cost of changing. A model called through an interface can be swapped for another in hours, if the call layer is properly isolated. A local installation represents hardware and human investment that nobody revisits every six months, and standing still has a cost in a field that moves quickly.
What self-hosting actually buys, and it is not the saving
Three things, and none of them is about price.
Control of location. The data do not leave. That is not a security argument in itself, since a server in a cupboard is not safer than a professional host. It is an argument about legal simplicity. No transfer, no additional processor to govern, no clause to verify.
Stability of behavior. A model installed on your hardware does not change until you change it. A model called over the network evolves, and an evolution can alter answers on cases you had already validated. On an industrialized process this is a real concern: what passed acceptance can behave differently three months later, which is exactly the kind of drift acceptance criteria are supposed to catch.
Independence from pricing. The price of a call is decided by somebody else. It has fallen substantially in recent years, but nothing guarantees the next few, and a usage embedded in a business process is a usage you can no longer stop easily.
What the API buys, and gets underestimated
No operations. Nothing to install, update or monitor. For an organization without technical staff, this line is frequently worth more than the difference in per-request price.
Access to the strongest models. On reasoning tasks, the gap between what runs on ordinary hardware and what is available over the network is real.
The freedom to be wrong cheaply. A trial that leads nowhere ends by revoking a key. The same trial run on purchased hardware leaves you with the hardware.
The three situations where self-hosting wins
An explicit constraint on data leaving. Verified, not assumed. Many organizations believe they are constrained and are not, and the reverse also happens. Where the constraint is real, it ends the discussion.
High volume on a simple, stable task. Classifying messages, extracting fields from documents, rewriting text. The task does not need the best model on the market, the volume is significant, and the calculation tips clearly.
Technical capability already present. If somebody already administers servers, the human time line changes nature: you are adding load to an existing capability rather than creating one.
Outside those three, the API is almost always the right starting point. Not on principle, but because it leaves the decision open.
The one precaution that keeps the question reversible
The worst choice is neither option. It is choosing irreversibly.
This is decided in one place, when you build. If your application calls the provider directly from ten places in the code, changing provider or moving in-house means revisiting all ten. If every call goes through a single internal layer, the change is confined to that layer.
The precaution costs almost nothing at build time and it changes the nature of the decision: starting with an API becomes a trial rather than a commitment. It is the same logic that makes a contract renegotiable, applied to architecture.
The question that comes first
Before comparing, you need to know what you are automating and at what volume.
Many projects settle self-hosting against API before establishing whether the task deserves automation at all, which is the wrong order. The real cost of an automation project rarely turns on the price of calls: it turns on preparing the data, integrating with the existing process, and verifying the output. On most of the work we do, the model is the smallest line in the budget, and it is the one that absorbs the entire discussion.
Which means the honest answer to “self-hosted or API” is often “that is not yet your question”, and the useful next step is the three questions that predict whether anything ships.
Further reading
- Why AI pilots stall before production: the questions that come before this one
- Measuring what an AI agent actually saves: what you find six months in
- Cloud or on-premise: the same calculation, on infrastructure
- When one person is your only backup: who maintains a local installation
- AI automation: how we scope this in practice
The questions this raises, answered straight.
Is a self-hosted model cheaper?
Per request, almost always, once the machine is paid for. On total cost it depends on volume and above all on human time.
A local installation needs somebody to maintain it, update it and diagnose it, and that time appears in no rate comparison. Below a certain request volume the API stays cheaper even paying for every call, because you are not paying for anyone to wait.
Are data sent to an API used to train the model?
That depends on the contract, and it is the first thing to check.
The business offerings of the major providers generally state that data sent through the programming interface are not used for training, unlike what may apply to free consumer versions. That commitment belongs in the processing agreement, not inferred from a help page that can change.
Is a self-hosted model less capable?
On complex reasoning, the strongest models remain available only through an interface, and the gap is real. On bounded tasks such as classification, extracting fields from a document, or rewriting text, models of modest size running on ordinary hardware are good enough.
The question is not which is better, it is which is sufficient for the task in front of you.
Can we start on an API and move to self-hosting later?
Yes, and it is often the right sequence, provided you built for it.
If the call to the model goes through one internal layer rather than being scattered through the code, changing provider or moving in-house becomes a localized replacement. Without that, the change means revisiting every call site, and what was meant to be a trial has become a commitment.
Recognize your situation? 20 minutes is enough to tell.
Describe what is happening on your side, and where your data and customers are. You get a plain answer on whether we can help, and a better route if there is one.

