Self-hosted AI agents: the production stack, what it costs, and what it does not fix

A self-hosted AI agent is an agent whose model, runtime and tools all run on infrastructure you control: your servers, your cloud account, your network rules. In 2026 the production version of that has five layers, and the model is the smallest of them. You need a model server (Ollama or vLLM), a runtime that owns the loop and its state, a tool boundary that decides which of the model's proposals become actions, a memory layer with a write policy, and an audit trail that records what crossed the boundary. Skip any of the last three and what you have is a chat window with shell access.

On cost, a single rented H100 runs about $1,500 to $2,900 a month around the clock on a specialist GPU cloud, and three to four times that on a hyperscaler. The hardware is never the expensive part. The expensive part is the person who keeps the thing honest. This guide walks the five layers, shows what self-hosting fixes and what it quietly leaves in place, puts numbers on three sizes of deployment, and ends with the 12-check readiness list we run before a self-hosted agent is allowed near anything real.

I have watched this go wrong the same way enough times that I can describe the room. A fintech team in Amsterdam, proud of a sentence they had earned: nothing leaves the building. Ollama on a rack server, Open WebUI in front of it, a tools plugin wired to a service account so the agent could pull documents for the analysts. The instinct was right. The data really did stay inside. Then an analyst asked the agent for "the latest on the Meyer account" and got a summary of a different client's file, because the service account could read every folder and nobody had told the agent otherwise. Nothing left the building. The wrong thing just reached the wrong desk inside it.

That is the whole argument of this piece in one story. Self-hosting moves the risk from the vendor to you. It does not remove it. Which is fine, honestly, as long as you noticed that you signed for it.

Why "self-hosted" is not the word for what you are worried about

Teams self-host for three reasons: data residency, cost at scale, and control. The first two are real and measurable. The third is where the confusion lives. You control the box. You do not, by owning the box, control the behaviour of what runs on it.

Most self-hosted agent setups I get asked to review are the same three pieces: a chat UI, a local model, and one tool with a broad credential. Sometimes the tool is a shell. The 24/7 personal agents that took off this year, OpenClaw being the loudest example, are exactly that pattern with a messaging channel on top. They are genuinely useful. They are also the reason a group of security researchers picked OpenClaw as the case study for MATRA, a threat-modeling framework for agentic systems, and found that the controls doing the real work were network sandboxing and least privilege, not anything about the model.

So the question is not "should we self-host". The question is "which of the five layers are we actually going to build". Here they are.

The five layers of a production self-hosted agent stack

The model server is the layer everyone argues about and the one that matters least to whether this survives contact with real users. I will cover it first because it is the fastest to settle, then spend the time where the incidents actually happen.

Model server: Ollama until there is a queue, vLLM the day there is one

Ollama (v0.33.3 as of September 2, 2026) is still the right first move for one person or a small team. One command, a model library, an OpenAI-compatible endpoint on localhost, and since the 0.33 line, native MLX on Apple silicon plus structured outputs. If you want the full setup with a UI in front of it, our Ollama and Open WebUI walkthrough covers it end to end, including the team-sized deployment notes.

vLLM (0.28 line, late August 2026) is what you move to when the second person is waiting on the first. Continuous batching and PagedAttention are the reason it exists; prefix caching, FP8 and INT4 quantization, structured outputs through xgrammar, and both an OpenAI-compatible server and an Anthropic Messages endpoint are why it became the default production server. Our vLLM production guide has the eight knobs that actually move throughput.

The rule I give teams: Ollama serves a person, vLLM serves a queue, and LM Studio serves a laptop. You do not need a benchmark to decide this. You need a headcount.

One thing worth saying out loud, because it changes how you budget: the runtime is where you will spend your engineering hours. The loop, the budgets, the typed terminal states, the escalation path the agent cannot control, that is the substance of the first weeks of Engineering Foundations, and it is the part most self-hosted setups skip because the chat UI already "works".

The tool boundary is the product

Here is the distinction that would have saved the Amsterdam team. A sandbox answers a reachability question: which files, processes, networks and credentials can this workload touch? Authorization answers a business question: which action may this principal take on which resource, for which purpose, right now? Neither one substitutes for the other. A process sandbox can block access to /etc and still let a permitted API refund the wrong account. An IAM policy can restrict refunds to the right tenant and still leave a shell able to read SSH keys.

Anthropic published the cleanest example of this I have seen, in its May 2026 write-up on how it contains Claude across products. A malicious file in a workspace instructed the agent to read the other files and upload them to the attacker's own account, through an allowed domain. The sandbox worked as configured. The domain check passed. The configuration had simply authorized too much, because a hostname is not a capability. The fix was to make the proxy accept only the session's own provisioned token and reject any credential the model supplied.

That post also has the number that should change how you think about approval prompts. Users approved roughly 93 percent of Claude Code's permission prompts. When Anthropic moved the boundary into an OS-level sandbox, prompts fell by 84 percent. The lesson is not that humans are careless. It is that a human asked to approve forty things an hour stops reading, and a reviewer who only sees the hardest two percent needs better context, not a bigger green button. Human attention is a control resource. Spend it on the decisions a deterministic boundary cannot make.

Three rules fall out of this, and they are the core of what we teach in the Security Engineering course:

• The run never holds the raw credential. A broker holds it, outside the model's context and ideally outside the sandbox, and binds the fields that matter (tenant, account, recipient, amount ceiling) from task state rather than from whatever the model wrote. The safest credential is the one the run never receives.
• Egress is a list of capabilities, not a list of domains. api.example.com contains upload, webhook, email and admin. Bind the account identity, restrict methods and endpoints, reject model-supplied authorization headers.
• Tools are executable trust relationships. A tool's description influences selection before it runs, its schema influences arguments, its output influences the next plan, and its update channel can change all three. Pin by digest. Requalify when the description changes.

Memory: the write policy matters more than the vector database

Persistent memory changes the attacker's horizon. They no longer need to control the current request; they can plant a record that will be retrieved later. Published attacks (MINJA, AgentPoison, PoisonedRAG) have shown this works through the ordinary query interface on evaluated memory-enabled agents, under their studied assumptions. So memory needs a write policy at least as strict as its read policy: provenance kept, tenant scope checked, instruction fields separated from factual claims, an expiry, a retraction path.

The one rule I would carve above the door: do not let the same model generate a lesson, declare it safe, promote it to shared memory, and retrieve it later as authority. Those are four different jobs. And when an incident happens, quarantine everything descended from the poisoned source. Rebuilding only the vector index leaves the summaries, plans and caches exactly as compromised as before, which is how a support engineer spends a weekend rebuilding an index and ships the poisoned summaries on Monday anyway.

Audit: log the boundary, not the person

The playbook principle is simple to state and easy to get wrong in both directions: log everything that crosses the middleware, log nothing that stays local. Every context request and what came back, every action submission and its outcome, every approval, every guardrail decision, cost per session. What stays out of the log is everything local: what the analyst typed to their own agent, the files on their laptop, the drafts they never sent.

The reason is not only privacy. It is adoption. A platform that feels like surveillance does not get resisted in meetings; it gets quietly circumvented, and then the data you self-hosted to protect is back to leaving the building through a personal phone. Over-logging is how you lose the thing you self-hosted for.

There is a second trade-off that Anthropic was unusually candid about: the same VM isolation that keeps the agent in also keeps the host's endpoint detection out. Isolation and visibility pull against each other. You will have to pick a point on that line. Pick it on purpose and write it down, rather than discovering it during an incident review.

What self-hosting fixes, and what it does not

Read the middle of that table twice. Every injection-class risk is identical on a self-hosted stack and on a hosted one, because the attack never needed the model's weights. It needed a path from untrusted text to a permitted action. That path is yours to close either way.

Build, fork, or buy the agent itself

Before the runtime, decide where the agent comes from. The Enterprise Agentic Platform playbook frames this as one of twelve decisions to make before writing code, and the table is worth stealing:

The playbook's recommendation, and mine: fork as the default client, and publish connectors so the people already living in Claude Code or Cursor can plug into the same boundary. Building from scratch is how a six-week project becomes a two-quarter project with a chat UI that still has no effect boundary at the end.

And one clarification that saves a lot of procurement pain: self-hosted is not the same as on-premises. Your own VPC with your own network rules counts. Start in cloud, move to hybrid when regulation asks for it, go fully on-prem only when your industry mandates zero external processing. Most teams who think they need the third option need the second.

The twelve decisions, the reference architecture and the 90-day plan are in The Enterprise Agentic Platform, a free PDF from our deployments.

What it costs: three sizes, with the line item people forget

GPU prices move monthly, so treat these as September 2026 anchors and recheck before you sign anything. Rental figures are on-demand rates from RunPod (July 2026) and CloudZero's H100 survey (updated August 2026). Monthly is 24/7 at those rates.

If you want to buy instead, a new H100 card is about $31,000 and an eight-GPU HGX node lands around $285,000, which is roughly seven to eight months of renting the same node. That arithmetic only works if the node is busy.

Which brings me to the line item. Cast AI looked at utilization across the clusters it manages and found an average of about five percent. At five percent, the effective cost of a delivered GPU-hour is twenty times the sticker price. That is the mechanism by which self-hosting becomes more expensive than the API you left: not the card, the idle card. Measure utilization at week four. If it is under twenty percent and not climbing, you do not have a self-hosting problem, you have a demand problem, and an API with a signed data processing agreement is the honest answer until demand exists.

The people column is the one that does not move with GPU prices, and it is the one that quietly decides whether the stack is still alive in a year.

The 12-check readiness list

This is the list we run before a self-hosted agent touches customer data. Each check is phrased so you can test it, because "we have a sandbox" is a feeling and "the run cannot reach the raw credential" is a fact you can demonstrate.

1. The model server is not on the internet by accident. Bound to localhost or a private interface, TLS through a reverse proxy, a token on every request. Test: try it from outside.
2. Every run has its own identity. Distinct from the human who asked and from the agent service. Test: find a run in the logs by its own ID without reading the transcript.
3. The run never holds the raw credential. A broker holds it and binds tenant, account and amount from task state. Test: grep the sandbox for the secret and find nothing.
4. Consequential actions pass an admission check. An allowlist of semantic operations with scopes, not hostnames. Test: ask the agent to refund a different tenant's account and watch it get denied with a typed reason.
5. Egress is a capability list. Service, account, method, endpoint family. Model-supplied authorization headers are rejected. Test: give the agent a file that asks it to upload elsewhere using a key in the file.
6. Tools are pinned and requalified. Digest on the package and the descriptor; a description change blocks loading until reviewed. Test: change one tool's description and confirm the run refuses it.
7. Memory writes carry provenance, scope and expiry. Instruction fields separated from claims; no self-promotion by the model that wrote them. Test: plant a note through the query interface and check it never becomes an instruction.
8. Every run has a budget and a named terminal state. Tokens, wall time, actions. Test: read last week's runs and confirm every one ended in a state you can name.
9. Revocation reaches in-flight runs. A kill switch that stops the loop, not just the next login. Test: pull the grant during a run this quarter, on purpose.
10. Audit sits at the boundary. Everything that crosses the middleware, nothing local, retained per policy, and a person reads it weekly. Test: name the person.
11. You have an eval set of your own tasks. Rerun after every model, tool or prompt change. Test: show the last run's date. The method is in The Agent Eval Playbook.
12. Someone owns it. A named person who can say stop, and a rota that answers when it breaks. Test: it is 3am, who is paged?

Scoring is blunt on purpose: twelve of twelve before it touches a customer, nine of twelve for internal read-only use. Below nine, it is a demo, and that is a perfectly good thing to have as long as everyone in the room calls it that.

What I'd do

Start with Ollama on the machine you already have, Open WebUI in front of it, and no tools at all for the first week, so people learn what the model is good at before it can act. Add one read-only tool through a broker, never directly. Write the twelve checks into a document before the second tool, because the document is the operating layer here, the thing the team actually runs on; that is the core idea behind the AI Operating System we teach, and it applies to a rack server as much as to a knowledge worker. Fork the agent, do not build it. Measure utilization at week four and be willing to turn the GPU off. And budget the person before you budget the card, because the card is a line on an invoice and the person is the reason the invoice keeps making sense. If you want the stack built properly rather than assembled, that is what the Engineering Series is for.

FAQ

What is a self-hosted AI agent?

An AI agent whose model, runtime and tools run on infrastructure you control, whether that is a server in your building or your own cloud account. The production version has five layers: model server, runtime, tool boundary, memory with a write policy, and boundary audit.

Is a self-hosted agent more secure than a cloud agent?

It is more private, which is different. Data residency improves. Prompt injection, tool poisoning and memory poisoning are unchanged, because those attacks need a path from untrusted text to a permitted action, not access to the model. Security comes from the tool boundary, wherever the model runs.

What hardware do you need to self-host an AI agent?

One person: a consumer GPU such as an RTX 4090 or 5090, or a recent Mac, for 8B to 32B quantized models. A team of ten: one L40S or A100 80GB. Production with concurrent users: H100-class GPUs on vLLM. Rental runs from under $1 to about $4 per GPU-hour on specialist clouds as of September 2026.

Ollama or vLLM for agents?

Ollama for one person or a small team with no queue; vLLM the day requests start waiting on each other. vLLM's continuous batching and PagedAttention are built for concurrency, and it exposes both OpenAI-compatible and Anthropic Messages endpoints.

Can a self-hosted agent still leak data?

Yes. The documented pattern is a file that instructs the agent to upload workspace data to an attacker's account through an allowed domain, using a key the file supplies. The fix is an egress policy that binds account identity and rejects model-supplied credentials, not a shorter allowlist.

How do you audit a self-hosted agent?

Log everything that crosses the middleware (context requests, action submissions, approvals, guardrail decisions, cost) and nothing that stays local to the user. Give every run its own identity so an incident can be reconstructed without reading transcripts, and name the person who reviews the log.

The Amsterdam team fixed their problem in a week, by the way, and the model never changed. They put a broker between the agent and the folders, scoped the service account per run, and wrote down who owned the pager. Nothing left the building before, and nothing leaves it now. The difference is that now the right thing reaches the right desk.

Self-hosting hands you the keys to the building. The locks are still your job.

Charafeddine MouzouniSeptember 5, 2026