Build a Self-Hosted AI Agent: Ollama + Open WebUI (2026)

To run a local ChatGPT alternative today: install Ollama (a desktop app on macOS and Windows, one script on Linux), start Open WebUI with a single docker run, open http://localhost:3000, create the first account (it becomes the admin) and pull a model such as llama3.2 or gemma3 from the UI. It works on a CPU; with an NVIDIA GPU you add --gpus all and the :cuda tag. Below: the Docker and pip installs, connecting the two, several models side by side, a small Python agent, and what changed since 2025.

Updated September 2026: commands checked against the Open WebUI 0.11.3 and Ollama 0.33.3 docs, a "what changed" note above the Docker block, current model names, a section on running it for a team (auth, users, exposure, backups, updates) and an FAQ. The original walkthrough and code are kept below.

Self-hosted is the easy half; making an agent reliable enough to ship is the harder half we teach in Cohorte's Building Accountable AI Agents course (E3).

1. Why Choose Ollama and Open WebUI?

Presentation Benefits

Intuitive Interface: Open WebUI offers a ChatGPT-like, responsive interface that works on desktops, laptops, and mobile devices.
Offline and Private: Running locally means your data stays secure and private, eliminating the dependency on third-party cloud services.
Multi-Model Flexibility: Manage and swap between various open-source models like Llama2, Llama3, and Gemma seamlessly.
Rapid Iteration: Whether via Docker or pip installations, you can have your environment running in minutes and easily update it using tools like Watchtower.

Real-World Use Cases

AI Chatbots & Assistants: Build custom conversational agents tailored to specific business workflows or personal productivity.
Local Knowledge Bases: Integrate document retrieval and local RAG (Retrieval-Augmented Generation) for private, on-premises assistants.
Developer Tools: Create code assistants integrated with your development environment, as demonstrated in popular open-source projects.

2. Supported Models and Advanced Options

Ollama acts as your model manager, letting you easily pull models from its library. Popular models include:

Llama2 and Llama3: Ideal for in-depth conversation and creative tasks.
Gemma Models: Lightweight options like gemma:2b deliver fast responses on modest hardware.
Custom Models: Use modelfiles to tailor models for specific tasks, allowing further fine-tuning on your local datasets.

With Open WebUI’s built-in pipelines and tools integration, you can even combine multiple models or integrate functions (like web search, code execution, or data retrieval) to create richer interactions.

3. Getting Started: Installation & Setup

A. Installing via Docker

What changed since this was written: the commands below still match the Open WebUI quick start (checked September 2026). The case this post skipped is now the common one: Ollama installed natively on the host (the macOS or Windows app) with Open WebUI in Docker. The README default for that adds a host alias so the container can reach port 11434; the GPU variant swaps the tag:

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

# NVIDIA GPU (NVIDIA Container Toolkit installed):
docker run -d -p 3000:8080 --gpus all --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:cuda

If the UI does not find Ollama, set the URL to http://host.docker.internal:11434 under Settings, Admin, Connections. The pip route now supports Python 3.11 and 3.12 (3.13 is not yet supported) and serves on port 8080, not 3000.

Using Docker is the quickest way to get started because it bundles dependencies and simplifies environment management. For example, to install Open WebUI bundled with Ollama (CPU-only), run:

docker run -d -p 3000:8080 \
  -v ollama:/root/.ollama \
  -v open-webui:/app/backend/data \
  --name open-webui \
  --restart always \
  ghcr.io/open-webui/open-webui:ollama

If your Ollama instance resides on another server, update the OLLAMA_BASE_URL environment variable:

docker run -d -p 3000:8080 \
  -e OLLAMA_BASE_URL=https://example.com \
  -v open-webui:/app/backend/data \
  --name open-webui \
  --restart always \
  ghcr.io/open-webui/open-webui:main

Tip: If you’re using a GPU-enabled setup, replace the image tag with :cuda and add --gpus all to the command. This approach is documented in the Open WebUI Quick Start guide.

B. Manual Installation via pip and uv

For users who prefer a non-Docker approach, install via pip with Python 3.11:

pip install open-webui
open-webui serve

For robust environment management, the recommended method is to use the uv runtime manager. On macOS/Linux, for example:

DATA_DIR=~/.open-webui uvx --python 3.11 open-webui@latest serve

This method isolates dependencies and minimizes conflicts, a practice highlighted in the official Open WebUI documentation (docs.openwebui.com).

C. Configuring Advanced Networking

If you plan to expose your Open WebUI interface externally:

Firewall Rules: Ensure that port 3000 (or your chosen port) is open on your server.
Reverse Proxy: For production, configure Nginx as a reverse proxy to secure your connection with HTTPS. A sample configuration might be:

server {
    listen 80;
    server_name openwebui.example.com;

    location / {
        proxy_pass http://127.0.0.1:3000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

hen use Certbot to obtain SSL certificates and secure your setup (see detailed steps in various guides like those on Vultr Docs docs.vultr.com).

4. Running Your First Model

Once installed, access Open WebUI at http://localhost:3000. During the first run, you’ll need to:

Create an Administrator Account: Follow the on-screen registration process.
Download a Model: Click on the settings icon, navigate to “Models”, and select a model (e.g., gemma:2b or llama2). Open WebUI will prompt you to download the model from Ollama (details are available in the Getting Started guide).
Test the Model: In the chat window, select your model and enter a prompt like “What is the future of AI?” to see it in action.

5. Building a Custom AI Agent: A Step-by-Step Example

A. Basic Command-Line Agent

Below is an extended example in Python to create a simple agent. This agent sends a prompt to an Ollama model and retrieves the response:

import subprocess

def run_model(prompt: str, model: str = "llama2") -> str:
    """
    Run the specified model via Ollama and return its response.

    :param prompt: The prompt to send to the model.
    :param model: The model tag (default: "llama2").
    :return: The model's response.
    """
    # Construct the command to run the model using Ollama CLI
    command = ["ollama", "run", model]
    try:
        # Launch the process and provide the prompt as input
        process = subprocess.Popen(
            command,
            stdin=subprocess.PIPE,
            stdout=subprocess.PIPE,
            stderr=subprocess.PIPE,
            text=True
        )
        stdout, stderr = process.communicate(input=prompt, timeout=30)
        if process.returncode != 0:
            return f"Error: {stderr.strip()}"
        return stdout.strip()
    except Exception as e:
        return f"Exception occurred: {e}"

# Example usage
if __name__ == "__main__":
    user_prompt = "Tell me a creative short story about the future of AI."
    response = run_model(user_prompt)
    print("Agent Response:", response)

Deep Dive:

Subprocess Handling: The script uses Python’s subprocess module to execute the Ollama CLI command and send a prompt.
Error Handling: Timeout and error capture ensure that your agent can gracefully report issues.
Model Flexibility: Easily switch the model tag to experiment with different LLMs.

B. Extending the Agent with Tools and Pipelines

For advanced users, integrate your agent with Open WebUI’s native tools. For instance, add a web search capability:

from duckduckgo_search import DDGS

def search_web(query: str) -> str:
    """
    Search the web using DuckDuckGo and return top 3 results.
    
    :param query: The search query.
    :return: A formatted string of results.
    """
    try:
        results = DDGS().text(query, max_results=3)
        return "\n".join([f"Title: {r['title']}\nURL: {r['href']}" for r in results])
    except Exception as e:
        return f"Web search error: {e}"

# Example integration
if __name__ == "__main__":
    search_query = "latest trends in AI"
    search_results = search_web(search_query)
    print("Search Results:\n", search_results)

Such tools can be integrated into Open WebUI as part of a larger pipeline, allowing your AI agent to augment its responses with live data.

6. Troubleshooting & Advanced Customization

Common Issues and Fixes

Connection Errors:
If Open WebUI cannot connect to Ollama (e.g., “Server Connection Error”), check your network settings. Using the --network=host flag in Docker can help if the container cannot reach the Ollama service running on 127.0.0.1:11434.

docker run -d --network=host -v open-webui:/app/backend/data -e OLLAMA_BASE_URL=http://127.0.0.1:11434 --name open-webui --restart always ghcr.io/open-webui/open-webui:main

Model Download Issues:
Ensure you have sufficient disk space and that your firewall isn’t blocking model downloads.
Performance Tuning:
Adjust the model parameters (like temperature and context length) via the Open WebUI Model Builder for better responses. More advanced users may integrate system resource monitoring to dynamically switch models based on load.

Customizing the Interface

Themes & Layouts:
Open WebUI supports custom themes and can be personalized through its settings. For example, use custom CSS or SVG icons to brand your interface.
Role-Based Access Control (RBAC):
In enterprise settings, manage user roles and permissions via Open WebUI’s RBAC features to secure your AI deployment.
Pipeline Integration:
Leverage the Pipelines Plugin Framework (discussed in various tutorials) to add extra functionalities such as voice input, dynamic document retrieval, or automated code execution.

Before this meets real users: The Agent Eval Playbook is the evaluation method we run on every agent before production. Free PDF.

What changed in Ollama and Open WebUI since this was written

Ollama. The current release is v0.33.3 (2 September 2026). Ollama is now a desktop app on macOS (14 Sonoma or later) and Windows, running in the background with the ollama CLI on your PATH; Linux keeps curl -fsSL https://ollama.com/install.sh | sh. The server still binds 127.0.0.1:11434 by default. llama2 and gemma:2b still pull but were last updated two years ago. The quickstart now opens with ollama run gemma4; on modest hardware, llama3.2 (1B or 3B) and gemma3:4b are the sane defaults, and gemma4 runs from e2b to 31b. Change model: str = "llama2" in the Python agent accordingly.

Open WebUI. The current release is v0.11.3 (31 August 2026). Two changes matter here. The licence: from v0.6.6 (April 2025) it is no longer BSD-3 and not OSI open source; still free to use and self-host, commercially too, but the branding stays unless you have 50 or fewer users in 30 days or an enterprise licence (details in the FAQ). The scope: it is no longer only an Ollama front end. The connections screen takes Ollama, OpenAI, Anthropic and any OpenAI-compatible endpoint, so one instance can mix local and cloud models (providers listed in the FAQ).

Taking this past a laptop? Self-hosted AI agents: the production stack covers the five layers above the model server, what self-hosting fixes and what it does not, costs for three sizes, and a 12-check readiness list.

Running it for a team, not just yourself

The defaults are team-shaped; the single-user shortcut (-e WEBUI_AUTH=False, login off, one user) belongs on a laptop only. With auth on, the first account created is the administrator; every later sign-up lands in pending (the DEFAULT_USER_ROLE) until an admin approves it, and ENABLE_SIGNUP=False closes registration entirely. Access is then roles, groups and per-resource permissions. For an existing directory the docs cover SSO, OIDC, LDAP and SCIM 2.0 provisioning. Set WEBUI_SECRET_KEY so sessions survive a container restart, and WEBUI_URL to the public address.

Exposure. Leave Ollama on 127.0.0.1:11434; only Open WebUI needs to be reachable. If Ollama must serve another machine, set OLLAMA_HOST=0.0.0.0:11434 (launchctl setenv on macOS, a systemd override on Linux, a user environment variable on Windows), restart it, and restrict who can reach that port at the firewall. Open WebUI goes behind the reverse proxy from section 3 with TLS.

Backups and updates. Everything lives in the open-webui volume: webui.db, uploads/, vector_db/, cache/ and audit.log. Find its host path with docker volume inspect open-webui and take the weekly copy with the container stopped. Updating is docker rm -f open-webui, docker pull ghcr.io/open-webui/open-webui:main, then the same docker run; the data is in the volume, not the container. If you run several replicas, update them all at once: rolling updates across schema changes are not supported.

FAQ

Is Open WebUI free?

Yes, free to download, self-host and use, including commercially. Since v0.6.6 (April 2025) the licence is not OSI open source: the Open WebUI branding must stay unless the instance has 50 or fewer users in a 30-day window, you are a contributor with written permission, or you hold an enterprise licence.

Do I need a GPU?

No. Both projects document CPU-only paths (the default :main image, the CPU :ollama bundle). A GPU makes larger models usable: on NVIDIA, install the Container Toolkit and use --gpus all with the :cuda tag. On a laptop, start with llama3.2:1b or gemma3:4b.

Can Open WebUI use OpenAI or Anthropic models too?

Yes. Settings, Admin, Connections accepts OpenAI, Anthropic and any OpenAI-compatible base URL and key (the docs name DeepSeek, Mistral, Groq, OpenRouter, Bedrock, Azure, and local llama.cpp, vLLM or LM Studio servers). Those chats leave your machine; keep sensitive work on Ollama models.

How do I update Open WebUI?

Docker: docker rm -f open-webui, docker pull ghcr.io/open-webui/open-webui:main, then the same docker run against the same volume. Compose: docker compose pull, then up -d. pip: pip install -U open-webui. Back up the volume first.

Can several people use one instance?

Yes, that is the default. The first account is the admin, later sign-ups wait in pending until approved, and access runs through roles, groups and per-resource permissions, with SSO, OIDC, LDAP and SCIM for company directories. Above 50 users in 30 days the branding clause applies.

Open WebUI vs LM Studio?

LM Studio is a desktop app (Apple Silicon Macs, Windows, Linux), free at home and at work since July 2025, with OpenAI-like endpoints on localhost or the network; it is one person's tool. Open WebUI is a web server with accounts, groups, RAG and plugins, and can use LM Studio as a backend. Alone on a laptop: LM Studio or plain Ollama. Anything shared: Open WebUI.

7. Final Thoughts and Future Directions

Combining Ollama with Open WebUI empowers you with a fully customizable, local AI platform that adapts to both personal and enterprise needs. Here are a few takeaways:

Modular and Extensible: The integration supports a wide range of models, making it ideal for varied tasks from casual conversations to complex workflows.
Secure and Private: Operating offline guarantees that sensitive data remains under your control.
Scalable Architecture: With built-in support for tools, pipelines, and filters, you can continuously enhance your system as new requirements emerge.

Looking Ahead

Enhanced Functionality: Future updates aim to improve tool integration and provide deeper customization options.
Community-Driven Improvements: The open-source nature of both projects means contributions from the community drive innovation and ensure rapid bug fixes.
Broader Model Support: As more open-source models become available, expect even more flexibility in tailoring your AI for specific use cases.

Whether you’re an individual developer or part of a large organization, this deep dive into using Ollama with Open WebUI offers the insight needed to build robust, self-hosted AI applications. Experiment, extend, and enjoy the journey of creating your own AI assistant!

Happy coding and exploring your AI ecosystem!

Charafeddine MouzouniMarch 31, 2025