Every "run your own ChatGPT" tutorial online eventually hits the same wall: SSH into a box, install Docker, write a compose file, figure out why two containers can't see each other on the network, then do it again next time you want to update. None of that is actually hard, it's just tedious enough that most people give up and go back to paying per token.
Here's the same result - your own model, your own chat interface, nothing leaving your server - without the detour through a terminal.
What you're actually running
Two containers, talking to each other:
- Ollama - runs the model itself (Llama 3, Mistral, Gemma, Qwen, whatever you pull) and exposes a local API on port 11434.
- Open WebUI - a ChatGPT-style browser interface that calls that API instead of OpenAI's.
Deployed separately, wiring them together is the annoying part - Open WebUI needs to find Ollama's address on the Docker network, and that address changes depending on how you named things. Deployed as a stack, DeployOS wires it for you: Open WebUI's OLLAMA_BASE_URL is pre-set to http://ollama:11434, Docker's internal service discovery does the rest.
The one-click version
-
Open Templates in the DeployOS desktop app, find the AI Workbench stack (Ollama + Open WebUI + Flowise bundled).
-
Pick the server to deploy it on and click deploy. DeployOS generates the compose file, wires the network, and requests HTTPS for you the same way it does for any other app.
-
Open Ollama's console (or its API) and pull a model:
ollama pull llama3 -
Open Open WebUI's domain in a browser, create the first account (it becomes admin), and start chatting. The model is running on your server; nothing about the conversation goes anywhere else.
If you don't want Flowise yet, deploy Ollama and Open WebUI individually instead of the bundled stack - same result, one less container.
Sizing the server
A quantized 7-8B model (Llama 3 8B, Mistral 7B) runs on CPU alone on a 4-8GB VPS - usable for personal use, not fast. If you want response times that don't feel like a dial-up connection, you want a GPU-backed box; Ollama picks up an NVIDIA GPU automatically if the host has one, no extra configuration needed on the DeployOS side.
For most people the right move is starting on a cheap CPU box to confirm the whole stack works the way you want, then moving to GPU hardware once you know you'll actually use it day to day - DeployOS's server migration handles moving the app over without a from-scratch redeploy.
Why bother, if it's slower than GPT-4
It won't out-argue a frontier model. What it will do is answer questions about documents you don't want leaving your infrastructure, run at 2am without a usage meter, and keep working if a provider changes its pricing or goes down. For a lot of day-to-day use - drafting, summarizing, chatting with your own notes - that trade is worth it. And once it's running, Flowise or Langflow sit on top of the same Ollama instance to build something more specific without starting over.