Install Ollama + Open WebUI on Windows in 20 minutes

What you need

• Windows 10/11, ~20GB free disk space
• An NVIDIA GPU with 8GB+ VRAM (recommended — it also works on CPU, just slower)
• Docker Desktop (only for Open WebUI in step 5 — steps 1–4 need nothing else)

Step 1 — Check your GPU (1 min)

Open a terminal (Win+X → Terminal) and run the command above. You should see your GPU name, driver version and memory (e.g. "24576MiB"). Note the VRAM — it decides your model in step 3. If the command is not found, install the current NVIDIA driver first from nvidia.com and reboot.

Step 2 — Install Ollama (2 min)

Close and reopen the terminal, then verify:

Ollama now runs as a background service and starts with Windows. It stores models under your user profile (.ollama\models) — on a small C: drive, set the OLLAMA_MODELS environment variable to another disk before pulling models.

Step 3 — Pull a model sized to your VRAM (5 min)

The first run downloads the model (4–16GB), then drops you into a chat in the terminal. Type something; while it answers, run nvidia-smi in a second terminal — VRAM usage should have jumped. That confirms GPU inference. Model-by-model details (quants, speeds, tips): our local model guides.

Step 4 — Fix the context window (3 min, don't skip)

Ollama defaults to a small context (num_ctx), silently truncating long chats and big documents — the #1 cause of "the model forgot what I said". Create a file named Modelfile with:

(Swap the FROM line for your model from step 3.) Higher context costs VRAM — if generation slows dramatically or offloads to CPU, step back to 8192.

Step 5 — Open WebUI: the chat interface (5 min)

Install Docker Desktop (docker.com, default settings, WSL2 backend), then run:

Open http://localhost:3000 — the first account you create is the admin. Pick your model from the dropdown and chat. Everything (accounts, history, settings) persists in the open-webui Docker volume.

Step 6 — Documents (RAG) and LAN access (4 min)

• Document Q&A: in Open WebUI go to Workspace → Knowledge, upload PDFs/docs, then reference them in chat with #. The model answers from your documents with citations.
• Share on your network: Open WebUI is already reachable at http://YOUR-PC-IP:3000 from other devices — allow port 3000 in Windows Firewall. Family/colleagues get accounts on YOUR machine; nothing goes to the cloud.

Troubleshooting

• Generation is very slow — the model probably doesn't fit your VRAM: run "ollama ps" — it should say 100% GPU. If not, pick a smaller model from the step-3 table or reduce num_ctx.
• Open WebUI shows no models — the container can't reach Ollama. Verify the OLLAMA_BASE_URL uses host.docker.internal exactly as in step 5, and that "ollama list" works in a terminal.
• Port 3000 already in use — change the mapping to -p 3001:8080 and open localhost:3001.
• Download interrupted — just rerun the same ollama run command; downloads resume.

Where to go next

• Try other models — model-by-model local guides
• Serving a whole team? Move to vLLM (tutorial)
• Understand quants to pick better model versions
• Scale this into a company assistant