Tools & UIs Reference
Tools & UIs Reference
Last verified: October 2026. The tools used to run and interact with local LLMs change constantly. Below is the current state of the main engines and user interfaces (UIs). Ollama was at v0.40.13 when this was checked.
The Core Engines (The Backend)
Engines do the heavy lifting: loading models into memory, determining GPU offloading, and exposing an API (usually OpenAI-compatible) so other apps can chat with them. You rarely interact with the engine directly unless you’re using the terminal.
1. Ollama
- Best for: Most users starting out. The standard starting point.
- Key Features: One-command installs (
ollama pull), MLX hardware acceleration for Macs, native tool calling, and simple setup. - Coding agents:
ollama launchsets up Claude Code, Codex, or OpenCode to run against a local or cloud model. - Platforms: Mac, Windows, Linux.
Quick Setup
curl -fsSL https://ollama.com/install.sh | sh
ollama pull qwen3.5:9b
The model name (qwen3.5:9b) comes from the Models reference page. Swap in any model listed there. On an 8 GB machine, try gemma4:e2b or qwen3.5:4b instead.
2. llama.cpp / llama-server
- Best for: Hardware tinkerers, low-resource machines, and those who want new quantization formats and KV-cache techniques as soon as they land.
- Key Features: A C/C++ backend optimized for low-level hardware access. If your hardware is unusual, llama.cpp probably supports it. It powers almost every other GUI behind the scenes.
- Platforms: Everything.
Quick Setup
# Mac
brew install llama.cpp
# Linux: build from source. See https://github.com/ggml-org/llama.cpp
llama-cli -m path/to/model.gguf
3. MLX Server (Apple Silicon Only)
- Best for: Mac users wanting maximum tokens-per-second out of Apple Unified Memory. Faster than llama.cpp on M-series for many models.
- Platforms: macOS with Apple Silicon (M1 and later).
Quick Setup
pip install mlx-lm
mlx_lm.server --model mlx-community/model-name
Replace model-name with any model available at mlx-community on HuggingFace.
4. vLLM
- Best for: Serving models at scale on NVIDIA GPUs. Not intended for local single-user use.
- Key Features: Production inference server with an OpenAI-compatible API. Supports tensor parallelism across multiple GPUs via a single flag. PagedAttention gives it higher throughput than naive serving.
- Platforms: Linux. NVIDIA GPUs are the main target; AMD and Intel GPUs and some CPUs are supported too.
Quick Setup
uv pip install vllm
vllm download-kernels
vllm serve Qwen/Qwen3.5-9B
Graphical Interfaces (The Desktop Apps)
If you don’t want to use the terminal, you want a GUI. These apps bundle an engine (usually llama.cpp) inside a chat interface.
1. LM Studio
- Best for: Individuals wanting a polished, out-of-the-box desktop experience.
- Key Features: Visual model downloading via HuggingFace, system specification checks (warns you if a model is too big), and LM Link, which connects your own devices over an encrypted Tailscale-based network so you can use models running on another machine.
- Platforms: Mac, Windows, Linux.
- Download: lmstudio.ai
Quick Setup
Download from lmstudio.ai. Open the app, go to the model browser, search for a model, and click Download. No terminal required.
2. Open WebUI
- Best for: Power users, organizations, and developers.
- Key Features: Runs in a browser via Docker. Granular Role-Based Access Control (RBAC), multi-user logins, integrated document RAG, and custom modular “Pipelines” for routing and prompt policies.
- Platforms: Self-hosted (Docker).
- Download: openwebui.com
Quick Setup
docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway ghcr.io/open-webui/open-webui:main
This connects to a running Ollama instance on the same machine. Open http://localhost:3000 in your browser.
3. Unsloth Studio
- Best for: Users who want to fine-tune models directly from their GUI.
- Key Features: Merges the chat experience with LoRA fine-tuning workflows, letting you adjust model behavior visually.
- Repo: github.com/unslothai/unsloth (Apache 2.0)
4. AnythingLLM
- Best for: Users who want to chat with their own documents without setting up a full RAG pipeline manually.
- Key Features: All-in-one desktop app for document Q&A (RAG). Supports multiple LLM backends including Ollama. No Docker required. Point it at a folder of PDFs or text files and start asking questions.
- Download: anythingllm.com
Low-Code Workflow UIs
If you want to orchestrate RAG pipelines or agents visually without writing Python:
Dify
- Best for: Production deployments and teams that want a managed platform.
- How it works: Cloud-hosted option available, also self-hostable. Built-in LLM orchestration, prompt management, and API publishing. Lets you deploy a working app from a visual flow without writing server code.
- Site: dify.ai
Langflow
- Best for: Rapid prototyping on your own machine.
- How it works: Python-based drag-and-drop flow builder you can run on your own machine. Lets you wire together document loaders, vector databases, LLMs, and tool-callers into a pipeline you can test locally.
- Site: langflow.org
Specialized Assistants
OpenClaw
- Repo: github.com/openclaw/openclaw (MIT)
- Best for: Users who want AI integrated into their operating system as a background personal assistant.
- How it works: Connects to services like WhatsApp and Slack, monitors for triggers, and can execute actions on your behalf. Generates and stores reusable “skills” locally so it doesn’t repeat the same reasoning twice.
Summary Comparison
| Tool | Type | Multi-User | GPU Tier | Target Use |
|---|---|---|---|---|
| Ollama | Backend Engine | No | Consumer | Developers / most users |
| LM Studio | Desktop App | No | Consumer | Solo users / researchers |
| Open WebUI | Web UI | Yes | Consumer | Teams / organizations |
| Unsloth Studio | Web / Desktop | No | Consumer | Fine-tuners |
| AnythingLLM | Desktop App | No | Consumer | Document Q&A |
| vLLM | Backend Engine | Yes | NVIDIA (multi-GPU) | Production inference |
| MLX Server | Backend Engine | No | Apple Silicon | Mac inference |
| llama.cpp | Backend Engine | No | Any | Tinkerers / embedded |
Related Pages
- Module 1: Get Running: Ollama installation walkthrough.
- Module 4: What Can You Do With This?: Setting up Open WebUI and AnythingLLM for everyday use.
- Module 8: Supercharge Your Setup: Connecting engines, UIs, and tools into a working local stack.