Text Generation WebUI (oobabooga): Local LLM Interface Guide
Text-generation-webui is the Swiss Army knife of local LLM interfaces. It supports more model formats than any other tool, provides granular control over every generation parameter, and has a powerful extension system for RAG, TTS, and API compatibility — making it the preferred choice for researchers and power users.
Related Products
on GumroadDeveloper Productivity Bundle
Stop rebuilding the same setup every project. 51 VSCode snippets, 40 Git aliases, 5 Docker Compose stacks, 5 GitHub Actions workflows, 5 README templates, and a dotfiles kit. Set up a professional dev environment in under an hour.
Buy on Gumroad →AI Prompt Engineering Toolkit
58 structured prompt templates for Claude, ChatGPT, and Gemini — organized by use case. Code Review, Debugging, Documentation, Architecture, and 10 model-specific tips. Stop writing prompts from scratch.
Buy on Gumroad →Developer Cheat Sheet Bundle
10 print-ready cheat sheets for the tools developers use every day: Git, Docker, Linux CLI, SQL, Regex, HTTP Status codes, VS Code shortcuts, Bash scripting, jq/JSON, and Markdown.
Buy on Gumroad →You Might Also Need
JSON Formatter & Validator
Format, beautify, and validate JSON online. Paste your JSON to instantly pretty-print it with 2-space, 4-space, or tab indentation. Minify JSON too. Free, runs in your browser.
Open free tool →
Regex Tester
Free regex tester. Test regular expressions against multiple strings with expected match/no-match assertions — see instant pass/fail results. Built-in presets for email, URL, IPv4, and phone. Pro: unl
Open free tool →
Base64 Encoder & Decoder
Free Base64 Encoder & Decoder. Convert text, images, and any file to Base64 and back. Supports URL-safe Base64, image data URIs, and file download from Base64. Runs 100% in your browser — no data sent
Open free tool →
More Free Tools
Related Articles
curl 深度使用手冊：API 測試從入門到進階
curl 是每位開發者必備的 CLI 工具。本文涵蓋 GET/POST/PUT/DELETE 請求、認證、header 設定、檔案上傳、除錯技巧，以及 REST API 測試的完整實戰範例。
Best AI Code Review Tools 2026: CodeRabbit, Sourcery, DeepCode & Bito Compared
Compare top AI code review tools for 2026: CodeRabbit, Sourcery, DeepCode, and Bito. Features, pricing, GitHub/GitLab integration, and choosing the right tool for your team.
AI Code Review Tools 2026: GitHub Copilot, CodeRabbit, Qodo Merge Ranked
Compare AI code review tools for 2026: GitHub Copilot code review, CodeRabbit, Qodo Merge, Sourcery, and PR-Agent. Pricing, accuracy, and GitHub integration.
AI Coding Tools Comparison 2026: GitHub Copilot vs Cursor vs Windsurf vs Cline
Comprehensive comparison of the best AI coding tools in 2026: GitHub Copilot, Cursor, Windsurf, and Cline. Pricing, features, model support, and real-world benchmarks.
AI Pair Programming in 2026: GitHub Copilot vs Cursor vs Claude Code
Master AI pair programming in 2026. Compare GitHub Copilot, Cursor, and Claude Code for autocomplete, chat, and agentic workflows. Includes setup guides and productivity tips.
Enjoying this tool? Get more like it.
Weekly developer tool guides, productivity playbooks & AI tricks. Free. No spam.
Unlock AI-Powered Dev Tools
- ⚡ AI Code Review, Doc Generator & SQL Builder
- ⚡ All premium templates & early access
- ⚡ Member discounts on Gumroad products
$9/mo after trial · Cancel anytime
Want the full toolkit?
Get DevPlaybook Pro
Every template, guide, boilerplate, and automation script in one bundle. 13 premium products — grab them all at once and save big.
MIT licensed · Instant download · No subscription
See what's included → Browse all deals →One-Click Installation
Text-generation-webui provides platform-specific installer scripts that set up a conda environment with the correct backends for your GPU:
# Download the latest release from GitHub:
# https://github.com/oobabooga/text-generation-webui/releases
# Windows — double-click or run from PowerShell:
start_windows.bat
# Linux
bash start_linux.sh
# macOS (Apple Silicon + Intel)
bash start_macos.sh
# The installer will ask for your GPU type:
# A) NVIDIA (CUDA)
# B) AMD (ROCm — Linux only)
# C) Apple Silicon (Metal)
# D) CPU only (no GPU)
# After installation, start the server:
python server.py --listen
# With API and OpenAI compatibility enabled:
python server.py --api --extensions openai --listen
# Access the UI at http://localhost:7860Loading Models (GGUF, GPTQ, AWQ, EXL2)
Place model files in the models/ directory. The UI auto-detects the format and selects the right loader. Here's what each format means and when to use it:
# Model format guide:
# GGUF — llama.cpp backend
# Best for: CPU inference, Apple Silicon, low VRAM scenarios
# Download from HuggingFace (TheBloke, bartowski repos are reliable)
# Place in: models/llama-3.2-7b-instruct-Q4_K_M.gguf
# GPTQ — AutoGPTQ backend (NVIDIA only)
# Best for: moderate VRAM (4-8 GB), fast NVIDIA inference
# Loader: AutoGPTQ (select in UI or --loader autogptq)
# Place in: models/Llama-3.2-7B-GPTQ/
# AWQ — AutoAWQ backend (NVIDIA only)
# Best for: faster than GPTQ, similar quality, 4-bit precision
# Loader: AutoAWQ
# Place in: models/Llama-3.2-7B-AWQ/
# EXL2 — ExLlamaV2 backend (NVIDIA only, fastest quantized option)
# Best for: maximum NVIDIA speed, various bits-per-weight settings
# Loader: ExLlamaV2 (set bits-per-weight in UI)
# Place in: models/Llama-3.2-7B-EXL2-4.0bpw/
# Load via UI: Models tab > Enter model name > Load
# Or command line: python server.py --model llama-3.2-7b --loader llama.cppKey Extensions
Enable extensions with --extensions ext1 ext2 at startup, or toggle them in the Session tab:
OpenAI-compatible API on port 5000. Drop-in replacement for OpenAI SDK integrations. Supports chat completions, completions, and embeddings.
Local RAG using ChromaDB. Upload documents and the model answers questions grounded in your data. Works entirely offline.
Generate images via Stable Diffusion (requires A1111 running). The LLM can create images on demand during conversations.
Persist conversation context across sessions using vector search. The model "remembers" past conversations with you automatically.
text-generation-webui vs Ollama
| Feature | text-gen-webui | Ollama |
|---|---|---|
| Model formats | GGUF, GPTQ, AWQ, EXL2, HQQ, FP16 | GGUF only |
| LoRA adapters | Yes | No |
| Sampling controls | 20+ parameters | Basic |
| OpenAI API | Via extension | Native |
| Ease of use | Moderate | Very easy |
| Installation | One-click script | Single binary |
| Best for | Power users, researchers | Developers, apps |
Frequently Asked Questions
What is text-generation-webui (oobabooga)?▼
Text-generation-webui, commonly called 'oobabooga' after its creator, is a feature-rich Gradio-based web interface for running LLMs locally. It supports the widest range of model formats (GGUF, GPTQ, AWQ, EXL2, HQQ) and inference backends of any local AI tool, making it the go-to choice for users who want maximum model compatibility and fine-grained control over generation parameters.
What model formats does text-generation-webui support?▼
Text-generation-webui supports GGUF (via llama.cpp), GPTQ (via AutoGPTQ), AWQ (via AutoAWQ), EXL2 (via ExLlamaV2 — fastest quantized format on NVIDIA), HQQ, and full-precision transformers models (FP16/BF16). This breadth of format support is unmatched by Ollama or LM Studio, making it essential for researchers and power users who work with diverse model quantizations.
How do I install text-generation-webui?▼
The easiest method is the one-click installer script for your OS (Windows, Linux, macOS). Download the release from GitHub, run the installer, and it sets up a conda environment with all dependencies. For Windows: run start_windows.bat. For Linux/macOS: run start_linux.sh or start_macos.sh. The installer auto-detects your GPU and installs the right CUDA/ROCm/Metal backend.
What is the OpenAI API extension?▼
The 'openai' extension (enabled in the --extensions flag) turns text-generation-webui into an OpenAI-compatible API server on port 5000 by default. It implements /v1/chat/completions, /v1/completions, and /v1/models. This lets LangChain, LlamaIndex, and any OpenAI SDK use your local models. Enable it with: python server.py --api --extensions openai.
What are text-generation-webui extensions?▼
Extensions are Python modules that add capabilities: 'api' exposes REST endpoints, 'openai' adds OpenAI compatibility, 'superbooga' implements local RAG with ChromaDB, 'sd_api_pictures' generates images via Stable Diffusion, 'gallery' shows model card information, 'long_term_memory' persists context across sessions, and 'silero_tts' adds text-to-speech. Load multiple extensions simultaneously.
text-generation-webui vs Ollama — when to use which?▼
Use Ollama when you want a simple, scriptable tool for running standard models in apps or scripts. Use text-generation-webui when you need: advanced quantization formats (GPTQ, AWQ, EXL2), fine-grained sampling controls (mirostat, repetition penalty, typical_p), LoRA adapter loading, grammar-constrained generation, or the superbooga RAG extension. Ollama is simpler; oobabooga is more powerful and configurable.
Related Products
on GumroadAI Prompt Engineering Toolkit
58 structured prompt templates for Claude, ChatGPT, and Gemini — organized by use case. Code Review, Debugging, Documentation, Architecture, and 10 model-specific tips. Stop writing prompts from scratch.
Buy on Gumroad →AI Developer Prompts Pack
55 battle-tested prompt templates for the workflows developers actually use: code review, debugging, architecture, documentation, and testing. Stop writing prompts from scratch.
Buy on Gumroad →Developer Productivity Bundle
Stop rebuilding the same setup every project. 51 VSCode snippets, 40 Git aliases, 5 Docker Compose stacks, 5 GitHub Actions workflows, 5 README templates, and a dotfiles kit. Set up a professional dev environment in under an hour.
Buy on Gumroad →You Might Also Need
AI Hallucination Detector
Scan LLM-generated text for hallucination risk signals: fabricated citations, overconfident claims, suspect statistics, and verification gaps. Risk score 0-100 with highlighted problem spans.
Open free tool →
AI Safety Checklist Generator
Generate a customized AI safety evaluation checklist for your deployment. Covers technical safety, content moderation, security, bias, privacy, and regulatory compliance. Export to Markdown.
Open free tool →
CrewAI Agent Config Generator
Build CrewAI agent configurations visually. Define roles, goals, backstories, and tools. Generate YAML or Python code for multi-agent crews.
Open free tool →
More Free Tools
Related Articles
LLM Observability Tools 2026: OpenTelemetry, Langfuse, Arize Phoenix Compared
Compare LLM observability tools for 2026: Langfuse, Arize Phoenix, Helicone, LangSmith, and OpenTelemetry for LLMs. Tracing, evals, cost tracking, and self-hosting options.
Prompt Engineering Techniques 2026: Chain-of-Thought, RAG, and Advanced Patterns
Advanced prompt engineering guide for 2026: Chain-of-Thought, Tree-of-Thought, RAG integration, few-shot learning, prompt chaining, and evaluation techniques.
LLM Observability 2026: LangSmith vs Helicone vs Arize Phoenix Compared
Complete guide to LLM observability platforms in 2026. Compare LangSmith, Helicone, and Arize Phoenix for tracing, evaluation, cost tracking, and debugging AI applications in production.
Prompt Engineering Techniques 2026: Chain-of-Thought, Few-Shot, and RAG Patterns
Master practical prompt engineering in 2026. Learn chain-of-thought prompting, few-shot examples, RAG integration patterns, and advanced techniques like tree-of-thought and self-consistency with real code examples.
AI Observability in 2026: How to Monitor, Debug, and Improve LLM Applications
A comprehensive guide to AI observability for LLM applications: key metrics, tool comparisons (Langfuse, Helicone, Arize Phoenix, LangSmith), tracing code examples, evaluation frameworks, and production best practices for debugging and cost control.
Enjoying this tool? Get more like it.
Weekly developer tool guides, productivity playbooks & AI tricks. Free. No spam.
Unlock AI-Powered Dev Tools
- ⚡ AI Code Review, Doc Generator & SQL Builder
- ⚡ All premium templates & early access
- ⚡ Member discounts on Gumroad products
$9/mo after trial · Cancel anytime
Want the full toolkit?
Get DevPlaybook Pro
Every template, guide, boilerplate, and automation script in one bundle. 13 premium products — grab them all at once and save big.
MIT licensed · Instant download · No subscription
See what's included → Browse all deals →