AnythingLLM: All-in-One Local AI with RAG & Agents

Docker Installation (Recommended)

AnythingLLM is best run via Docker. The container includes the frontend, backend, and a default SQLite + vector store — zero external dependencies required to start.

# Pull and run AnythingLLM
docker pull mintplexlabs/anythingllm

# Run with persistent storage
docker run -d \
  -p 3001:3001 \
  --cap-add SYS_ADMIN \
  -v $(pwd)/anythingllm_storage:/app/server/storage \
  --name anythingllm \
  mintplexlabs/anythingllm

# Open http://localhost:3001 and complete the setup wizard

# Docker Compose (recommended for teams)
# docker-compose.yml:
# version: '3.8'
# services:
#   anythingllm:
#     image: mintplexlabs/anythingllm
#     ports: ["3001:3001"]
#     cap_add: [SYS_ADMIN]
#     volumes: ["./storage:/app/server/storage"]
#     environment:
#       - SERVER_PORT=3001
#       - LLM_PROVIDER=ollama
#       - OLLAMA_BASE_PATH=http://host.docker.internal:11434

Connecting AnythingLLM to Ollama or LM Studio

In the AnythingLLM admin panel, navigate to Settings > LLM Preferences and select your backend. For Ollama running on the same machine as Docker, use the special Docker host URL:

# Ollama connection settings (in AnythingLLM UI):
# LLM Provider: Ollama
# Ollama Base URL: http://host.docker.internal:11434  (Docker on Mac/Windows)
# Ollama Base URL: http://172.17.0.1:11434            (Docker on Linux)
# Model: llama3.2 (or any model you have pulled)

# LM Studio connection settings:
# LLM Provider: LM Studio
# LM Studio Base URL: http://host.docker.internal:1234

# For embedding (required for RAG):
# Embedding Provider: Ollama
# Embedding Model: nomic-embed-text  ← pull this first
# ollama pull nomic-embed-text

# Verify connection by sending a test message in any workspace

Workspaces, Documents & RAG

Workspaces are isolated AI environments with their own document collections, model settings, and conversation history. Think of each workspace as a separate AI assistant with specialized knowledge.

# Workspace RAG workflow:
# 1. Create a workspace (e.g., "Company Docs", "Research", "Codebase")
# 2. Upload documents — supported formats:
#    PDF, DOCX, TXT, MD, CSV, JSON, XML, HTML, YouTube URLs, web URLs
# 3. AnythingLLM auto-embeds documents into the vector store
# 4. Ask questions — responses cite source documents

# Example questions after uploading a product manual PDF:
# "What are the troubleshooting steps for error code E5?"
# "Summarize the maintenance schedule from chapter 3"
# "Compare the specs of model A vs model B"

# Vector DB options (configure in Settings > Vector DB):
# - LanceDB (default, zero config, embedded)
# - Chroma (for external persistence)
# - Pinecone, Weaviate, Qdrant (cloud/self-hosted)

AI Agents: Tools & Capabilities

Enable agent mode in a workspace to give the LLM access to external tools. The agent can reason about which tools to use and chain multiple calls together:

AnythingLLM vs Open WebUI