Ollama: Run LLMs Locally on Mac, Windows & Linux

Installation

Ollama installs as a single binary with no dependencies. Choose your platform:

# macOS (Homebrew)
brew install ollama

# Linux (one-liner)
curl -fsSL https://ollama.com/install.sh | sh

# Windows
# Download the installer from https://ollama.com/download
# Or via winget:
winget install Ollama.Ollama

After installation, Ollama runs as a background service. Verify with ollama --version.

Pulling and Running Models

Use ollama pull to download a model and ollama run to start an interactive chat. Popular models and their approximate RAM requirements:

# Pull models (downloads to ~/.ollama/models)
ollama pull llama3.2          # Llama 3.2 3B — ~2 GB, fast on CPU
ollama pull llama3.2:7b       # Llama 3.2 7B — ~4.7 GB
ollama pull mistral           # Mistral 7B — ~4.1 GB
ollama pull gemma3            # Gemma 3 4B — ~3 GB
ollama pull phi4              # Phi-4 14B — ~8.9 GB
ollama pull deepseek-r1:7b    # DeepSeek-R1 7B — ~4.7 GB

# Start interactive chat
ollama run llama3.2

# Run with a one-shot prompt
ollama run mistral "Explain Docker in one paragraph"

# List downloaded models
ollama list

# Remove a model
ollama rm phi4

Models are stored in ~/.ollama/models on macOS/Linux and C:\Users\<user>\.ollama\models on Windows.

Using the Ollama REST API

Ollama starts an HTTP server on http://localhost:11434 automatically. It exposes two API styles: the native Ollama API and an OpenAI-compatible endpoint at /v1/chat/completions.

# Native Ollama API — streaming chat
curl http://localhost:11434/api/chat -d '{
  "model": "llama3.2",
  "messages": [{"role": "user", "content": "Why is the sky blue?"}],
  "stream": false
}'

# OpenAI-compatible endpoint (drop-in replacement)
curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama3.2",
    "messages": [{"role": "user", "content": "Write a Python hello world"}]
  }'

# Python with openai SDK pointing at Ollama
# pip install openai
from openai import OpenAI
client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")
response = client.chat.completions.create(
  model="llama3.2",
  messages=[{"role": "user", "content": "Hello!"}]
)
print(response.choices[0].message.content)

Modelfile: Custom Model Configuration

A Modelfile lets you create a named model variant with a custom system prompt, parameters, and prompt template — similar to a Dockerfile for LLMs.

# Create a file named "Modelfile"
FROM llama3.2

# Set a persistent system prompt
SYSTEM """
You are a senior software engineer who gives concise, production-ready
code examples with minimal explanation. Always use modern best practices.
"""

# Tune generation parameters
PARAMETER temperature 0.3
PARAMETER num_ctx 8192
PARAMETER top_p 0.9

# Build and run
# ollama create dev-assistant -f Modelfile
# ollama run dev-assistant "Write a FastAPI endpoint for user auth"

Custom models appear in ollama list alongside downloaded models and can be shared with ollama push.

Ollama vs LM Studio vs llama.cpp