Ollama Model Explorer

Ollama is an open-source tool that lets you run large language models locally on your machine. It handles model downloading, GPU acceleration, and exposes an OpenAI-compatible REST API at localhost:11434. You can use it to run models like Llama 3, Mistral, Gemma, and hundreds more without any API key or internet connection.

Roughly: 7B models need 4–8 GB VRAM (or 8–16 GB RAM for CPU-only), 13B models need 8–16 GB VRAM, 70B models need 40–80 GB VRAM. For CPU-only inference, multiply by ~2x for RAM. Smaller models like Phi-3.5 (3.8B) and Llama 3.2 (3B) run well even on laptops with 8 GB RAM. Embedding models are very lightweight (under 1 GB).

Install langchain-ollama: pip install langchain-ollama. Then use: from langchain_ollama import ChatOllama; llm = ChatOllama(model='llama3.2'). You can also use any OpenAI-compatible SDK by pointing the base_url to http://localhost:11434/v1 — no API key needed for local models.

Chat models (Llama, Mistral, Gemma) are general-purpose instruction-following models. Code models (CodeLlama, DeepSeek Coder) are fine-tuned on code datasets and excel at generation, completion, and debugging. Vision models (Llama 3.2 Vision) accept both text and image inputs. Embedding models (nomic-embed-text, mxbai-embed-large) convert text to vector representations for semantic search and RAG — they don't generate text.

For RAG, you need two types of models: (1) An embedding model to convert documents to vectors — use nomic-embed-text or mxbai-embed-large. (2) A chat model to answer questions using retrieved context — Llama 3.1 8B or Mistral Nemo work well. The combination gives you a fully local RAG pipeline with no API costs.

Yes. Ollama can keep multiple models loaded simultaneously (subject to VRAM limits). By default it keeps the last used model in memory for faster subsequent requests. You can configure the keep_alive parameter per request. The REST API handles concurrent requests, so you can run multiple pipelines against different models at once.