Vector DB Schema Designer

Vector dimensions are the length of the numerical array that represents a piece of text, image, or other data as an embedding. The dimension is fixed by your embedding model — you cannot change it after creating an index. Common choices: 384 (all-MiniLM-L6-v2, fast and lightweight), 768 (all-mpnet-base-v2, good balance), 1536 (OpenAI ada-002 and text-embedding-3-small, most widely used), 3072 (OpenAI text-embedding-3-large, highest quality). Higher dimensions generally produce better retrieval quality at the cost of more storage and slower similarity search. Always match the dimension of your embedding model to the index dimension exactly.

Cosine similarity measures the angle between vectors — it ignores magnitude and only cares about direction. This is ideal for text embeddings where you want semantic similarity regardless of document length. Euclidean (L2) distance measures the straight-line distance between vectors — it is sensitive to magnitude. Use it for image or audio embeddings where intensity or scale carries meaning. Dot product is the fastest option but requires normalized vectors (unit length) to behave like cosine similarity. If your embeddings are already L2-normalized (most OpenAI and Sentence Transformers models output normalized vectors), cosine and dot product give equivalent results. When in doubt, use cosine — it is the safest default for text search.

Pinecone is a fully managed cloud service — zero infrastructure, instant scale, generous free tier. Best for teams that want to start fast without ops overhead. Downside: vendor lock-in, data leaves your infrastructure. Weaviate is open-source with a strong GraphQL API, built-in vectorizers, and multi-tenancy. Best for teams that want self-hosted control with rich schema features. Qdrant is open-source, written in Rust, extremely fast, and has excellent filtering capabilities. Best for high-performance self-hosted deployments. All three support cosine/euclidean/dot metrics, metadata filtering, and Python SDKs. For greenfield projects: Pinecone for speed, Qdrant for performance, Weaviate for rich schema/GraphQL needs.

Vector search alone returns the most semantically similar items across your entire dataset. Metadata filtering lets you pre- or post-filter results by structured attributes — for example, 'find the most similar documents, but only from source=docs.myapp.com and published after 2024'. This is critical for multi-tenant RAG systems (each user only sees their own data), time-bounded search, category filtering, and access control. All three supported databases (Pinecone, Weaviate, Qdrant) support efficient pre-filtering. In Qdrant, you define payload indexes for fields you plan to filter on. In Pinecone, metadata is stored as a JSON object alongside each vector. In Weaviate, properties are first-class schema objects with typed filtering.