Prompt Engineering Techniques 2026: Chain-of-Thought, Few-Shot, and RAG Patterns

Prompt engineering has evolved from “write better instructions” into a discipline with documented patterns, failure modes, and measurable outcomes. The gap between a mediocre LLM application and a great one is often not model choice or architecture — it’s prompt quality.

This guide covers the core patterns every AI developer needs in 2026: chain-of-thought reasoning, few-shot examples, RAG integration, and the newer techniques that separate hobbyists from professionals.

Foundations: Why Prompts Matter More Than You Think

Before techniques, a mental model. LLMs are next-token predictors trained on human-generated text. When you write a prompt, you’re setting up a context that the model will try to “complete” in a way consistent with its training data.

This means:

• Prompts that resemble well-written examples from training data → better outputs
• Prompts that are ambiguous → the model picks a plausible interpretation (often wrong)
• Prompts that constrain output format → more reliable, parseable responses
• Role/persona framing → activates different knowledge clusters in model weights

With that foundation, let’s look at the patterns.

---

1. Zero-Shot Prompting

The baseline. You describe the task and expect the model to perform it without examples.

from openai import OpenAI

client = OpenAI()

def classify_sentiment(text: str) -> str:
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {
                "role": "system",
                "content": "You are a sentiment classifier. Respond with exactly one word: positive, negative, or neutral."
            },
            {
                "role": "user",
                "content": f"Classify the sentiment: {text}"
            }
        ]
    )
    return response.choices[0].message.content.strip().lower()

When zero-shot works: Well-defined tasks the model has seen millions of examples of in training (classification, translation, summarization, simple Q&A).

When zero-shot fails: Specialized domains, unusual output formats, multi-step reasoning, tasks requiring consistent style or tone.

---

2. Few-Shot Prompting

Show the model examples of what you want before asking it to do the task. This is the single highest-ROI prompt engineering technique.

def extract_structured_data(raw_text: str) -> dict:
    examples = [
        {
            "input": "Order #1234 from [email protected] for 3x Widget A at $29.99 each, shipped to 123 Main St",
            "output": '{"order_id": "1234", "email": "[email protected]", "items": [{"name": "Widget A", "qty": 3, "price": 29.99}], "address": "123 Main St"}'
        },
        {
            "input": "Invoice 5678, customer: Alice Smith ([email protected]), 1x Premium Plan subscription $99/month",
            "output": '{"order_id": "5678", "email": "[email protected]", "items": [{"name": "Premium Plan", "qty": 1, "price": 99.0}], "address": null}'
        }
    ]

    few_shot_prompt = "Extract order data as JSON.\n\n"
    for ex in examples:
        few_shot_prompt += f"Input: {ex['input']}\nOutput: {ex['output']}\n\n"
    few_shot_prompt += f"Input: {raw_text}\nOutput:"

    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": few_shot_prompt}],
        response_format={"type": "json_object"}
    )
    return json.loads(response.choices[0].message.content)

Few-Shot Best Practices

Diversity matters more than quantity: 3-5 varied examples usually outperform 10 similar ones. Cover edge cases.

Order matters: Later examples have more influence than earlier ones (recency bias). Put your most representative example last.

Format consistency is critical: If your examples use Output: as a prefix, use it for the query too. The model pattern-matches format, not just content.

Label balance: For classification, include roughly equal examples per class unless you intentionally want to bias toward certain outputs.

---

3. Chain-of-Thought (CoT) Prompting

The breakthrough technique from Google’s 2022 paper. Adding “Think step by step” to complex reasoning tasks dramatically improves accuracy — particularly for math, logical deduction, and multi-step problems.

Zero-Shot CoT

def solve_math_problem(problem: str) -> str:
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {
                "role": "user",
                "content": f"{problem}\n\nThink step by step before giving your final answer."
            }
        ]
    )
    return response.choices[0].message.content

Few-Shot CoT

Show examples that include the reasoning chain, not just answers:

COT_EXAMPLES = """
Q: Roger has 5 tennis balls. He buys 2 more cans of 3 tennis balls each. How many tennis balls does he have now?

A: Let me work through this:
- Roger starts with 5 tennis balls
- He buys 2 cans × 3 balls per can = 6 new balls
- Total: 5 + 6 = 11 tennis balls

Answer: 11

---

Q: A library has 120 books. They receive a donation of 45 books but need to discard 30 damaged ones. How many books do they have?

A: Let me work through this:
- Starting books: 120
- Books received: +45 → 120 + 45 = 165
- Books discarded: -30 → 165 - 30 = 135 books

Answer: 135
"""

def solve_with_cot(problem: str) -> str:
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {"role": "system", "content": "You solve problems step-by-step, showing all reasoning before the final answer."},
            {"role": "user", "content": f"{COT_EXAMPLES}\n---\nQ: {problem}\n\nA:"}
        ]
    )
    return response.choices[0].message.content

When CoT Helps (and Doesn’t)

CoT helps significantly:

• Multi-step arithmetic and algebra
• Logical deduction with multiple premises
• Planning tasks requiring intermediate steps
• Complex classification requiring reasoning

CoT hurts or is neutral:

• Simple factual retrieval (“What’s the capital of France?”)
• Pure classification with clear examples
• Tasks where intermediate reasoning isn’t needed

---

4. Self-Consistency Sampling

Sample multiple independent CoT chains and take the majority vote. This trades cost for accuracy on tasks where correctness matters more than speed.

from collections import Counter
import concurrent.futures

def self_consistent_solve(problem: str, samples: int = 5) -> str:
    def single_sample(_):
        response = client.chat.completions.create(
            model="gpt-4o",
            messages=[
                {"role": "user", "content": f"{problem}\nThink step by step. End with 'Final Answer: [answer]'"}
            ],
            temperature=0.7  # Higher temp for diversity
        )
        content = response.choices[0].message.content
        # Extract final answer
        if "Final Answer:" in content:
            return content.split("Final Answer:")[-1].strip()
        return content.strip()

    with concurrent.futures.ThreadPoolExecutor(max_workers=samples) as executor:
        answers = list(executor.map(single_sample, range(samples)))

    # Majority vote
    most_common = Counter(answers).most_common(1)[0][0]
    return most_common

Cost-accuracy tradeoff: Self-consistency with 5 samples costs 5x more but can improve accuracy on reasoning tasks by 10-20%. Use it for high-stakes, low-volume decisions.

---

5. RAG (Retrieval-Augmented Generation) Prompt Patterns

RAG separates where knowledge lives (a vector database) from where reasoning happens (the LLM). The prompt engineering challenge is integrating retrieved context effectively.

Basic RAG Prompt Structure

from openai import OpenAI
import chromadb

client = OpenAI()
chroma = chromadb.Client()
collection = chroma.get_collection("docs")

def rag_answer(question: str, k: int = 5) -> str:
    # Retrieve relevant chunks
    results = collection.query(
        query_texts=[question],
        n_results=k
    )

    context_chunks = results['documents'][0]
    context = "\n\n---\n\n".join(context_chunks)

    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {
                "role": "system",
                "content": """You are a helpful assistant. Answer questions based ONLY on the provided context.
If the context doesn't contain enough information to answer confidently, say so explicitly.
Never make up information not present in the context."""
            },
            {
                "role": "user",
                "content": f"""Context:
{context}

Question: {question}

Answer:"""
            }
        ]
    )
    return response.choices[0].message.content

Advanced RAG: Query Decomposition

For complex questions, decompose into sub-queries before retrieval:

def decompose_query(question: str) -> list[str]:
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {
                "role": "system",
                "content": "Break down complex questions into 2-4 simpler sub-questions that can be answered independently. Return as a JSON array of strings."
            },
            {"role": "user", "content": question}
        ],
        response_format={"type": "json_object"}
    )
    data = json.loads(response.choices[0].message.content)
    return data.get("sub_questions", [question])

def multi_hop_rag(question: str) -> str:
    sub_questions = decompose_query(question)
    all_context = []

    for sub_q in sub_questions:
        results = collection.query(query_texts=[sub_q], n_results=3)
        all_context.extend(results['documents'][0])

    # Deduplicate and combine context
    unique_context = list(dict.fromkeys(all_context))
    context = "\n\n---\n\n".join(unique_context[:8])  # Limit total context

    # Final synthesis
    return synthesize_answer(question, context)

RAG Prompt Anti-Patterns

Don’t: Dump all retrieved context without structure

# Bad - overwhelming unstructured context
context = " ".join(all_chunks)

Do: Structure context with clear delineation and source attribution

# Good - structured with sources
context_parts = []
for i, (chunk, source) in enumerate(zip(chunks, sources)):
    context_parts.append(f"[Source {i+1}: {source}]\n{chunk}")
context = "\n\n".join(context_parts)

---

6. Structured Output Prompting

Forcing structured output (JSON, XML, specific formats) makes downstream processing reliable.

from pydantic import BaseModel
from openai import OpenAI

class ProductReview(BaseModel):
    sentiment: str  # positive, negative, neutral
    score: int      # 1-10
    key_pros: list[str]
    key_cons: list[str]
    summary: str

def analyze_review(review_text: str) -> ProductReview:
    response = client.beta.chat.completions.parse(
        model="gpt-4o",
        messages=[
            {"role": "system", "content": "Analyze product reviews and extract structured data."},
            {"role": "user", "content": review_text}
        ],
        response_format=ProductReview
    )
    return response.choices[0].message.parsed

---

7. System Prompt Architecture

For production applications, the system prompt is architectural infrastructure. Here’s a battle-tested structure:

SYSTEM_PROMPT = """## Role
You are a technical documentation assistant for {product_name}. You help developers understand the API and troubleshoot integration issues.

## Knowledge Scope
- You have access to {product_name} API documentation (v{version})
- You do NOT have access to user account data, billing information, or usage metrics
- Your knowledge cutoff is {knowledge_date}

## Behavior Rules
1. Always cite specific API endpoints or documentation sections when answering
2. If a question requires account-specific information, direct users to: {support_url}
3. For questions outside your scope, say: "I don't have information about that. Please contact support at {support_email}"
4. Never guess at API behavior — if unsure, say so and recommend testing in the sandbox

## Output Format
- Use code blocks for all code examples
- Use markdown headers to organize long responses
- Keep responses concise — prefer bullet points over paragraphs for lists of options

## Tone
Professional but approachable. You're a helpful colleague, not a formal support bot.
"""

This template pattern — Role, Knowledge Scope, Behavior Rules, Output Format, Tone — consistently outperforms unstructured system prompts.

---

Putting It Together: Production Prompt Engineering Checklist

Before deploying any LLM feature:

• Define success criteria — how will you measure if the prompt works?
• Build an eval dataset — at least 20 representative examples with expected outputs
• Test failure modes — adversarial inputs, edge cases, injection attempts
• Version your prompts — treat prompts as code, store in version control
• Log everything — use LangSmith, Helicone, or Phoenix to trace production calls
• A/B test improvements — measure before deploying prompt changes

The teams shipping the best AI products in 2026 aren’t necessarily using better models — they’re being more systematic about their prompts.