Hugging Face

Overview

Hugging Face is the GitHub of AI — a platform where the ML community shares models, datasets, and demos. The transformers library provides a unified API for loading and running thousands of open-source models locally.

Using Models Locally

from transformers import pipeline

# Text generation
generator = pipeline("text-generation", model="meta-llama/Llama-3.1-8B-Instruct")
result = generator("Explain transformers in simple terms:", max_new_tokens=200)

# Embeddings
from transformers import AutoTokenizer, AutoModel
import torch

tokenizer = AutoTokenizer.from_pretrained("sentence-transformers/all-MiniLM-L6-v2")
model = AutoModel.from_pretrained("sentence-transformers/all-MiniLM-L6-v2")

inputs = tokenizer(["Hello world", "Hi there"], return_tensors="pt", padding=True)
with torch.no_grad():
    outputs = model(**inputs)
    embeddings = outputs.last_hidden_state.mean(dim=1)

Inference API (Serverless)

import requests

API_URL = "https://api-inference.huggingface.co/models/mistralai/Mistral-7B-Instruct-v0.3"
headers = {"Authorization": "Bearer hf_..."}

response = requests.post(API_URL, headers=headers, json={
    "inputs": "What is retrieval augmented generation?",
    "parameters": {"max_new_tokens": 200}
})

Model Card Best Practices

When using models from the Hub:

1. Check the model card for intended use and limitations
2. Verify the license (MIT, Apache 2.0 vs commercial restrictions)
3. Look at the community tab for known issues
4. Check the model size vs available VRAM

Key Libraries

• transformers: Load and run models
• datasets: Load and process datasets
• accelerate: Multi-GPU and mixed-precision training
• peft: Parameter-efficient fine-tuning (LoRA, QLoRA)
• trl: Reinforcement learning from human feedback (RLHF)

Use Cases

Semantic search and RAG pipelines: Use Hugging Face embedding models (like sentence-transformers/all-MiniLM-L6-v2 or BAAI/bge-m3) to embed documents and queries into vectors for similarity search. These run locally and have no per-call cost — ideal for embedding large document collections for RAG applications.

Fine-tuning open models for domain tasks: With peft and trl, fine-tune a 7B or 13B model on your own dataset using QLoRA on a single consumer GPU. Use Cases include training a model on your company’s internal documentation, customer support transcripts, or domain-specific code.

Prototyping before committing to a paid API: The Hub lets you evaluate dozens of open models before deciding whether to pay for GPT-4o or Claude. Run the same benchmark prompt against Llama, Mistral, Gemma, and Qwen locally to find the smallest model that meets your quality bar — reducing inference costs significantly.

Deploying private model demos: Hugging Face Spaces lets you host Gradio or Streamlit apps backed by any model from the Hub. This is the fastest way to share a demo of your fine-tuned model or a custom AI pipeline with stakeholders — free for public Spaces, paid for private.

Concrete Use Case: Fine-Tuning Sentiment Analysis on Customer Support Tickets

A SaaS company receives 3,000 customer support tickets per day. Their existing sentiment classification — a rule-based system using keyword lists — misclassifies 30% of tickets because customer language in their domain is nuanced: “I love how your export feature crashes every time I use it” registers as positive due to the word “love,” and “This is fine for now but we’ll need to evaluate alternatives at renewal” reads as neutral when it signals churn risk. The team has 18 months of historically labeled tickets (50,000 examples) where support agents tagged each ticket as positive, negative, neutral, or escalation-risk.

The ML engineer downloads a pre-trained distilbert-base-uncased model from the Hugging Face Hub — a compact 66M-parameter model that runs efficiently on a single T4 GPU. Using the datasets library, they load the labeled ticket data and split it 80/10/10 for train/validation/test. With peft, they apply QLoRA (4-bit quantized LoRA) to fine-tune only the classification head and a small set of adapter weights, keeping the total trainable parameters under 2 million. Training completes in 40 minutes on a single GPU. The fine-tuned model achieves 91% accuracy on the held-out test set — up from 70% with the keyword-based system — and correctly identifies sarcasm and implicit churn signals that the rule-based approach missed entirely.

For deployment, the engineer pushes the fine-tuned model to a private Hugging Face repository and creates a dedicated Inference Endpoint. The endpoint auto-scales from zero to three replicas based on request volume, handling the 3,000 daily tickets with p99 latency under 150ms per classification. The total monthly cost is approximately $45 for the inference endpoint — significantly cheaper than routing tickets through a general-purpose LLM API like GPT-4, which would cost $200+/month for the same volume and adds unnecessary latency for a task that a small specialized model handles better. The support team integrates the endpoint into their Zendesk workflow via a webhook, and escalation-risk tickets are now automatically flagged and routed to a senior agent within minutes of submission.

When to Use Hugging Face

Use Hugging Face when:

• You need access to a wide range of pre-trained models for NLP, computer vision, audio, or multimodal tasks — the Hub’s 500K+ models cover virtually every ML use case
• You want to fine-tune an open-source model on your own domain-specific data using parameter-efficient methods (LoRA, QLoRA) that run on consumer-grade GPUs
• You need to run model inference locally or on your own infrastructure to avoid per-call API costs, maintain data privacy, or comply with data residency requirements
• You are building a RAG pipeline and need high-quality embedding models (like sentence-transformers) that run locally with no API dependency
• You want to prototype and compare multiple open models before committing to a paid API provider — the Hub lets you benchmark Llama, Mistral, Gemma, and others side by side

When NOT to use Hugging Face:

• You need the highest-quality reasoning and instruction-following available today and cost is secondary — frontier models from OpenAI (GPT-4o) and Anthropic (Claude) still outperform most open models on complex tasks, and their managed APIs are simpler to integrate
• Your team has no ML engineering experience and you need a plug-and-play AI solution — the Transformers ecosystem has a learning curve around model selection, quantization, GPU memory management, and fine-tuning configuration
• You need guaranteed uptime and SLAs for production inference — Hugging Face Inference Endpoints are suitable for many workloads, but enterprise-critical applications may need dedicated infrastructure or a managed ML platform like AWS SageMaker or Google Vertex AI
• Your use case is simple text generation or classification with low volume — calling an API like OpenAI or Anthropic is faster to integrate and cheaper at low volume than setting up and maintaining your own model infrastructure