Consul — Service Mesh & Service Discovery Guide

Discover services, enforce health checks, and secure inter-service communication with mTLS zero-trust networking — HashiCorp Consul runs anywhere from bare metal to Kubernetes.

What is HashiCorp Consul?

HashiCorp Consul is a distributed systems tool that provides service discovery, health checking, key-value storage, and service mesh networking in a single unified platform. Released in 2014, Consul was designed from the ground up for dynamic, large-scale distributed environments where services come and go constantly. Unlike static load balancers or DNS with long TTLs, Consul provides real-time service registries that immediately reflect the actual state of your infrastructure.

Consul agents run on every node in your infrastructure — either as servers (which form the consensus cluster and hold state) or clients (which register local services, run health checks, and forward queries to servers). The recommended production setup is 3 or 5 server nodes for high availability via Raft consensus, with client agents on every host. Consul clusters can be federated across data centers, enabling multi-region service discovery and network segmentation.

Key Features

• Service Registry — real-time catalog of services with tags, metadata, and addresses
• Health Checks — HTTP, TCP, gRPC, script, and TTL checks; unhealthy instances filtered from DNS
• DNS Interface — service-name.service.consul resolves to healthy instance IPs, zero client changes
• HTTP API — query catalog, KV store, and agent programmatically from any language
• Intentions — declarative access control between services, enforced by Envoy mTLS
• Envoy Sidecar (Connect) — transparent L7 proxy with automatic mTLS, observability, and retries
• KV Store — distributed, consistent key-value storage for configuration and coordination
• Multi-datacenter — federate clusters across regions with WAN gossip protocol

Service Registration

# /etc/consul.d/web-api.json
{
  "service": {
    "id": "web-api-1",
    "name": "web-api",
    "tags": ["v2", "primary"],
    "address": "10.0.1.10",
    "port": 8080,
    "meta": {
      "version": "2.1.0",
      "environment": "production"
    },
    "checks": [
      {
        "id": "api-http",
        "name": "HTTP health check",
        "http": "http://localhost:8080/health",
        "interval": "10s",
        "timeout": "2s"
      },
      {
        "id": "api-tcp",
        "name": "TCP connectivity",
        "tcp": "localhost:8080",
        "interval": "30s"
      }
    ],
    "connect": {
      "sidecar_service": {
        "proxy": {
          "upstreams": [
            {
              "destination_name": "postgres",
              "local_bind_port": 5432
            },
            {
              "destination_name": "redis",
              "local_bind_port": 6379
            }
          ]
        }
      }
    }
  }
}

Service Discovery CLI & DNS

# Query healthy service instances via API
curl http://127.0.0.1:8500/v1/catalog/service/web-api?passing=true

# DNS lookup — resolves to healthy instance IPs
dig @127.0.0.1 -p 8600 web-api.service.consul

# Filter by tag
dig @127.0.0.1 -p 8600 v2.web-api.service.consul

# Query a specific datacenter
dig @127.0.0.1 -p 8600 web-api.service.us-east-1.consul

# Register a service dynamically via API
curl -X PUT http://127.0.0.1:8500/v1/agent/service/register \
  -d '{"Name":"my-service","Port":9000,"Check":{"HTTP":"http://localhost:9000/health","Interval":"10s"}}'

# List all registered services
consul catalog services

# Watch for changes (long-poll, returns on any change)
consul watch -type=service -service=web-api cat

Service Mesh Intentions

# Allow web frontend to call the API service
consul intention create web-frontend web-api

# Allow API to call the database
consul intention create web-api postgres

# Deny all traffic from external services (default deny is preferred)
consul intention create -deny '*' postgres

# List all intentions
consul intention list

# Intentions via config entry (declarative, version-controlled)
# intentions.hcl
Kind = "service-intentions"
Name = "web-api"
Sources = [
  {
    Name       = "web-frontend"
    Action     = "allow"
  },
  {
    Name   = "*"
    Action = "deny"
  }
]

consul config write intentions.hcl

KV Store Operations

# Write and read key-value pairs
consul kv put config/myapp/db_host postgres.service.consul
consul kv put config/myapp/max_connections 100
consul kv get config/myapp/db_host

# Get with metadata (ModifyIndex for CAS operations)
consul kv get -detailed config/myapp/db_host

# List all keys under a prefix
consul kv get -recurse config/myapp/

# Delete a key
consul kv delete config/myapp/old_key

# Watch for changes (blocks until key changes)
consul watch -type=key -key=config/myapp/db_host cat

# Atomic compare-and-swap (CAS) — only update if ModifyIndex matches
consul kv put -cas -modify-index=42 config/myapp/db_host new-value

Consul vs Service Mesh Alternatives

Use Cases

• Microservices architectures where services need to discover each other dynamically without hardcoded IPs
• Hybrid cloud / multi-datacenter environments running both VMs and Kubernetes
• Zero-trust network segmentation — enforce which services can communicate via intentions
• Service mesh for organizations not yet fully on Kubernetes (Consul works on bare metal and VMs)
• Configuration distribution via KV store with Consul Template for dynamic config file generation
• Replacing static load balancer configs with real-time service-health-aware routing

Pros and Cons