Apache Kafka vs RabbitMQ vs Redis Pub/Sub vs NATS: Message Broker Comparison 2026

Picking the wrong message broker can torpedo your architecture. Too little throughput, surprise data loss, or a system that simply doesn’t fit how your app communicates — these are expensive mistakes to fix later. In 2026, the four most widely-deployed options are Apache Kafka, RabbitMQ, Redis Pub/Sub, and NATS. Each solves a different problem exceptionally well, and none is universally “best.”

This guide gives you the technical depth to choose correctly for your workload.

Quick Comparison: At a Glance

	Apache Kafka	RabbitMQ	Redis Pub/Sub	NATS
Throughput	Millions msg/sec	~50k–100k msg/sec	~1M+ msg/sec	~10M+ msg/sec
Latency	5–15ms (default)	1–5ms	<1ms	<1ms
Persistence	Yes (log-based, configurable retention)	Yes (optional, per-queue)	No (fire-and-forget)	JetStream only
Message ordering	Per partition	Per queue	None guaranteed	JetStream per stream
Consumer groups	Yes (offset-based replay)	Yes (competing consumers)	No	Yes (JetStream)
Best for	Event streaming, audit logs, analytics	Task queues, complex routing	Lightweight pub/sub, cache invalidation	Microservices, edge, IoT
Operational complexity	High (Zookeeper/KRaft, brokers)	Medium (Erlang runtime)	Low (if Redis is already present)	Low
Protocol	Custom binary (Kafka protocol)	AMQP, MQTT, STOMP	Redis protocol (RESP)	NATS protocol
Cloud managed	AWS MSK, Confluent Cloud	CloudAMQP, AmazonMQ	Upstash Redis, Redis Cloud	Synadia Cloud

Apache Kafka: Event Streaming at Scale

How Kafka Works

Kafka is a distributed commit log, not a traditional message queue. Producers write to topics which are divided into partitions. Each partition is an append-only log replicated across broker nodes. Consumers maintain their own offset (position in the log), enabling:

Independent consumer groups reading at different speeds
Replay: rewind and reprocess from any point in history
Long-term event storage (days, weeks, or forever)

Topic: orders
├── Partition 0: [msg1, msg2, msg5, msg8, ...]
├── Partition 1: [msg3, msg6, msg9, ...]
└── Partition 2: [msg4, msg7, msg10, ...]
         Consumer A: offset=5   Consumer B: offset=2

Kafka Node.js Example

// Install: npm install kafkajs

const { Kafka } = require('kafkajs');

const kafka = new Kafka({
  clientId: 'order-service',
  brokers: ['kafka:9092']
});

// Producer
async function produceOrderEvent(order) {
  const producer = kafka.producer();
  await producer.connect();

  await producer.send({
    topic: 'orders',
    messages: [
      {
        key: order.customerId, // key determines partition — ensures ordering per customer
        value: JSON.stringify(order),
      }
    ]
  });

  await producer.disconnect();
}

// Consumer
async function consumeOrderEvents() {
  const consumer = kafka.consumer({ groupId: 'fulfillment-service' });
  await consumer.connect();
  await consumer.subscribe({ topic: 'orders', fromBeginning: false });

  await consumer.run({
    eachMessage: async ({ topic, partition, message }) => {
      const order = JSON.parse(message.value.toString());
      console.log(`[Partition ${partition}] Processing order ${order.id}`);
      await fulfillOrder(order);
      // Offset committed automatically after eachMessage resolves
    }
  });
}

Kafka Performance Characteristics

Kafka’s throughput comes from batching, sequential disk I/O, and zero-copy reads. A single broker can sustain 100MB/s+ writes to disk because it writes sequentially rather than randomly.

Where Kafka shines:

Audit logs that must never be deleted
Event sourcing (rebuild state from event history)
Stream processing pipelines (Kafka Streams, Apache Flink)
Analytics data ingestion (feed ClickHouse, Snowflake, etc.)
Multiple downstream consumers reading the same events independently

Kafka’s weaknesses:

Per-message latency is higher than alternatives (buffering batches)
Operational complexity: partition rebalancing, lag monitoring, schema registry
Overkill for simple task queues
KRaft (Zookeeper replacement) is still stabilizing in production setups

RabbitMQ: Flexible Routing and Task Queues

How RabbitMQ Works

RabbitMQ implements AMQP semantics: producers publish to exchanges, exchanges route messages to queues via bindings. This routing layer is what differentiates RabbitMQ from simpler brokers.

Producer → Exchange
              ├── [direct] → Queue A (routing_key = "payment.completed")
              ├── [fanout] → Queue B, Queue C (broadcasts to all bindings)
              ├── [topic]  → Queue D (payment.* matches payment.completed, payment.failed)
              └── [headers]→ Queue E (match on message headers)

Four exchange types give you enormous routing flexibility without application-level routing logic.

RabbitMQ Node.js Example

// Install: npm install amqplib

const amqp = require('amqplib');

// Producer: publish a task
async function enqueueEmailTask(emailData) {
  const conn = await amqp.connect('amqp://rabbitmq:5672');
  const channel = await conn.createChannel();

  const queue = 'email_tasks';
  await channel.assertQueue(queue, { durable: true }); // survive broker restart

  channel.sendToQueue(
    queue,
    Buffer.from(JSON.stringify(emailData)),
    { persistent: true } // persist to disk
  );

  await channel.close();
  await conn.close();
}

// Consumer: process tasks with acknowledgement
async function processEmailTasks() {
  const conn = await amqp.connect('amqp://rabbitmq:5672');
  const channel = await conn.createChannel();

  await channel.assertQueue('email_tasks', { durable: true });
  channel.prefetch(1); // only one unacknowledged message per consumer (fair dispatch)

  channel.consume('email_tasks', async (msg) => {
    const emailData = JSON.parse(msg.content.toString());

    try {
      await sendEmail(emailData);
      channel.ack(msg); // acknowledge success → message deleted
    } catch (err) {
      channel.nack(msg, false, true); // nack → re-queue for retry
    }
  });
}

// Topic exchange: publish/subscribe with routing
async function setupTopicExchange() {
  const conn = await amqp.connect('amqp://rabbitmq:5672');
  const channel = await conn.createChannel();

  await channel.assertExchange('events', 'topic', { durable: true });

  // Publisher
  channel.publish('events', 'payment.completed', Buffer.from(JSON.stringify({
    orderId: '123', amount: 99.99
  })));

  // Subscriber: matches payment.* (payment.completed, payment.failed, etc.)
  const q = await channel.assertQueue('', { exclusive: true });
  await channel.bindQueue(q.queue, 'events', 'payment.*');

  channel.consume(q.queue, (msg) => {
    console.log('Payment event:', JSON.parse(msg.content.toString()));
    channel.ack(msg);
  });
}

RabbitMQ Performance Characteristics

RabbitMQ achieves low latency (sub-5ms typical) because messages move through memory with optional disk persistence. With lazy queues and quorum queues (Raft-based replication), it provides strong durability guarantees.

Where RabbitMQ shines:

Task queues with competing workers (email, image processing, notifications)
Complex routing rules (content-based, topic-based, header-based)
Request/reply patterns (RPC over messaging)
Dead letter queues for failed message handling
Delayed message processing (with plugins)

RabbitMQ’s weaknesses:

Messages are deleted after acknowledgement — no replay
Consumer groups can’t independently replay history
Queue depth monitoring and backpressure handling requires care
Erlang runtime can be unfamiliar to debug

Redis Pub/Sub: Lightweight and Ephemeral

How Redis Pub/Sub Works

Redis Pub/Sub is fire-and-forget: publishers send messages to channels, and all currently-connected subscribers receive them. There is no message queue, no persistence, and no acknowledgement. If a subscriber is offline when a message is published, it misses the message entirely.

Publisher → PUBLISH channel "message"
                    ↓
         All current subscribers receive it
         (offline subscribers get nothing)

Redis 5.0+ added Streams (XADD/XREAD) which do provide persistence and consumer groups — but that’s a different feature from classic Pub/Sub.

Redis Pub/Sub Node.js Example

// Install: npm install ioredis

const Redis = require('ioredis');

// Publisher
async function publishCacheInvalidation(key) {
  const pub = new Redis({ host: 'redis', port: 6379 });

  await pub.publish('cache:invalidate', JSON.stringify({
    key,
    timestamp: Date.now()
  }));

  // Redis pub/sub and regular commands need separate connections
  await pub.quit();
}

// Subscriber
function subscribeToInvalidations(onInvalidate) {
  const sub = new Redis({ host: 'redis', port: 6379 });

  sub.subscribe('cache:invalidate', (err, count) => {
    console.log(`Subscribed to ${count} channel(s)`);
  });

  sub.on('message', (channel, message) => {
    const data = JSON.parse(message);
    console.log(`Cache invalidate: ${data.key}`);
    onInvalidate(data.key);
  });

  return sub; // keep reference alive
}

// Redis Streams (persistent alternative)
async function streamProducer() {
  const redis = new Redis();

  // XADD: append to stream, auto-generate ID
  await redis.xadd('orders', '*',
    'orderId', '123',
    'amount', '99.99',
    'status', 'pending'
  );
}

async function streamConsumer(groupName, consumerName) {
  const redis = new Redis();

  // Create consumer group (once)
  await redis.xgroup('CREATE', 'orders', groupName, '0', 'MKSTREAM')
    .catch(() => {}); // ignore if already exists

  while (true) {
    const results = await redis.xreadgroup(
      'GROUP', groupName, consumerName,
      'COUNT', 10, 'BLOCK', 1000,
      'STREAMS', 'orders', '>'
    );

    if (results) {
      for (const [stream, messages] of results) {
        for (const [id, fields] of messages) {
          const order = Object.fromEntries(
            fields.reduce((acc, _, i) => (i % 2 === 0 ? [...acc, [fields[i], fields[i+1]]] : acc), [])
          );
          console.log('Processing:', order);
          await redis.xack('orders', groupName, id);
        }
      }
    }
  }
}

Where Redis Pub/Sub shines:

Cache invalidation across multiple app servers
Real-time notifications where message loss is acceptable (live score updates)
Feature flag propagation
Presence/heartbeat signals
When Redis is already in your stack — zero additional infrastructure

Redis Pub/Sub’s weaknesses:

No persistence — subscribers must be online to receive messages
No acknowledgement or delivery guarantee
Not suitable for task queues or anything that requires at-least-once delivery
Classic Pub/Sub doesn’t scale to millions of subscribers well

NATS: Speed-First for Microservices and Edge

How NATS Works

NATS is built around subject-based messaging with a flat namespace and wildcard subscriptions. The NATS server is a single 17MB binary with no external dependencies. Core NATS is fire-and-forget like Redis Pub/Sub — but JetStream (built into NATS since 2.2) adds persistence, consumer groups, and exactly-once semantics.

Subject hierarchy:
orders.created       → matches orders.created
orders.>             → matches orders.created, orders.shipped, orders.*.refunded
orders.*             → matches orders.created, orders.shipped (single token wildcard)

NATS also supports request/reply as a first-class pattern: the client sends a message with a reply-to subject, the responder publishes back to that subject. This makes RPC over NATS natural.

NATS Node.js Example

// Install: npm install nats

const { connect, StringCodec, JSONCodec } = require('nats');

const jc = JSONCodec();

// Publisher/Subscriber (core NATS — ephemeral)
async function coreNATSExample() {
  const nc = await connect({ servers: 'nats://nats:4222' });

  // Subscribe with wildcard
  const sub = nc.subscribe('orders.*');
  (async () => {
    for await (const msg of sub) {
      const order = jc.decode(msg.data);
      console.log(`[${msg.subject}]`, order);

      // Request/reply: if caller wants a response
      if (msg.reply) {
        msg.respond(jc.encode({ status: 'received', orderId: order.id }));
      }
    }
  })();

  // Publish
  nc.publish('orders.created', jc.encode({ id: '123', amount: 99.99 }));

  // Request (RPC pattern)
  const reply = await nc.request('orders.created', jc.encode({ id: '124', amount: 49.99 }), {
    timeout: 5000 // 5s timeout
  });
  console.log('Reply:', jc.decode(reply.data));
}

// JetStream (persistent, at-least-once delivery)
async function jetStreamExample() {
  const nc = await connect({ servers: 'nats://nats:4222' });
  const js = nc.jetstream();
  const jsm = await nc.jetstreamManager();

  // Create a stream (once)
  await jsm.streams.add({
    name: 'ORDERS',
    subjects: ['orders.*'],
    retention: 'limits',
    max_age: 86400e9 // 24 hours in nanoseconds
  }).catch(() => {}); // ignore if exists

  // Publish with JetStream (persisted)
  const pubAck = await js.publish('orders.created', jc.encode({
    id: '125', amount: 29.99
  }));
  console.log('Sequence:', pubAck.seq);

  // Consume with durable consumer
  const consumer = await js.consumers.get('ORDERS', 'fulfillment');

  const messages = await consumer.consume({ max_messages: 10 });
  for await (const msg of messages) {
    const order = jc.decode(msg.data);
    console.log('Processing:', order);
    msg.ack(); // explicit acknowledgement
  }

  await nc.close();
}

Where NATS shines:

Microservice-to-microservice communication (lower overhead than HTTP)
IoT and edge deployments (single binary, tiny footprint)
Request/reply RPC patterns
Service meshes and sidecars
High-throughput, low-latency notification systems
Kubernetes-native deployments (NATS Helm chart is clean)

NATS’s weaknesses:

Core NATS has no persistence (JetStream adds it but with complexity)
Less ecosystem tooling than Kafka or RabbitMQ
JetStream monitoring and observability is less mature
Smaller community than Kafka/Rabbit

Performance Benchmarks

These numbers reflect representative real-world deployments, not lab maximums:

Scenario	Kafka	RabbitMQ	Redis Pub/Sub	NATS Core
Single producer throughput	500k–2M msg/s	50k–100k msg/s	1M+ msg/s	5M–10M msg/s
End-to-end latency (p99)	15–50ms	5–10ms	1–2ms	1–3ms
Latency with persistence	15–50ms	10–20ms	N/A	5–15ms (JetStream)
Memory per connection	~1KB	~150KB	~30KB	~2KB
Multi-consumer fan-out	Excellent (partitions)	Good	Excellent	Excellent

Key insight: Kafka’s “slow” latency is intentional — it batches writes to maximize throughput. If you need low latency AND high throughput, NATS JetStream is worth evaluating. If you need guaranteed ordering within a customer/entity, Kafka with partition keys is the right answer.

Architecture Diagrams

Kafka: Event Streaming Pipeline

┌──────────┐    ┌─────────────────────────────────────┐
│ Producer │───▶│  Kafka Cluster                      │
│  (App)   │    │  Topic: orders                      │
└──────────┘    │  ├─ Partition 0 ──────────────────▶ │──▶ Consumer Group A
                │  ├─ Partition 1 ──────────────────▶ │──▶ Consumer Group B
                │  └─ Partition 2 ──────────────────▶ │──▶ Consumer Group C
                └─────────────────────────────────────┘
                         (each group reads independently,
                          offsets maintained per group)

RabbitMQ: Task Queue with Dead Letter

┌──────────┐    ┌──────────┐    ┌──────────┐    ┌──────────────┐
│ Producer │───▶│ Exchange │───▶│  Queue   │───▶│   Worker 1   │
└──────────┘    │ (direct) │    │ (durable)│    │   Worker 2   │
                └──────────┘    └────┬─────┘    │   Worker 3   │
                                     │ failed   └──────────────┘
                                     ▼
                              ┌─────────────┐
                              │  Dead Letter│
                              │    Queue    │
                              └─────────────┘

NATS: Request/Reply Microservices

┌──────────────┐                    ┌──────────────┐
│   Service A  │──▶ orders.create ─▶│   Service B  │
│              │◀── reply subject ──│  (responds)  │
└──────────────┘                    └──────────────┘

Subject hierarchy allows scoped subscriptions:
orders.*    → matches: orders.create, orders.cancel
orders.>    → matches: orders.create, orders.items.add

When to Use Each

Use Kafka when:

You need event sourcing — rebuilding state from event history
Multiple independent services must consume the same events
You need data retention for analytics pipelines or compliance
Throughput exceeds what traditional brokers handle (millions/sec)
You’re feeding data into Flink, Spark, or a data warehouse
Ordering matters within an entity/customer (use partition keys)

Use RabbitMQ when:

You need a task queue where work items are consumed once (email, image resizing, notifications)
Complex routing based on message content, topics, or headers
You need dead letter queues for failed task handling
Request/reply patterns (RPC over messaging)
You want flexible per-message TTL and priority queues
Team is familiar with AMQP semantics

Use Redis Pub/Sub when:

Redis is already in your stack — no new infrastructure
Messages are ephemeral and loss is acceptable
Cache invalidation across multiple instances
Real-time notifications where best-effort delivery is fine
You need extremely low latency (<1ms)
Scale is modest (thousands, not millions of subscribers)

Consider Redis Streams instead if you need persistence within Redis.

Use NATS when:

Microservices communication — lower overhead than HTTP/gRPC
IoT and edge deployments with constrained resources
Request/reply RPC as first-class pattern
Kubernetes-native workloads (NATS operator is excellent)
High fan-out to many subscribers (IoT sensor broadcast)
Simple operational model is a priority

Decision Flowchart

START: Do you need message persistence?
│
├─ NO → Do you need low latency (<5ms)?
│       ├─ YES → Is Redis already in your stack?
│       │         ├─ YES → Redis Pub/Sub
│       │         └─ NO  → NATS Core
│       └─ NO → NATS Core (simplest)
│
└─ YES → Do multiple independent services need to read the same events?
         │
         ├─ YES → Do you need replay / audit / analytics?
         │         ├─ YES → Apache Kafka
         │         └─ NO  → Apache Kafka (still best fit)
         │
         └─ NO → Is routing/filtering complexity high?
                  ├─ YES → RabbitMQ
                  └─ NO  → Is throughput the primary concern?
                            ├─ YES → Kafka or NATS JetStream
                            └─ NO  → RabbitMQ (simplest durable option)

Migration Guide: RabbitMQ → Kafka

Teams commonly outgrow RabbitMQ when they need event replay or multi-consumer fan-out. Here’s what the migration involves:

Conceptual mapping

RabbitMQ	Kafka equivalent	Notes
Queue	Topic partition	Kafka topics are multi-consumer by design
Exchange (fanout)	Topic with multiple consumer groups	Each group gets all messages
Exchange (direct)	Topic + consumer group filter	Application-level filtering
Message ACK	Consumer offset commit	Offset commit = “I processed up to here”
Dead letter queue	Separate error topic	Common pattern: `orders.errors`
Message TTL	Topic retention policy	Set at topic level, not per message
Message priority	Not native	Use separate topics per priority

Migration approach

Phase 1: Dual-write (safest)
  Producer writes to both RabbitMQ AND Kafka
  Old consumers read from RabbitMQ
  New consumers read from Kafka
  Run in parallel until confident

Phase 2: Cutover
  Stop writing to RabbitMQ
  Drain existing RabbitMQ queues
  All consumers move to Kafka

Phase 3: Decommission
  Remove RabbitMQ infrastructure

Key behavioral differences to handle:

Kafka has no concept of “message deleted after consumption” — design consumers to be idempotent
Ordering is per-partition, not global — use partition keys for entity-level ordering
Kafka consumers must commit offsets explicitly (or auto-commit) — failures don’t auto-requeue
Schema changes need a schema registry strategy (Avro/Protobuf with Confluent Schema Registry)

Cloud Managed Options

Running these yourself adds operational burden. Managed options to consider in 2026:

Apache Kafka

AWS MSK (Managed Streaming for Kafka): tight IAM integration, MSK Serverless removes capacity planning. ~$0.75/hour for multi-AZ cluster + storage costs
Confluent Cloud: best ecosystem (Schema Registry, KSQL, Connectors). More expensive but fully managed
Upstash Kafka: serverless pricing (per request), good for variable workloads
Aiven for Kafka: multi-cloud, strong observability

RabbitMQ

CloudAMQP: purpose-built, generous free tier (Little Lemur: 1M messages/month free). Simple pricing
Amazon MQ: AWS-managed RabbitMQ or ActiveMQ. Good if you’re already AWS-native
Aiven for RabbitMQ: multi-cloud managed option

Redis Pub/Sub

Upstash Redis: serverless, per-command pricing, free tier available. Best for variable load
Redis Cloud: official Redis Labs offering, strong SLA
AWS ElastiCache: tight AWS integration, but Redis Cluster mode complicates Pub/Sub

NATS

Synadia Cloud (formerly NGS): official NATS cloud, global footprint, JetStream included
Self-hosted on Kubernetes: NATS Helm chart + NATS Operator is the most common production approach for Kubernetes shops
Fly.io: NATS JetStream deploys cleanly on Fly machines

Operational Checklist

Before committing to any message broker in production:

For Kafka:

Partition count planned (can’t decrease later; affects parallelism)
Replication factor set (3 for production)
Retention policy defined (size-based vs time-based)
Consumer lag monitoring (Prometheus + Grafana or Datadog)
Schema registry strategy (if using structured data)
KRaft vs Zookeeper decision made

For RabbitMQ:

Queue durability and persistence configured
Dead letter exchanges set up for critical queues
Prefetch count tuned per consumer
Quorum queues used for critical workloads (not classic queues)
Management plugin enabled for observability
Queue depth alerts configured

For Redis Pub/Sub:

Accepted that messages can be lost (subscriber offline = missed)
Consider Redis Streams if persistence needed
Publisher and subscriber use separate connections
Tested subscriber reconnect behavior

For NATS:

Core NATS vs JetStream decision made
JetStream stream retention configured if using JetStream
NATS server clustering (3+ nodes) for HA
Leaf node topology planned for edge deployments
Consumer acknowledgement strategy defined (JetStream)

Summary

If you need…	Use
Event replay, multi-consumer, audit trails	Apache Kafka
Task queues, complex routing, dead letters	RabbitMQ
Lightweight notifications, cache invalidation	Redis Pub/Sub
Microservice RPC, IoT, low-latency fan-out	NATS
Persistent + low-latency + Kubernetes-native	NATS JetStream

The most common mistake is reaching for Kafka for everything because it handles scale. Kafka is operationally heavy — if your use case is “process these 1000 emails/day”, RabbitMQ is simpler, cheaper, and better suited. Use Kafka when you genuinely need event streaming semantics: replay, independent consumers, and long-term event retention.

When in doubt: start with RabbitMQ or NATS, and migrate to Kafka when you hit its ceiling or need its unique semantics. Migration is painful but manageable with dual-write patterns.

Related DevPlaybook tools: API Rate Limit Calculator · JSON Formatter · Base64 Encoder/Decoder