Artillery

Modern, open-source performance and load testing framework. Define realistic traffic scenarios in YAML — ramp-up phases, think times, loops, and assertions — across HTTP, WebSockets, gRPC, and Socket.io. Built for CI/CD integration from day one.

What is Artillery?

Artillery is an open-source performance testing framework designed for modern cloud-native applications. Unlike tools that require writing code to define test logic, Artillery uses a YAML-based configuration format that lets you describe multi-phase load patterns and realistic user scenarios with minimal friction. A basic test can go from zero to running in under five minutes.

Artillery supports multiple protocols — HTTP/HTTPS, WebSockets, Socket.io, gRPC, and Amazon Kinesis — making it suitable for testing APIs, real-time applications, and event-driven services with the same toolchain. Tests integrate directly into GitHub Actions, GitLab CI, Jenkins, and any other CI/CD system that can run an npm script.

Key Features

• YAML-first configuration — define targets, load phases, and user scenarios without writing test code for common cases
• Realistic traffic phases — warm-up, ramp-up, sustained load, and spike phases with configurable arrival rates
• Multi-protocol — HTTP/HTTPS, WebSockets, Socket.io, gRPC (plugin), and Amazon Kinesis
• Think time & loops — simulate realistic user behaviour with pauses, loops, and conditional branching
• Built-in metrics — response time percentiles, error rates, RPS, and virtual user counts reported per phase
• Expect plugin — add assertions (status codes, JSON properties, regex) to fail tests on contract violations
• Variable capture — extract values from responses (e.g. auth tokens) and use them in subsequent requests
• CI/CD integration — exits with code 1 on threshold violations; works with GitHub Actions, GitLab, Jenkins

Installation

# Install globally
npm install -g artillery

# Or add to a project as a dev dependency
npm install --save-dev artillery

# Verify installation
artillery version

# Run a quick smoke test against any URL (no config file needed)
artillery quick --count 20 --num 10 https://example.com/api/health

Writing a YAML Scenario

An Artillery config file has two top-level sections: config (target URL, load phases, default headers) and scenarios (user flows with weighted traffic distribution). The example below defines a three-phase load test with a warm-up, ramp-up, and sustained load period.

# load-test.yml — a complete Artillery scenario config
config:
  target: "https://api.example.com"
  phases:
    - duration: 60      # warm-up: 1 minute at 10 RPS
      arrivalRate: 10
      name: "Warm up"
    - duration: 120     # ramp-up: 2 minutes, 10 → 50 RPS
      arrivalRate: 10
      rampTo: 50
      name: "Ramp up"
    - duration: 300     # sustained load: 5 minutes at 50 RPS
      arrivalRate: 50
      name: "Sustained load"
  defaults:
    headers:
      Content-Type: "application/json"
      Accept: "application/json"

scenarios:
  - name: "Browse and create item"
    weight: 7           # 70% of virtual users run this scenario
    flow:
      - get:
          url: "/items"
          expect:
            - statusCode: 200
      - think: 2        # simulate user reading for 2 seconds
      - post:
          url: "/items"
          json:
            name: "Test item {{ $randomString() }}"
            price: "{{ $randomNumber(1, 100) }}"
          expect:
            - statusCode: 201

  - name: "Health check"
    weight: 3           # 30% of virtual users run this scenario
    flow:
      - get:
          url: "/health"
          expect:
            - statusCode: 200

Running Tests

Use artillery run to execute a test and print a live summary to stdout. Save results with --output and generate an HTML report with artillery report.

# Run a test from a YAML config file
artillery run load-test.yml

# Run and save results to a JSON report file
artillery run load-test.yml --output results.json

# Generate an HTML report from saved results
artillery report results.json

# Run with an environment override (override config.target)
artillery run load-test.yml --target https://staging.example.com

# Run only a specific scenario by name
artillery run load-test.yml --scenario "Browse and create item"

# Run with extra variables passed from the command line
artillery run load-test.yml --variables '{"userId":"abc123"}'

Assertions with the Expect Plugin

The @artillery/plugin-expect plugin adds assertion support directly inside YAML flow steps. Capture tokens from login responses and use them in subsequent authenticated requests — all within the same scenario.

# artillery-expect plugin — add assertions to your scenarios
# Install: npm install --save-dev @artillery/plugin-expect

config:
  target: "https://api.example.com"
  plugins:
    expect: {}           # enable the expect plugin
  phases:
    - duration: 30
      arrivalRate: 5

scenarios:
  - name: "Login and get profile"
    flow:
      - post:
          url: "/auth/login"
          json:
            email: "[email protected]"
            password: "secret"
          capture:
            - json: "$.token"    # capture JWT from response body
              as: token
          expect:
            - statusCode: 200
            - hasProperty: "token"
      - get:
          url: "/users/me"
          headers:
            Authorization: "Bearer {{ token }}"
          expect:
            - statusCode: 200
            - contentType: json
            - hasProperty: "email"

Artillery vs k6

Pros and Cons