How to Perform Load Testing with k6 using GitHub Actions

You can find a collection of k6 scripts and GitHub workflows referenced in this tutorial here.

📖What you will learn

• How to integrate load testing with k6 into GitHub Actions
• Different implementation paths, and when to use each

Introduction

In this tutorial, we will look into how to integrate performance testing in your development process with GitHub Actions and k6.

k6 is an open-source load testing tool for testing the performance of APIs, microservices, and websites. Developers use k6 to test a system's performance under a particular load to catch performance regressions or errors.

GitHub Actions is a new tool that enables developers to create custom workflows for their software development lifecycle directly inside their GitHub repositories. As of mid 2019, GitHub Actions now supports full CI/CD pipelines.

If you've not used GitHub Actions before, we recommend looking at the following links to get a hold of how it works:

• Workflow syntax for GitHub Actions
• Events that trigger workflows

Writing your performance test

We'll start small by writing a simple test that measure the performance of a single endpoint. As with most, if not all, development efforts, performance testing yields the best results if we work in small increments, iterating and expanding as our knowledge increases.

Our test will consist of three parts:

1. An HTTP request against our system under test.
2. A load configuration controlling the test duration and amount of virtual users.
3. A performance goal, or service level objective, expressed as a threshold.

Creating the test script

When we execute our test script, each virtual user will execute the default function as many times as possible until the duration is up. To make sure we dont flood our system under test, we'll make the virtual user sleep for a second before it continues.

// ./test.js
import http from 'k6/http';
import { sleep } from 'k6';
export default function () {
  const res = http.get('https://test.k6.io');
  sleep(1);
}

Configuring the load

We'll configure our test to run 50 virtual users continuously for one minute. Because of the sleep we added earlier, this will result in just below 50 iterations per second, givinig us a total of about 2900 iterations.

// ./test.js
import http from 'k6/http';
import { sleep } from 'k6';
export const options = {
  duration: '1m',
  vus: 50,
};
export default function () {
  const res = http.get('https://test.k6.io');
  sleep(1);
}

If you have installed k6 in your local machine, you can run your test locally in your terminal using the command: k6 run test.js.

Configuring our thresholds

The next step is to define your service level objectives, or SLOs around your application performance. SLOs are a vital aspect of ensuring the reliability of your systems and applications. If you do not currently have any defined SLAs or SLOs, now is an excellent time to consider your requirements.

You can define SLOs as Pass/Fail criteria with Thresholds in your k6 script. k6 evaluates them during the test execution and informs about the Threshold results. If any of the thresholds in our test fails, k6 will return with a non-zero exit code, communicating to the CI tool that the step has failed.

Now, we will add one Threshold to our previous script to validate than the 95th percentile response time must be below 500ms.

// ./test.js
import http from 'k6/http';
import { sleep } from 'k6';
export const options = {
  duration: '1m',
  vus: 50,
  thresholds: {
    http_req_duration: ['p(95)<500'],
  },
};
export default function () {
  const res = http.get('https://test.k6.io');
  sleep(1);
}

Thresholds are a powerful feature providing a flexible API to define various types of Pass/Fail criteria in the same test run. For example:

• The 99th percentile response time must be below 700 ms.
• The 95th percentile response time must be below 400 ms.
• No more than 1% failed requests.
• The content of a response must be correct more than 95% of the time.

Check out the Thresholds documentation for additional details on the API and its usage.

Setting up the GitHub Actions workflow

To have GitHub Actions pick up and execute our load test, we need to create a workflow configuration and place it in .github/workflows. Once this file has been pushed to our repository, each commit to our repository will result in the workflow being run.

# .github/workflows/load-test.yml
k6_load_test:
  name: k6 Load Test
  runs-on: ubuntu-latest
  steps:
    - name: Checkout
      uses: actions/checkout@v1
    - name: Run local k6 test
      uses: k6io/action@v0.1
      with:
        filename: test.js

To avoid having to either install k6 on the runner or download the k6 docker image, we're utilising the official k6 action available on the GitHub marketplace.

At this point, commit and push your changes and then go to the actions tab of your GitHub repository. GitHub Actions will now have picked up our new workflow and executed it:

and if we select the k6 Load Test job:

Running k6 cloud tests

There are two common execution modes to run k6 tests as part of the CI process.

• Locally on the CI server.
• In k6 cloud, from one or multiple geographic locations.

You might want to use cloud tests in these common cases:

• If you're going to run a test from multiple geographic locations (load zones).
• If you're going to run a high-load test, that will need more compute resources than available in the runner.

If any of those reasons fit your needs, then running k6 cloud tests is the way to go for you.

⚠️ Try it locally first

Before we start with the configuration, it is good to familiarize ourselves with how cloud execution works, and we recommend you to test how to trigger a cloud test from your machine.

Check out the cloud execution guide to learn how to distribute the test load across multiple geographic locations and more information about the cloud execution.

Now, we will show how to trigger cloud tests using GitHub actions. If you do not have an account with k6 Cloud already, you should go here and start your free trial.

After that, get your account token from the cloud app and add this token to your GitHub project's Secrets page.

# .github/workflows/load-test.yml
k6_cloud_test:
  name: k6 cloud test run
  runs-on: ubuntu-latest
  steps:
    - name: Checkout
      uses: actions/checkout@v1
    - name: Run k6 cloud test
      uses: k6io/action@v0.1
      with:
        filename: test.js
        cloud: true
        token: ${{ secrets.K6_CLOUD_API_TOKEN }}

As you can see, the only changes needed in our workflow file is setting cloud to true and pass our API token to the action.

Once we commit and push these changes, k6 will now run our test using the k6 cloud, and output the URL to our test results as part of the workflow logs:

And if we copy the highlighted URL and navigate to it in a new tab

Variations

Using a different runner

GitHub provides Windows and macOS environments to run your workflows. You can also setup custom runners that operate on your premises or your cloud infrastructure.

The k6-load-testing workflow we have used above is based on the official k6 action, provided through the GitHub Marketplace. This action, however, currently only runs on Linux. To be able to run it on a Windows or macOS runner, we'll have to install k6 as part of our pipeline.

Using a Windows runner

on: [push]
jobs:
  k6_local_test:
    name: k6 local test run on windows
    runs-on: windows-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v1
      - name: download and extract k6 release binaries
        run: |
          curl -L https://github.com/loadimpact/k6/releases/download/v0.26.2/k6-v0.26.2-win64.zip -o k6.zip
          7z.exe e k6.zip
        shell: bash
      - name: k6 test
        run: ./k6.exe run ./test.js
        shell: bash

Here is the most up-to-date for k6 Windows installation instructions. I would recommend using Chocolatey Package Manager to ensure your script grabs the latest k6 version.

Using a macOS runner

# .github/workflows/load-test.yml
on: [push]
jobs:
  k6_local_test:
    name: k6 local test on macos
    runs-on: macos-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v1
      - name: Install k6 by homebrew
        run: brew install k6
      - name: Local k6 test
        run: k6 run ./test.js

The brew package manager is the best tool for grabbing and installing the latest version of k6 whenever the workflow is run.

Nightly Builds

Triggering a subset of performance tests at a specific time is a best-practice for automating your performance testing.

It's common to run some performance tests during the night when users do not access the system under test. For example, to isolate more extensive tests from other types of testing or to generate a performance report periodically.

To configure scheduled nightly build that runs at a given time of a given day or night, head over to your GitHub action workflow and update the on section. Here is an example that triggers the workflow every 15 minutes:

# .github/workflows/load-test.yml
on:
  schedule:
    # * is a special character in YAML, so you have to quote this string
    - cron: '*/15 * * * *'

You'll have to use POSIX cron syntax to schedule a workflow to run at specific UTC times. Here is an interactive tool for creating crontab scheduling expressions.

Simply save, commit, and push the file. GitHub will take care of running the workflow at the time intervals you specified.

Using the docker image

Using the docker image directly is almost as easy as the marketplace app. The example below uses the cloud service, but you could just as easily use it for local execution as well

# .github/workflows/load-test.yml
on: [push]
jobs:
  k6_cloud_test:
    name: k6 cloud test run
    runs-on: ubuntu-latest
    container: docker://loadimpact/k6:latest
    steps:
      - name: Checkout
        uses: actions/checkout@v1
      - name: Cloud k6 test
        env:
          K6_CLOUD_TOKEN: ${{ secrets.k6_cloud_token }}
        run: k6 cloud ./test.js

Summary

The official k6 GitHub Action, as well as the other possible configurations mentioned throughout the article, provide the same flexibility and capabilities as you're used to from running k6 locally.

Integrating k6 performance tests into a new or existing GitHub Actions pipeline is quick and easy, especially using the official marketplace app. By running your performance tests continuously, and automated, you'll be able to identify and correct performance regressions as they occur.