How to Perform Load Testing with k6 using Atlassian Bamboo Platform

You can find a collection of k6 scripts and build specification for Bamboo in this tutorial here.

📖What you will learn

• How to integrate load testing with k6 into Atlassian Bamboo Platform.
• Different implementation paths, and when to use each.

Introduction

In this guide, you will learn how to implement performance testing in your development workflow with Atlassian Bamboo and k6.

k6 is an open-source load testing tool for testing the performance of APIs, microservices, and websites. Developers use k6 to test a system's performance under a particular load in order to catch performance regressions or errors.

Atlassian Bamboo is a continuous integration and deployment server that is used for automating software development workflows such building, testing and deploying releases into production.

Writing your performance test

We'll start by writing a simple performance test using JavaScript. Later on, we'll update the script with additional testing logic. Our first test will be made up of the following:

1. HTTP request to a site we want to test
2. Test parameters i.e. test duration and number of virtual users
3. Performance goal, or service level objective, expressed as a threshold.

Creating the test script

The following test script will simulate a virtual user accessing a specified website. To ensure we don't overload the system under test, we'll pause the virtual user for one second before continuing.

// ./test.js
import http from 'k6/http';
import { sleep } from 'k6';

export default function () {
  const res = http.get('https://test.k6.io');
  sleep(1);
}

Configuring the load

In this step, we'll define the test parameters. For this example, we'll have 50 virtual users accessing the specified website for the duration of one minute. Due to the one second sleep we specified earlier, the script will generate 50 interactions per second, resulting in a total of about 2900 iterations.

// ./test.js
import http from 'k6/http';
import { sleep } from 'k6';

export const options = {
  duration: '1m',
  vus: 50,
};

export default function () {
  const res = http.get('https://test.k6.io');
  sleep(1);
}

If you have installed k6 in your local machine, you can run your test locally in your terminal using the command: k6 run test.js.

Configuring our thresholds

Next, we'll define our Service Level Objectives(SLOs). Basically, this is measurement of one or metrics that indicate the desired performance levels at a given load. With k6, this is done using Thresholds where you can define SLOs as Pass/Fail criteria.

If a threshold fails a test, k6 will stop the execution and return a non-zero exit code, communicating to the CI tool that the step has failed. Let's now add a Threshold to our previous script. We'll ensure that the 95th percentile response time remains below 500ms.

// ./test.js
import http from 'k6/http';
import { sleep } from 'k6';

export const options = {
  duration: '1m',
  vus: 50,
  thresholds: {
    http_req_duration: ['p(95)<500'],
  },
};

export default function () {
  const res = http.get('https://test.k6.io');
  sleep(1);
}

Thresholds are powerful features that allow us to define different types of Pass/Fail criteria in the same test run. For example:

• The 99th percentile response time must be below 700 ms.
• The 95th percentile response time must be below 400 ms.
• No more than 1% failed requests.
• The content of a response must be correct more than 95% of the time.

Take a look at our Thresholds documentation for additional details on the API and its usage.

Setting up the Bamboo workflow

This section assumes you already have a fully installed and running Bamboo server. There are two ways of running k6 tests with Bamboo:

1. Using an executable
2. Using docker

For this guide, we'll use the the executable method since it's the easiest to setup. You'll need to:

1. Install k6 locally on your machine
2. Specify k6 as an executable under Bamboo administration > Server capabilities

• Capability type: Executable
• Type: Command
• Executable label: k6
• Path: (path to k6 executable e.g. /usr/bin/k6)

3. If you haven't set up one yet, you'll need an agent to run your builds. You can add a local agent under Bamboo administration > Agents

Your Bamboo server is now ready to execute k6 commands. Before we proceed with the next step, you'll need to have your project with the load testing scripts uploaded to a remote repository. At the time of writing, Bamboo supports a limited number of repositories which includes:

• Bitbucket
• GitHub
• Perforce

Unlike most CI/CD platforms, Bamboo's YAML specs are difficult to setup. Even if you have a working spec configuration, there's no guarantee that it will work out-of-the-box on another Bamboo server instance. Hence, we'll use the provided web user interface to define our automation workflow.

The main steps are as follows:

1. Setup a new project via the top bar Create menu

   

2. Create a plan

   

3. Link your remote repository

   

4. Create a job

   

In the above configuration, we have specified k6 to run the performance test locally. Clicking the 'Run' button will execute the specified job:

Running cloud tests

There are two common execution modes to run k6 tests as part of the CI process.

• Locally on the CI server.
• In Grafana Cloud k6, from one or multiple geographic locations.

You might want to use cloud tests in these common cases:

• If you're going to run a test from multiple geographic locations (load zones).
• If you're going to run a high-load test, that will need more compute resources than available in the runner.

If any of those reasons fit your needs, then running k6 cloud tests is the way to go for you.

⚠️ Try it locally first

Before we start with the configuration, it is good to familiarize ourselves with how cloud execution works, and we recommend you to test how to trigger a cloud test from your machine.

Check out the cloud execution guide to learn how to distribute the test load across multiple geographic locations and more information about the cloud execution.

Now, we will show how to trigger cloud tests using Atlassian Bamboo. If you do not have an account with Grafana Cloud already, you should go here and start your free trial. After that, get your account token and save it somewhere temporarily.

Next, let's create a modified version of the testing script and call it cloud-test.js:

import http from 'k6/http';
import { sleep } from 'k6';

export const options = {
  ext: {
    loadimpact: {
      name: 'cloud-test',
    },
  },
  duration: '1m',
  vus: 50,
  thresholds: {
    http_req_duration: ['p(95)<500'],
  },
};

export default function () {
  const res = http.get('https://test.k6.io');
  sleep(1);
}

The ext property enables passing of the script filename which is required for cloud execution. To perform cloud runs on Bamboo, we need first to specify. the K6_CLOUD_TOKEN as an environment variable. This is done at Plan configuration > Variables tab:

Next, we need to create a new task called 'k6 Cloud Login'. To reference the token environment variable, we need to pass the expression ${bamboo.K6_CLOUD_TOKEN}. Fill in the form as shown below.

After specifying the cloud login task, we need to define a task for running our cloud_test.js:

Make sure the 'k6 Cloud Login' task comes before the 'Perform load testing' task. After saving the new configuration, you can run the plan. You should expect the following log output:

In the build output log, you should note a similar URL highlighted above. Opening this URL will take you to the test results as shown below:

Scheduling Builds

It is highly recommended to perform load testing at a time when most users aren't accessing your system. This is because performance testing often places significant stresses on your infrastructure which may cause user experience issues.

Bamboo has support for scheduling which can be enabled by navigating to Plan configuration > Trigger tab, click the 'Add trigger' button. Fill in the form as indicated below. Bamboo provides a user friendly schedule editor that allows you specify interval and time using a simple form.

POSIX cron syntax is also supported. Here is a cheat sheet for crontab scheduling expressions.

Summary

Bamboo is an excellent CI/CD software and a great alternative to Jenkins. It is much easier to configure through it's well designed web interface. It also allows advanced custom automation workflows where you can choose to run jobs in sequence or in parallel. Unfortunately, this advance customizability makes it an overkill for running simple jobs. However, that's a small price to pay compared to the freedom it provides to DevOp engineers.

As this guide has shown, it is quite easy to include performance testing as part of your CI/CD pipeline in Bamboo using k6. Testing your production releases early helps identify performance issues before they affect your customers.