How to Perform Load Testing with k6 using Buddy CI/CD

Introduction

In this guide, we'll look at adding performance testing to your development workflow with Buddy and k6.

k6 is an open-source load testing tool for testing the performance of APIs, microservices, and websites. Developers can use k6 to test a system's performance under a particular load to catch performance regressions or errors.

Buddy is a continuous integration and deployment platform used for automating software development workflows such as building, testing, and deploying releases into production.

Creating the performance test

Let's start by writing a simple performance test in JavaScript. First, we'll take care of the basics to which we'll add additional logic and create a more advanced scenario as we go.

In its final form, the test will be made up of these parts:

• An HTTP request sent to the address with our application or website.
• The definition of testing parameters, such as the duration of the test and the number of virtual users used to put load on the app.
• A threshold, which is the SLO (service level objective) or a performance goal that our test looks for.

In the first iteration, our script simulates a virtual user accessing the specified website. To give the tested site a bit of a breather, we'll use a 1-second-long sleep to create a pause between the subsequent requests.

// ./test.js
import http from 'k6/http';
import { sleep } from 'k6';

export default function () {
  const res = http.get('https://test.k6.io');
  sleep(1);
}

Defining test parameters

With the basics in place, let's define the test parameters. In the code block below, you can see that we added the options array that defines the number of virtual users (vus) k6 uses to load the specified site and the time (duration) over which this load is applied.

In this setup, we'll see 50 virtual users sending HTTP GET requests to the specified website. The duration of the test is set to 1 minute. With the 1-second-long sleep we implemented, the test will end after running about 2900 iterations.

// ./test.js
import http from 'k6/http';
import { sleep } from 'k6';

export const options = {
  duration: '1m',
  vus: 50,
};

export default function () {
  const res = http.get('https://test.k6.io');
  sleep(1);
}

You can run this test locally if you installed k6 on your machine. Open a terminal and run this command: k6 run test.js.

Adding Thresholds

To make this test a true guardian of quality in our CI/CD setup, let's define the performance goal the test checks for. The performance goal or SLO is a measurement of a selected metric that indicates the desired performance at a given load.

In k6, this is handled by using Thresholds that allow you to set SLOs as the test's pass/fail criteria. If the results of the test don't meet the specified threshold, k6 ends the test run with a non-zero exit code. This lets Buddy know that the performance test failed and, by marking the pipeline execution as failed, gives you a clear signal that you should take a closer look at your code.

For this example, let's add a request response time Threshold. With the new lines added to the script, k6 considers the test as passed only when the 95th percentile request response time stays below 500ms. It's a great starting point that you can modify and expand upon to better suit your needs.

// ./test.js
import http from 'k6/http';
import { sleep } from 'k6';

export const options = {
  duration: '1m',
  vus: 50,
  thresholds: {
    http_req_duration: ['p(95)<500'],
  },
};

export default function () {
  const res = http.get('https://test.k6.io');
  sleep(1);
}

Thresholds in k6 are a powerful feature that allows us to define multiple Pass/Fail criteria in the same test. Basically, any of the metrics tested by k6 can be made into a Threshold, for example:

• The 99th percentile request response time must be lower than 600 ms.
• The 95th percentile request response time must be lower than 400 ms.
• The number of failed requests cannot exceed 1%.
• The number of responses without errors must not fall below 95%.

Take a look at the k6 Thresholds documentation for additional details on the API and its usage.

Setting up the workflow in Buddy

Creating CI/CD workflows in Buddy is done by adding and configuring actions, which represent the subsequent steps of your code's journey. You can easily create a pipeline for k6 load tests, or add the test to any existing workflow using the Custom Build action.

The Custom Build action launches an isolated Ubuntu container and allows you to configure its environment and run any commands you need.

There are two prerequisites for this section:

• Upload the .js file with the load test to your repository
• Create a free account in Buddy

After you've done that, sign in to Buddy and follow these steps:

1. Create a new project. Choose the Git provider you use to host your repository and select the repo with which you want to work. Buddy works with GitHub, GitLab, Bitbucket, private Git servers, and offers in-house Git hosting.

   

2. Add a new pipeline. Define the name of the pipeline, choose the trigger mode (which defines when the pipeline runs), and select the branch the pipeline is triggered by. The available trigger modes are:

   - Manual

   - On push - your pipeline runs whenever new commits are pushed to the selected branch.

   - Recurrently - allows you to set specific time intervals at which the pipeline runs.

   

3. Add the Custom Build action to your pipeline. Use the search box to find it, or scroll through the list of available, purpose-built actions.

   

4. Configure the Ubuntu container's environment. Switch to the Environment tab, set the user to root, and paste the k6 installation commands into the terminal:

   apt-get update &&  apt-get install ca-certificates gnupg2 -y
   apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys C5AD17C747E3415A3642D57D77C6C491D6AC1D69
   echo "deb https://dl.k6.io/deb stable main" | tee /etc/apt/sources.list.d/k6.list
   apt-get update
   apt-get install k6

   Copy

   These commands run only once, on the action's first execution. As a result, subsequent executions of this action are even quicker.

   

5. Switch to the Run tab and enter the command to run the k6 test in the terminal. The action has access to your repository which is copied to the pipeline's filesystem on the first execution. You can use this exact command if you uploaded the script to the root level of your repo:

   k6 run test.js

   Copy

   

6. When you're done, click Add this action to finish the configuration.

   At this point, all that's left to do is to run the pipeline. No matter which trigger mode you choose, you can always run your pipeline by clicking the Run pipeline button.

   

   From there, you can choose the revision for which you want to run the pipeline and manage additional options. If you just want to test the latest commit, hit Run now.

   As the pipeline runs, click the Logs button to view the logs of any running or completed action. In our case, the logs for the Custom Build action contain the results of the k6 test:

   

Triggering test execution in Grafana Cloud k6

You can run k6 tests in your CI/CD pipeline in one of two ways:

• Locally on the CI server, tasking it to perform the test.
• In Grafana Cloud k6, using k6's horsepower and one or multiple geographic locations.

Running the tests in the cloud can come in handy when:

• You want to run the test from different geographic locations (load zones).
• You want to run a resource-intensive test and delegate it to infrastructure other than this of your CI.

If any of those reasons fit your needs, then running k6 cloud tests is the way to go for you.

⚠️ Try it locally first

Before we start with the configuration, it is good to familiarize ourselves with how cloud execution works, and we recommend you test how to trigger a cloud test from your machine.

Check out the cloud execution guide to learn how to distribute the test load across multiple geographic locations and more information about the cloud execution.

Now, we will show how to trigger cloud tests using Buddy. If you do not have an account with Grafana Cloud already, you should go here and start your free trial. After that, get your account token and save it somewhere.

All you have to do to run the cloud test is to modify the Custom Build action we set up in the previous section.

In the terminal in the action's Run tab, add a command to sign in to k6 and adjust the other command to run the created test in the cloud:

If you're planning to work on your pipelines in collaboration with other users, it's a good idea to hide the token used to sign in to k6 cloud. You can do this easily by adding the token as an encrypted environment variable.

You can do this right from the Custom Build action's configuration view. Go to the Variables tab and add a new variable with encryption enabled.

With the environment variable set, you can use it in the command that signs in to Grafana Cloud k6:

Scheduled testing

Load testing is best performed at the time of low traffic at the specified address. After all, you wouldn't want the load you put on the application by running k6 tests to negatively impact your users' experience.

In Buddy, you can have your pipelines run on a schedule by choosing the Reccurently trigger mode.

This mode allows you to quickly configure recurrence in the simple mode, or tailor the schedule precisely to your needs in the advanced mode, which supports cron syntax.

Check out this useful online tool for creating cron expressions which you can use to schedule pipeline runs in Buddy.

Summary

Buddy is a powerful CI/CD solution that is a great alternative to Jenkins, Circle CI, and other DevOps tools. Its main draw is the simplicity of use thanks to the tool's intuitive UI. It offers everything you need to create custom, advanced workflows and empowers all developers on the team to create and maintain pipelines without the need for a dedicated DevOps engineer.

As this guide has shown, it is remarkably easy to add performance testing as part of your CI/CD pipeline in Buddy using k6. Testing your production releases early helps identify performance issues before they affect your customers.