Testing without limits: xk6 and k6 extensions

warning

This article is out of date, and some instructions might not work properly.

For up-to-date documentation please see our k6 extensions guides.

Introduction

k6 v0.29.0 introduced xk6 and k6 extensions to the k6 community. 🎁🎉💪

You can now extend the functionality of k6 using Go-based k6 extensions and import them as JS modules in your k6 script.

This feature opens the gates for anyone to use existing k6 extensions and write custom Go extensions for special requirements.

Before this release, importing JavaScript libraries was the only possibility to extend k6. While this approach works in many cases, it has two significant limitations:

• lack of JS support for system APIs.
• performance penalties due to the JS runtime.

k6 extensions overcome these limitations. They benefit from having excellent performance and the flexibility of using any Go libraries.

k6 ecosystem

xk6 and k6 extensions provide the foundation for unlocking a lot of functionality for the k6 ecosystem.

Traditionally, if you wanted to extend k6, you had to fork the k6 repository and submit a PR with your changes. Merging a change that could potentially affect thousands of other k6 users or introduce maintenance overheads is always a difficult proposition that k6 maintainers are very cautious about, which in some cases can cause delays in introducing new features.

Extensions allow the community to innovate and enable faster development for:

• Building new k6 integrations.
• Adding further protocol support and clients.
• Creating new APIs for high-performant or custom functions.

Extensions will accelerate the innovation of the k6 ecosystem providing a framework to validate new ideas and technologies. Finally, the k6 core team will consider extensions with strong community interest and a solid foundation for "graduation" and becoming part of k6.

About licensing

The core k6 project is licensed under the GNU AGPLv3. This means that while extensions may use a GPL-compatible license, when the extension is built with xk6 the product will be licensed as AGPLv3.

To avoid any legal concerns, we suggest extension authors use the Apache 2.0 license.

How xk6 works

xk6 is an experimental framework for extending k6 that allows you to build a custom k6 binary by adding any combination of extensions easily, without any need to know how to code. For example, building a k6 v0.39.0 binary that allows you to use the SQL and Kafka extensions in your test scripts is as simple as:

$ xk6 build v0.39.0 \
  --with github.com/grafana/xk6-sql \
  --with github.com/mostafa/xk6-kafka

This will build a k6 binary you can then use to run custom scripts that use the additional functionality.

$ ./k6 run some-script-with-sql-and-kafka.js

A couple of things to note here:

• Extensions are primarily created by other k6 users and external contributors. As such they're not supported by the k6 team and it's up to the user to determine the safety and functionality of each extension. Keep in mind that unlike a JavaScript library imported in a test script, extensions are written in Go which doesn't have the sandbox restrictions of running in a virtual machine. So make sure you carefully evaluate and trust any extension that you use. In the future the k6 team will provide a registry of trusted extensions, but for now some caution should be taken when using xk6.
• As binaries produced by xk6 can have any number of extensions and functionality, they're not supported in the cloud service. You can however run any k6 version locally, bundled with any extensions you want by xk6, and have it output its metrics to k6 Cloud!

Tutorial - Creating a Redis extension

This short tutorial will walk you through the steps to build a k6 extension for Redis and use it in your k6 test.

• Creating the k6 extension
• Building the extension with xk6
• Using the extension

Creating the k6 extension

Creating a new k6 extension is very simple if you're familiar with Go, JavaScript and the k6 APIs, but should be approachable by anyone with some programming background.

A good starting point is always seeing how existing extensions work, but in this section we're going to guide you through the steps for creating a new extension from scratch.

For this example, we're going to build an extension for a Redis client. This is a good candidate for a k6 extension as the official JavaScript library only works in Node.js and can't be directly imported in k6 test scripts.

xk6, like k6 and Go itself, is cross-platform, so while these examples are written with Linux in mind, they should work equally as well on macOS or Windows.

To start, you'll need to setup some prerequisites:

• The latest version of Go for your system. Consider using a version manager like g.
• Git is needed to pull all dependencies.
• Your favorite text editor, of course!

Since k6 extensions are plain Go modules in standard Git repositories, let's first initialize both:

$ mkdir xk6-redis
$ cd xk6-redis
$ git init
$ go mod init github.com/grafana/xk6-redis

We suggest extension authors follow the naming convention of xk6-<short name> to keep it consistent with current extensions, but this is optional and not validated in any way.

Next let's create our main Go module. Save the following as redis.go:

1package redis
2
3import (
4    "context"
5    "time"
6
7    "github.com/go-redis/redis/v8"
8
9    "go.k6.io/k6/js/common"
10    "go.k6.io/k6/js/modules"
11)
12
13// Register the extension on module initialization, available to
14// import from JS as "k6/x/redis".
15func init() {
16    modules.Register("k6/x/redis", new(Redis))
17}
18
19// Redis is the k6 extension for a Redis client.
20type Redis struct{}
21
22// Client is the Redis client wrapper.
23type Client struct {
24    client *redis.Client
25}
26
27// XClient represents the Client constructor (i.e. `new redis.Client()`) and
28// returns a new Redis client object.
29func (r *Redis) XClient(ctxPtr *context.Context, opts *redis.Options) interface{} {
30    rt := common.GetRuntime(*ctxPtr)
31    return common.Bind(rt, &Client{client: redis.NewClient(opts)}, ctxPtr)
32}
33
34// Set the given key with the given value and expiration time.
35func (c *Client) Set(key, value string, exp time.Duration) {
36    c.client.Set(c.client.Context(), key, value, exp)
37}
38
39// Get returns the value for the given key.
40func (c *Client) Get(key string) (string, error) {
41    res, err := c.client.Get(c.client.Context(), key).Result()
42    if err != nil {
43        return "", err
44    }
45    return res, nil
46}

A few things to note here:

• We register the extension in the init() function, which will be called once when this package is first imported. k6 extensions are required to have a k6/x/ prefix in their import path in order to distinguish them from other built-in modules, and the full path must be unique in each build of k6.
• The XClient method uses a feature of the Go-JS bridge in k6 that exposes a more idiomatic constructor API to JS if a method begins with an X and uses the js/common functions to bind an object to the JS runtime. This is not required, but it's preferable so that scripts can use a more natural new redis.Client() instead of e.g. redis.NewClient().
• When a client is created we store the reference to the underlying redis.Client instance. We could choose to return this instance as is, but exposing the Go API directly to JS would make some methods difficult to work with (because of the use of context.Context and other Go APIs), so we chose to wrap it instead.
• We expose only the Get and Set commands here, but other commands can be easily added.
• The error returned from the Get method will be handled by the Go-JS bridge in k6 and only a single value (string in this case) will be returned in JS. If err is not nil, it will be thrown with the appropriate message in the JS script.

This is pretty much it for the extension implementation! The only thing left is to commit all your changes to the Git repo, and optionally publish it to GitHub or elsewhere. Publishing is not necessary for building the k6 binary, as xk6 can work with a local directory as well, but the k6 community would appreciate it if you did. :)

If you do publish it on GitHub, make sure to add the xk6 topic to the repo so that other users can easily find it.

Building the extension with xk6

First install xk6 with:

$ go install go.k6.io/xk6/cmd/xk6@latest

Next, let's build the k6 binary. To use the published version of the extension run:

$ xk6 build v0.39.0 --with github.com/grafana/xk6-redis

Or if you're working with a local directory run the following, replacing the path as needed:

$ xk6 build v0.39.0 \
  --with github.com/grafana/xk6-redis="/absolute/path/to/xk6-redis"

The first argument to xk6 build is the k6 version to use, so instead of v0.32.0 you can use any branch name or commit SHA here as well, as long as it's newer than v0.29.0 since this was the first version to support xk6.

Running the above should output something like:

2021/05/14 12:31:38 [INFO] Temporary folder: /tmp/buildenv_2021-05-14-1231.020008841
2021/05/14 12:31:38 [INFO] Writing main module: /tmp/buildenv_2021-05-14-1231.020008841/main.go
2021/05/14 12:31:38 [INFO] Initializing Go module
2021/05/14 12:31:38 [INFO] exec (timeout=10s): /home/ivan/.local/go/bin/go mod init k6
go: creating new go.mod: module k6
go: to add module requirements and sums:
        go mod tidy
2021/05/14 12:31:38 [INFO] Replace github.com/grafana/xk6-redis => /home/ivan/Projects/grafana/xk6-redis
2021/05/14 12:31:38 [INFO] exec (timeout=10s): /home/ivan/.local/go/bin/go mod edit -replace github.com/grafana/xk6-redis=/home/ivan/Projects/grafana/xk6-redis
2021/05/14 12:31:38 [INFO] Pinning versions
2021/05/14 12:31:38 [INFO] exec (timeout=0s): /home/ivan/.local/go/bin/go get -d -v go.k6.io/k6@v0.32.0
go get: added go.k6.io/k6 v0.32.0
2021/05/14 12:31:39 [INFO] Build environment ready
2021/05/14 12:31:39 [INFO] Building k6
2021/05/14 12:31:39 [INFO] exec (timeout=0s): /home/ivan/.local/go/bin/go mod tidy
go: found github.com/grafana/xk6-redis in github.com/grafana/xk6-redis v0.0.0-00010101000000-000000000000
2021/05/14 12:31:39 [INFO] exec (timeout=0s): /home/ivan/.local/go/bin/go build -o /home/ivan/Projects/grafana/xk6-redis/k6 -ldflags -w -s -trimpath
2021/05/14 12:31:39 [INFO] Build complete: ./k6
2021/05/14 12:31:39 [INFO] Cleaning up temporary folder: /tmp/buildenv_2021-05-14-1231.020008841

And produce a k6 binary in the current directory.

Using the extension

Now we can write our k6 test script that uses the Redis extension.

Create a test.js file with the following contents:

1import redis from 'k6/x/redis';
2
3const client = new redis.Client({
4  addr: 'localhost:6379',
5  password: '',
6  db: 0,
7});
8
9export default function () {
10  client.set('mykey', 'myvalue', 0);
11  console.log(`mykey => ${client.get('mykey')}`);
12}

Notice that we create the Redis client in the init context, passing it the server address and some options for demostration purposes. Then in the default function we set a key and log the retrieved value.

Speaking of the server, before you run this script ensure that Redis is actually running of course. You can do this easily with Docker and the official Redis image by running:

$ sudo docker run -it --rm --name redis -p 127.0.0.1:6379:6379 redis

Now we can finally run our test script with:

$ ./k6 run test.js

Which should output something like:

          /\      |‾‾| /‾‾/   /‾‾/
     /\  /  \     |  |/  /   /  /
    /  \/    \    |     (   /   ‾‾\
   /          \   |  |\  \ |  (‾)  |
  / __________ \  |__| \__\ \_____/ .io

  execution: local
     script: test.js
     output: -

  scenarios: (100.00%) 1 scenario, 1 max VUs, 10m30s max duration (incl. graceful stop):
           * default: 1 iterations for each of 1 VUs (maxDuration: 10m0s, gracefulStop: 30s)

INFO[0000] mykey => myvalue                              source=console

running (00m00.0s), 0/1 VUs, 1 complete and 0 interrupted iterations
default ✓ [======================================] 1 VUs  00m00.0s/10m0s  1/1 iters, 1 per VU

    data_received........: 0 B 0 B/s
    data_sent............: 0 B 0 B/s
    iteration_duration...: avg=834.68µs min=834.68µs med=834.68µs max=834.68µs p(90)=834.68µs p(95)=834.68µs
    iterations...........: 1   54.622575/s

We can also confirm the key was written with redis-cli:

$ sudo docker exec -it redis redis-cli get mykey
"myvalue"

Existing k6 extensions

The k6 extension ecosystem is still in its infancy, but we believe the tooling is approachable enough for it to grow beyond our expectations. For now, here are a few extensions written by the k6 team and passionate users, and you can always find an up-to-date list by checking out the xk6 topic on GitHub and our curated list of extensions.

• xk6-datadog: an extension for querying Datadog metrics.
• xk6-kafka: produce and consume Kafka messages in Avro format.
• xk6-notification: a Slack and Teams notification library.
• xk6-redis: an alternative Redis client similar to the one we created in this tutorial.
• xk6-sql: a SQL extension to test against PostgreSQL, MySQL and SQLite.
• xk6-url: an extension for parsing and normalizing URLs.
• xk6-zmq: a ZeroMQ client.
• xk6-ssh: a client for performing commands over SSH.
• xk6-docker: a client for interacting with Docker.

Help and feedback

While we strived to make this system as easy to use for extension writers and k6 users, it's still a project in the experimental phase, and we'd love to hear your feedback about it! Likewise if you need help troubleshooting an extension or xk6 itself feel free to jump on Slack or the community forum. Have fun!