k6 v0.30.0 released

k6 v0.30.0 is here! 🎉 It was a bit of a slow after-holiday release, but it still packs a few major new features and improvements that users have been requesting for a long time. For additional details, see the full release notes on GitHub

Share memory between VUs using read-only arrays

Being able to share memory between VUs has long been a common wish at the top of the feature wishlist. This feature is especially useful while running tests that require huge amounts of test data, at scale.

Let's say you have a huge users.json file with test users for your application:

[
  { username: 'user1', password: 'password1', other: 'some-long-data-....-1' },
  { username: 'user2', password: 'password2', other: 'some-long-data-....-2' },

  // ...

  { username: 'user999999', password: 'password999999', other: 'some-long-data-....-999999' },
];

If you just use JSON.parse(open('users.json')) in your script, then every VU will have a copy of the whole huge data set. Every VU in k6 is a separate JavaScript runtime, so there wasn't a thread-safe way to share data between them.

Until now, that is! 🎉

SharedArray

We've added a new built-in SharedArray object in the new k6/data module that allows VUs to share read-only data:

import { SharedArray } from 'k6/data';
import { sleep } from 'k6';
import http from 'k6/http';

const users = new SharedArray('someName', function () {
  // This function will be called only once, in the first init context
  // execution. Every other VU will just get a memory-safe read-only reference
  // to the already loaded data.
  console.log('Loading users.json, this happens only once...');
  // You are not restricted to JSON, you can do anything - parse a CSV or XML
  // file, generate random data, etc. - as long as you return an array.
  return JSON.parse(open('users.json'));
});

export const options = { vus: 10, duration: '30s' };
export default function () {
  const randomUserID = Math.floor(Math.random() * users.length);
  const user = users[randomUserID]; // alternatively, we can also use __VU and/or __ITER
  console.log(`VU ${__VU} is running iteration ${__ITER} with user ${user.username}...`);
  http.post('https://httpbin.test.k6.io/post', JSON.stringify(user));
  sleep(Math.random() * 2); // or, better yet, use arrival-rate
}

Notice how Loading users.json is logged only once, but each VU uses the users variable like a normal JS array. The data is read only once and only a single copy of the huge array will be stored in memory.

Behind the scenes, k6 uses a JS Proxy to transparently copy only the row each VU requests in users[randomUserID] to it. This on-demand copying is a bit inefficient, but it's still leagues better than having a copy of the huge array in every VU!

This also means that you can avoid the copying in every iteration by pinning the data used by every VU and having let user = users[randomUserID] in the init context! And yes, you can have multiple SharedArray objects in the same script, just make sure to give them unique names - this is what someName in the script above is used for.

Because VUs are independent JS runtimes, we need some way to differentiate between the different shared memory objects, so we require them to have unique names. These names are also the IDs that any xk6 extensions would need to use to access them.

Availability

This feature is available both locally and in the cloud. We advise everyone who deals with large data files to wrap them in a SharedArray and give this new feature a try. The required script changes should be minimal, while the memory usage should be significantly lower.

Hopefully, we can finally consider one of the biggest blockers k6 users have had for a long time solved! 🎉

Generating custom summaries at the end of a test

You can now export a function called handleSummary() and k6 will call it at the end of the test run.

After teardown(), handleSummary() will be called with a JS object containing the same information used to generate the end-of-test summary and --summary-export. This allows users to completely customize how the end-of-test summary looks like.

Besides customizing the end-of-test CLI summary (if handleSummary() is exported, k6 will not print the default), you can also transform the summary data to various machine or human-readable formats and save it to files. This allows the creation of JS helper functions that generate JSON, CSV, XML (JUnit/xUnit/etc.), HTML, etc. from the summary data.

Even binary formats like PDF are not out of reach, potentially, with an appropriate JS library that works in k6. You can also send the generated reports to a remote server by making an HTTP request with them (or using any of the other protocols k6 already supports).

Example

import http from 'k6/http';
import k6example from 'https://raw.githubusercontent.com/grafana/k6/master/examples/thresholds_readme_example.js';

export default k6example; // use some predefined example to generate some data
export const options = { vus: 5, iterations: 10 };

// These are still very much WIP and untested, but you can use them as is or write your own.
import { jUnit, textSummary } from 'https://jslib.k6.io/k6-summary/0.0.1/index.js';

export function handleSummary(data) {
  console.log('Preparing the end-of-test summary...');

  // Send the results to some remote server or trigger a hook
  const resp = http.post('https://httpbin.test.k6.io/anything', JSON.stringify(data));
  if (resp.status != 200) {
    console.error('Could not send summary, got status ' + resp.status);
  }

  return {
    'stdout': textSummary(data, { indent: ' ', enableColors: true }), // Show the text summary to stdout...
    'junit.xml': jUnit(data), // but also transform it and save it as a JUnit XML...
    'summary.json': JSON.stringify(data), // and a JSON with all the details...
    // And any other JS transformation of the data you can think of,
    // you can write your own JS helpers to transform the summary data however you like.
  };
}

k6 expects handleSummary() to return a {key1: value1, key2: value2, ...} map. The values can be string or ArrayBuffer and represent the generated summary report contents. The keys should be strings and determine where the contents will be displayed or saved: stdout for standard output, stderr for standard error, or a path to a file on the system (which will be overwritten).

The format of the data parameter is similar but not identical to the data format of --summary-export. The format of --summary-export remains unchanged, for backwards compatibility, but the data format for this new k6 feature was made more extensible and had some of the ambiguities and issues from the previous format fixed.

We can't cover the new format in the release notes, though you can easily see what it contains by using return { 'stdout': JSON.stringify(data)}; in handleSummary(). 😄

Availability

This feature is only available for local k6 run tests for now, though we plan to support k6 cloud tests eventually. And, as mentioned in the snippet above, the JS helper functions that transform the summary in various formats are far from final, so keep an eye on jslib.k6.io for updates.

Or, better yet, submit PRs with improvements and more transformations at grafana/jslib.k6.io 😄

Breaking changes

• --no-summary now also disables --summary-export. You can recreate the previous behavior of k6 run --no-summary --summary-export=summary.json script.js by having an empty exported handleSummary() function in your script (so that the default text summary is not shown by k6) and executing only k6 run --summary-export=summary.json script.js. Another option is omitting --summary-export as well and using handleSummary() as shown above.
• Integer values for duration and similar time values in the exported script options and environment variables are now treated as milliseconds. Similarly, the timeout option in http.Params can now be "stringy", e.g. "30s", "1m10s", etc. This was previously undefined behavior, but instead of k6 erroring out, it silently accepted and treated such values as nanoseconds.
• The official docker image now sets the workdir to /home/k6.

Other enhancements

• Windows releases of k6 are now digitally signed, reducing both the amount and severity of warnings seen on execution.
• Further improvements to the js runtime, making --compatibility-mode=base even more feature-rich.
• k6 now accepts ArrayBuffer values for the HTTP request body.
• The official docker image now sets the workdir to /home/k6.
• A lot of bugfixes and testing improvements.

As always, we appreciate the community feedback on our tool, k6. Please test it, and report any issues, either on GitHub or the community forum. We also welcome any contributions.