By default, at the end of every local test run, k6 prints a summary report to stdout that contains a general overview of your test results. It includes aggregated values for all built-in and custom metrics and sub-metrics, thresholds, groups, and checks. It can look somewhat like this:
A few options can affect how this report behaves:
- The --summary-trend-stats option allows you to define which stats for Trend metrics will be calculated and shown.
- The --summary-time-unit option forces k6 to use a fixed time unit for all time values in the summary.
- no-summary completely disables the report generation. Since k6 v0.30.0 that includes --summary-export and handleSummary() (see below).
Summary export to a JSON file
Since version 0.26.0 k6 has had the --summary-export=path/to/file.json option for local test runs. It exports some of the summary report data to a JSON file format.
Unfortunately, the exported format is somewhat limited and has a few confusing peculiarities. For example, groups and checks are unordered. Threshold values are also somewhat unintuitive - they signify whether the threshold has been crossed. So, true is the "bad" threshold value, i.e. when the threshold has failed, and false is the "good" value...
We couldn't change the --summary-export data format because it would have broken backwards compatibility in a feature that people depended on in CI, so it still works how it used to. However, in k6 v0.30.0, we introduced handleSummary() - a new and better way to make JSON exports of the summary data, as well as any other format (CSV, XML (JUnit/xUnit/etc.), HTML, TXT, etc.) that may be required. We strongly recommend everyone to use handleSummary() instead of --summary-export. For more details, see the next section in this document...
handleSummary() callback
Starting with k6 v0.30.0, users can now completely customize the end-of-test summary report!
You can now export a function called handleSummary() and k6 will call it at the end of the test run, even after teardown(). handleSummary() will be called with a JS object containing the same information that is used to generate the end-of-test summary and --summary-export, and allows users to completely customize how the end-of-test summary looks like.
Besides customizing the end-of-test CLI summary (if handleSummary() is exported, k6 will not print the default), you can also transform the summary data to various machine or human-readable formats and save it to files. This allows the creation of JS helper functions that generate JSON, CSV, XML (JUnit/xUnit/etc.), HTML, etc. files from the summary data.
You can also send the generated reports to a remote server by making an HTTP request with them (or using any of the other protocols k6 already supports)! Here's a simple example:
k6 expects handleSummary() to return a {key1: value1, key2: value2, ...} map. The values can be a string or ArrayBuffer, and represent the generated summary report contents. The keys should be strings and determine where the contents will be displayed or saved:
- stdout for standard output
- stderr for standard error,
- or any relative or absolute path to a file on the system (which will be overwritten)
The format of the data parameter is similar but not identical to the data format of --summary-export. The format of --summary-export remains unchanged, for backwards compatibility, but the data format for this new k6 feature was made more extensible and had some of the ambiguities and issues from the previous format fixed.
To get an idea how data would look like in your specific test run, just add return { 'raw-data.json': JSON.stringify(data)}; in your handleSummary() function and inspect the resulting raw-data.json file. Here's a very abridged example of how it might look like:
This feature is only available for local k6 run tests for now, though we plan to support k6 cloud tests eventually. And, as mentioned in the snippet above, the JS helper functions that transform the summary in various formats are far from final, so keep an eye on jslib.k6.io for updates. Or, better yet, submit PRs with improvements and more transformations at https://github.com/loadimpact/jslib.k6.io