We’re excited to launch xk6-browser at Grafana ObservabilityCON today, an extension to k6 adding support for browser automation via the Chrome Devtools Protocol (CDP).
k6 was built because we weren’t satisfied with the developer experience offered by existing load testing solutions. We believe in shifting performance testing, and with it observability, to the left. We promote a proactive approach to building and operating reliable systems, uncovering issues before they’re released to the end user. We encourage the inclusion of load tests in your CI pipeline to prevent system updates from introducing performance regressions.
With xk6-browser, you can interact with the browser to test your web applications end-to-end while accessing all of the k6 core features, including protocol-level APIs and other k6 extensions. It's a single tool for both protocol and browser-level testing.
Why browser testing?
Well, for starters it's been the most requested feature for years. We had resisted adding browser-level testing support to k6 as we wanted to focus on testing backend systems first, making sure we provided a good developer experience for those use cases before tackling other areas of testing.
The golden rule of web performance states that 80-90% of the load time of a web page or application is spent in the frontend. It thus makes sense to start optimizing performance in the frontend. Our counter argument to this has always been that this ratio between load time spent in backend vs frontend very quickly can flip when a backend system approaches its concurrency limit (as illustrated by the graphic below). In our experience it's very common that teams aren't aware of what their system's concurrency limit is and thus have no idea of when the flip will happen.
xk6-browser allows for mixing browser-level and protocol-level APIs. You can now simulate the bulk of traffic with protocol-level scenario(s) as usual, and, at the same time, have a Virtual User interacting with your web site or application using a real browser to collect frontend metrics (DOM content loaded, load, first contentful paint, etc.). You can finally understand both sides of the coin with k6!
How to get started
We have initially built this browser-level capability as a k6 extension. This allows us to experiment without changes to the k6 codebase. To use browser-level APIs in your k6 test, you need a k6 binary built with the xk6-browser extension. You can either download one of the release binaries from GitHub, or build a custom version of k6 from source.
To build from source, first ensure you have the prerequisites:
- Go toolchain
Build k6 version.
xk6 creates the xk6-browser binary in the current working directory. This file can be used exactly the same as the main k6 binary, with the addition of being able to run xk6-browser scripts.
Create a k6 test and import the k6/x/browser module.
Run the k6 test using the xk6-browser binary created on the first step.
When xk6-browser launches and interacts with the browser, it will automatically collect frontend metrics and report them as with other k6 metrics.
With xk6-browser, k6 will now report the following list of new frontend metrics:
- browser_dom_content_loaded: Time from start of document load until DOMContentLoaded event is fired.
- browser_loaded: Time from start of document load until load event is fired.
- browser_first_paint: Time from start of document load until first paint event happens.
- browser_first_contentful_paint: Time from start of document load until first contentful paint happens.
- browser_first_meaningful_paint: Time from start of document load until first meaningful paint happens.
The xk6-browser API aims for rough compatibility with the Playwright API for NodeJS, meaning k6 users don't have to learn an entirely new API.
Note that because k6 does not currently run in NodeJS or support the Event Loop. xk6-browser APIs are synchronous and can slightly differ from their Playwright counterpart.
For browser support, the initial focus is on providing solid support for Chromium-based browsers. Over time we want to support Firefox and Webkit-based browsers as well.
The team is working on project stability and reliability. End-to-end tests are sometimes known to be flaky and unreliable, producing failures and inconsistent results. In that case, the testing process can become very inefficient. Reducing flakiness is a primary goal, so you can be confident in your testing.
We’re adding more APIs and looking forward to feedback from users to aid us in prioritizing what to build next! Join the #xk6-browser channel on the k6 Slack and let us know what you think.
You can find out more by reading the documentation and GitHub repository. If you get stuck or have any questions for the team, please get in touch with us on the k6 Forum, GitHub, or Slack.
Special acknowledgement to the authors of Playwright and Puppeteer. This project is heavily influenced, and in some regards, based on their code.