09 January 2023

Get Started with xk6-browser

Marie Cruz

This post explains how to get started with xk6-browser, a k6 extension that adds browser-level APIs to interact with browsers and collect web performance metrics as part of your k6 tests.

Testing Beyond Protocol Level

Over the years, k6 has become known as a performance testing tool that provides the best developer experience. Most of our efforts have focused on providing a tool that helps test your servers or backend systems. However, we acknowledge that backend performance testing only addresses half of your performance testing efforts.

Suppose you test the user experience of your website and verify that there are no performance issues on a specific user journey. In that case, you need to drive some of your performance testing efforts from a browser perspective and consider a more realistic end-to-end test of the user flow.

Most load-testing tools focus on testing API endpoints, but it's different from what your users normally interact with. Your users interact with the browser, so it's also vital to test the browser's performance to get an end-to-end perspective of what's happening when interacting with your web applications.

Here at k6, we want to start expanding our performance testing use case and also test beyond the protocol level.

This is where xk6-browser comes in.

What is xk6-browser?

xk6-browser brings browser automation and end-to-end web testing to k6 while supporting core k6 features. It enables you to get insights from your front-end application during performance testing.

You can mix browser-level and protocol-level tests in a single and unified script using xk6-browser. This can simulate the bulk of your traffic from your protocol-level tests and run one or two virtual users on a browser level to mimic how a user interacts with your website, thus leveraging a hybrid approach to performance testing.

xk6-browser offers a unique solution as you don’t have to use separate tools to test your frontend and backend systems. xk6-browser also offers a simplified experience and aggregated view of performance metrics.

Get Started with xk6-browser

To get started, the first thing that you need to do is install xk6-browser. Currently, it’s being developed as a k6 extension. However, we have plans soon to add it as an experimental module in k6 core.

Writing the test

Once you have installed xk6-browser, you can copy one of our example scripts to get started as part of our xk6-browser documentation.

script.js
import { chromium } from 'k6/x/browser';
export default function () {
const browser = chromium.launch({ headless: false });
const page = browser.newPage();
}

Let’s break down what’s happening with the preceding code.

  1. We are importing chromium from the k6/x/browser module. chromium is of type BrowserType, which is xk6-browser’s entry point into launching a browser process.
  2. Next, we use the scenario function, an existing k6 functionality, to define our VU (virtual user) code.
  3. To create a new browser instance, we use the launch method of chromium, which returns a Browser object. You can pass different parameters within launch and one of the parameters you can pass is headless, which you can use to show the browser or not.
  4. To create a new page in your browser instance, we use the browser.newPage().

Now, on to the fun part! Let’s simulate a user visiting a test application and logging in.

script.js
import { chromium } from 'k6/x/browser';
import { check } from 'k6'
export default function () {
const browser = chromium.launch({ headless: false });
const page = browser.newPage();
page
.goto('https://test.k6.io/my_messages.php', { waitUntil: 'networkidle' })
.then(() => {
// Enter login credentials and login
page.locator('input[name="login"]').type('admin');
page.locator('input[name="password"]').type('123');
// Wait for asynchronous operations to complete
return Promise.all([
page.waitForNavigation(),
page.locator('input[type="submit"]').click(),
]).then(() => {
check(page, {
'header': page.locator('h2').textContent() == 'Welcome, admin!',
});
});
}).finally(() => {
page.close();
browser.close();
});
}

There are a lot of things happening in the preceding code especially the introduction of asynchronous operations so let’s break it down again.

  1. We visit the page by using page.goto and pass the test application URL. We are also waiting for the network to be idle, which will succeed if there are no network connections for at least 500 ms. page.goto is also an asynchronous operation, so this returns a promise.
  2. Once the promise has been resolved, we use page.locator to interact with the elements we want. In the example, we are creating two locators. One for the login name and another one for the login password. We use the type method to type the name and password into the fields.
  3. To click the login button, we use the click method, which is an asynchronous operation. Clicking the submit button also causes page navigation which we need to wait to load, so page.waitForNavigation(), another asynchronous operation, is needed because the page won't be ready until the navigation completes.
  4. Since there are two asynchronous operations, we need to use Promise.all([]) to wait for the two promises to be resolved before continuing to avoid any race conditions.
  5. Next, we use the check feature from k6 to assert the text content of a specific element.
  6. Finally, we close the page and the browser.

Running the Test

To run the test, open your terminal of choice, navigate to the directory where you have the xk6-browser binary installed, and use the following command.

xk6-browser run script.js

If you face any issues running the command, please check out our documentation for running the test.

You should see a similar test run as the video below:

With the browser launching, this provides a more visual experience as to what your users actually see so you can also find blind spots and catch issues related to browsers that won't be detected on a protocol-level.

Browser Metrics

When it's finished running, apart from the metrics that k6 already tracks, additional browser metrics are tracked as part of the k6 summary output.

browser_dom_content_loaded.......: avg=36.72ms min=544µs med=22.61ms max=87.02ms p(90)=74.14ms p(95)=80.58ms
browser_first_contentful_paint...: avg=47.52ms min=22.01ms med=47.52ms max=73.02ms p(90)=67.92ms p(95)=70.47ms
browser_first_meaningful_paint...: avg=75.22ms min=75.22ms med=75.22ms max=75.22ms p(90)=75.22ms p(95)=75.22ms
browser_first_paint..............: avg=45.72ms min=21.96ms med=45.72ms max=69.49ms p(90)=64.73ms p(95)=67.11ms
browser_loaded...................: avg=38.14ms min=5.28ms med=22.45ms max=86.68ms p(90)=73.83ms p(95)=80.26ms
note

The metrics above are still subject to change. Our goal is to be able to report the Core Web Vitals as well as Other Web Vitals for a better understanding of user experience.

xk6-browser API

The xk6-browser API aims to provide a rough compatibility with the Playwright API for NodeJS, meaning k6 users don't have to learn an entirely new API.

At the moment, the k6 API is synchronous. However, since many browser operations happen asynchronously, and in order to follow the Playwright API more closely, we are working on migrating most xk6-browser methods to be asynchronous. This means that while xk6-browser is ready to be used, be warned that our API is still undergoing a few changes.

At the moment, few methods such as page.goto(), page.waitForNavigation() and Locator.click() return JavaScript promises. In the future, we will be able to support async and await syntax for simplicity.

For more examples on how to use the xk6-browser API, please check out xk6-browser examples.

A Hybrid Approach to Performance Testing

If you only consider web performance, this can lead to false confidence in overall application performance when the amount of traffic against an application increases.

It's still highly recommended to also test your backend systems to have a complete picture of your application’s performance, via the protocol-level.

However, there are problems associated with testing via the protocol level, such as:

  • not being closer to the user experience since it’s skipping the browser,
  • scripts can be lengthy to create and difficult to maintain as the application grows,
  • browser performance metrics are ignored.

On the other hand, if you perform load testing by spinning up a lot of browsers, this requires significantly more load-generation resources, which can end up quite costly.

To address the shortcomings of each approach, a recommended practice is to adopt a hybrid approach to performance testing, which is a combination of testing both the backend and frontend systems via protocol and browser level. With a hybrid approach, you spin up the majority of your load via the protocol level, then simultaneously have one or two browser-level virtual users, so you can also have a view of what’s happening on the front end.

An illustration of k6 testing frontend system as well as backend systems for a hybrid approach to performance testing
A Hybrid Approach to Performance Testing

The great thing with xk6-browser is that it can offer you this hybrid approach to performance testing. While you can do this with multiple tools, the beauty of using xk6-browser is that it's built on top of k6, which means that you can have a protocol-level and a browser-level test in the same script.

Let's see how that translates to code.

Writing the test

A common scenario that we recommend is to mix a smaller subset of browser-level tests with a larger protocol-level test. To run a browser-level and protocol-level test concurrently, you can use scenarios.

script.js
1import { chromium } from 'k6/x/browser';
2import { check } from 'k6';
3import http from 'k6/http';
4
5export const options = {
6 scenarios: {
7 browser: {
8 executor: 'constant-vus',
9 exec: 'browser',
10 vus: 1,
11 duration: '10s',
12 },
13 news: {
14 executor: 'constant-vus',
15 exec: 'news',
16 vus: 20,
17 duration: '1m',
18 },
19 },
20};
21
22export function browser() {
23 const browser = chromium.launch({ headless: false });
24 const page = browser.newPage();
25
26 page
27 .goto('https://test.k6.io/browser.php', { waitUntil: 'networkidle' })
28 .then(() => {
29 page.locator('#checkbox1').check();
30
31 check(page, {
32 'checkbox is checked': (p) =>
33 p.locator('#checkbox-info-display').textContent() === 'Thanks for checking the box',
34 });
35 })
36 .finally(() => {
37 page.close();
38 browser.close();
39 });
40}
41
42export function news() {
43 const res = http.get('https://test.k6.io/news.php');
44
45 check(res, {
46 'status is 200': (r) => r.status === 200,
47 });
48}

Let's break down the preceeding code again.

  1. We are using options to configure our test-run behaviour. In this particular script, we are declaring two scenarios to configure specific workload, one for the browser-level test called browser and one for the protocol-level test called news.
  2. Both the browser and news scenario are using the constant-vu executor which introduces a constant number of virtual users to execute as many iterations as possible for a specified amount of time.
  3. Next, there are two JavaScript functions declared, browser() and news(). These functions contain the code that will be executed by a virtual user. The browser() function, represents our browser-level test and simply visits a test URL, clicks a checkbox and verifies if the checkbox has been ticked successfully while the news() function, which represents our protocol-level test, sends a GET request to a different URL and checks if the status code is returning 200.
  4. Since we are using scenarios, the two functions are independent from each other and therefore, runs in parallel.

Running the test

Using the same xk6-browser run command as above, you should see a similar test output as below:

running (1m00.1s), 00/21 VUs, 12953 complete and 0 interrupted iterations
browser ✓ [======================================] 1 VUs 10s
news ✓ [======================================] 20 VUs 1m0s
✓ status is 200
✓ checkbox is checked
browser_dom_content_loaded.......: avg=12.27ms min=68µs med=11.76ms max=26.77ms p(90)=25.56ms p(95)=26.36ms
browser_first_contentful_paint...: avg=21.93ms min=12.5ms med=25.32ms max=26.19ms p(90)=25.83ms p(95)=26.01ms
browser_first_paint..............: avg=21.88ms min=12.45ms med=25.27ms max=26.14ms p(90)=25.78ms p(95)=25.96ms
browser_loaded...................: avg=12.18ms min=984µs med=11.74ms max=25.65ms p(90)=24.37ms p(95)=25.19ms
checks...........................: 100.00% ✓ 129530
data_received....................: 21 MB 341 kB/s
data_sent........................: 1.4 MB 24 kB/s
http_req_blocked.................: avg=2.14ms min=1µs med=2µs max=290.37ms p(90)=4µs p(95)=4µs
http_req_connecting..............: avg=1.09ms min=0s med=0s max=195ms p(90)=0s p(95)=0s
http_req_duration................: avg=90.59ms min=80.93ms med=92.8ms max=542.92ms p(90)=96.89ms p(95)=102.83ms
{ expected_response:true }.....: avg=90.49ms min=80.93ms med=92.8ms max=542.92ms p(90)=96.86ms p(95)=102.81ms
http_req_failed..................: 0.00% ✓ 012946
http_req_receiving...............: avg=154.41µs min=17µs med=47µs max=97ms p(90)=67µs p(95)=76µs
http_req_sending.................: avg=20.36µs min=0s med=14µs max=32.36ms p(90)=20µs p(95)=21µs
http_req_tls_handshaking.........: avg=1.16ms min=0s med=0s max=183.1ms p(90)=0s p(95)=0s
http_req_waiting.................: avg=90.41ms min=80.85ms med=92.72ms max=542.86ms p(90)=96.77ms p(95)=102.74ms
http_reqs........................: 12960 215.658713/s
iteration_duration...............: avg=93.58ms min=81.05ms med=92.95ms max=1.83s p(90)=97.54ms p(95)=103.37ms
iterations.......................: 12953 215.542231/s
vus..............................: 20 min=20 max=21
vus_max..........................: 21 min=21 max=21

Since it's all in one script, this allows for greater collaboration amongst teams and a unified view of the performance metrics from a browser-level and protocol-level perspective.

Future Plans

We aim for xk6-browser to become part of the k6 core once it reaches its stability goals. We consider browser automation an important part of web application testing, and we have big goals for xk6-browser. Our roadmap details essential status updates and our short, mid, and long-term goals.

Get Involved

We need your help! Since xk6-browser is still relatively new, we need help from the community to try out the tool and give us feedback.

Check our GitHub project, read our documentation, and play with the tool. If you find any issues, please raise them on our GitHub project or check out our community forum for additional support.

< Back to all posts