How to do 10 things in Jmeter vs k6

We recently released k6, the new load testing tool written in Go and scriptable in Javascript. It has seen a phenomenal reception (over 3,000 stars on Github already!) but there isn’t yet much about k6 on Stackoverflow (please do ask some questions there, if you have any!)

Obviously, some things that can be done with Jmeter cannot be done with k6, and vice versa. So I have focused mainly on questions that make k6 look good, uh, err, I mean that I have focused on user scenario creation mainly. In my developer-centric view user-scenario creation in k6 is waaay nicer than in Jmeter, of course :)

And, to be clear, the Jmeter answers have not been tested. They have been taken from SO threads so they may contain errors.

So, well, here are all the questions:

1. How to save an HTTP response in a variable
2. How to test an expected 404 response
3. How to reuse custom test code in different tests
4. How to implement nested loop
5. How to extract first element from JSON array
6. How to get random row from a file
7. How to get SLA metric
8. How to generate sha512 hash
9. How to do parallel requests
10. How to change font size :)

Let's look at the questions and responses in detail.

1. How to save an HTTP response in a variable

...in Jmeter

1. Add a Beanshell Postprocessor as a child of the request which returns the response you’re looking for
2. Put the following code into the PostProcessor’s "Script” area: vars.put("response”, new String(data));
3. Refer extracted value as ${'{response}'} where required

...in k6

Use the following script code:

 let response = http.get("http://some.url/”);
 check(res, {
     "Status is 404": (r) => r.status === 404
 });

Notes:

In both cases, the data ends up in the responsevariable. Main difference is that you have to add a Beanshell PostProcessor to the request in Jmeter, before you can add the code snippet.

https://stackoverflow.com/questions/34463412/how-to-save-response-in-a-variable-in-jmeter

2. How to test an expected 404 response

...in Jmeter

Create a new Response Assertion under the test. In the "Response Field to Test" section of the assertion, make sure to check the box for "Ignore Status".

You can then add other assertions as you'd like, such as setting the radio in "Response Field to Test" to "Response Code" and setting the "Patterns to Test" to 404.

[referral to another SO article - https://jmeter.512774.n5.nabble.com/Making-HTTP-404-a-test-success-tp5713923p5713941.html]

...in k6

Use the following script code:

let response = http.get("http://some.url/”);
check(res, {
"Status is 404": (r) => r.status === 404
});

Notes:

The main difference here is the fact that doing this in Jmeter requires you to click through a GUI and fill in values in entry fields, while with k6 you write a couple of lines of code instead.

https://stackoverflow.com/questions/31317077/how-to-test-an-expected-404-response-with-jmeter

3. How to reuse custom test code in different tests

The question concerned being able to call a logTransaction() function defined in one file, from 150 different test configurations

...in Jmeter

1. Add the next line to user.properties file (located in the "bin" folder of your JMeter installation) beanshell.sampler.init=BeanShellSampler.bshrc
2. Put your logTransactionfunction in the BeanShellSampler.bshrc file (same location, JMeter's "bin" folder)
3. Next time you start JMeter you will be able to call the function from any Beanshell Sampler in any script

...in k6

1. Put your logTransaction() function in a file - e.g. "logTransaction.js”
2. Use the following statement to import the function in any script: import {'{ logTransaction }'} from "/path/to/logTransaction.js";

Notes:

In k6, any Javascript file can be used directly as an importable module, which allows you to organize files any way you want. You can also import modules directly over the network: import {'{ logTransaction }'} from "s3.amazonaws.com/path/to/logTransaction.js";

https://stackoverflow.com/questions/46470481/jmeter-how-to-reuse-custom-java-function-between-different-jmx-file

4. How to implement nested loop

The question was:

"I am thinking about how to test a servlet with two parameters: X and Y, using JMeter.X and Y are random numbers from 0 to 100.I am thinking of implement a nested loop which is something like":

for (int x = 0; x <= 100; x++)
     for (int y = 0; y <= 100; y++)
           servlet?param1=x&param2=y

...in Jmeter

Your schema may be like the following below:

• Thread Group
  • User Defined Variables
    • maxX = 100 maxY = 100
    • Loop Controller X
      • Loop Count: ${__BeanShell(Integer.parseInt(vars.get("maxX"))+1)}
      • Counter X
        • Start: 0
        • Increment: 1
        • Maximum: ${maxX}
        • Reference Name: loopX
      • Loop Controller Y
        • Loop Count: ${__BeanShell(Integer.parseInt(vars.get("maxY"))+1)}
        • Counter Y
          • Start: 0
          • Increment: 1
          • Maximum: ${maxY}
          • Reference Name: loopY
  • YOUR HTTP Request servlet?param1=${loopX}&param2=${loopY} . . .

...in k6

Use the following script code:

 for (var x = 0; x <= 100; x++)
     for (var y = 0; y <= 100; y++)
         http.get("http://some.domain/servlet?param1=” + x + "&param2=” + y);

Notes:

Doing more complex logical branching is of course where a GUI-based approach often becomes quite unwieldy compared to writing the logic in plain code. The k6 solution is very similar to the pseudo-code used in the question to describe the problem.

https://stackoverflow.com/questions/12993754/how-to-implement-nested-loop-in-jmeter

5. How to extract first element from JSON array

The question was how to extract the "srcId” value from the first element of a JSON array looking something like this:

[
   { "some-key": "some-value", "srcid": "3526" }, <--- we want this srcId value: 3526
   { "some-key": "some-other-value", "srcId": "3537" }
]

...in Jmeter

Please follow the below steps to retrieve srcId.

1. Add a JSON Path Extractor to your request and configure below values.Destination Variable Name - myVarJSON Path Expression - \$..cohortDefinition.srcId - this will extract all the srcIDs from the JSON.Default Value - Not Found or Err
   
   
2. Add a Debug Sampler and View Results Tree to your test plan.
3. Save it and execute.
4. In Debug Sampler, you can view all the srcId as shown below.
   
   

...in k6

Use the following script code to get "3526” stored in the myVar variable:

 let response = http.get("http://some.url/”);
 let decoded = JSON.parse(response.body);
 let myVar = decoded[0]["srcId"];

Notes:

https://stackoverflow.com/questions/43901398/jmeter-how-to-extract-first-element-from-json-array

6. How to get a random row from a file

...in Jmeter

You can do it with the Beanshell sampler using the following code:

import org.apache.commons.io.FileUtils; //necessary import
List lines = FileUtils.readLines(new File("/path/to/your/file")); // read file into lines array
int random = new Random().nextInt(lines.size()); // get random line number
String randomLine = lines.get(random); // get random line
vars.put("randomLine", randomLine); // store the random line into ${randomLine} variable
vars.put("response”, new String(data));

You will be able to access the random line as {'${randomLine}'} where required

...in k6

Use the following script code:

let lines = open("/path/to/your/file”).split("\n”);
let rowno = Math.floor(Math.random() * lines.length);
let randomLine = lines[rowno];

Notes:

In both cases, the data ends up in the randomLine variable. Main differences is that with Jmeter you have to configure a Beanshell sampler connected to the request, and use somewhat more code.

https://stackoverflow.com/questions/34463412/how-to-save-response-in-a-variable-in-jmeter

7. How to get SLA metric

The question was how to get Jmeter to report the number of requests that were completed within 400 ms, which was a threshold set by an SLA.

...in Jmeter

1. Simplest solution is to use Simple Data Writer to save Label, Elapsed Time and / or Latency to a CSV file, which will generate raw output like this:
   
   

   elapsed,label
   423,sampler1
   452,sampler2
   958,sampler1
   152,sampler1

   Copy

   And from here you can take it to any other tool (awk, Excel, etc.) to filter results you want.

2. Another option is to use BeanShell Listener to generate such report on the fly. Something like this:
   
   

   long responseTime = sampleResult.getEndTime() - sampleResult.getStartTime();
   if (responseTime < 400) {
       FileOutputStream f = new FileOutputStream("myreport.csv", true);
       PrintStream p = new PrintStream(f);
       this.interpreter.setOut(p);
       print(sampleResult.getSampleLabel() + "," + responseTime);
       f.close();
   }

   Copy

   This method, though, may not be performant enough if you are planning to run a stress test with many (more than 200-300) users and many operations that "fit" the filter.

...in k6

Use the following script code to make k6 print the total # of requests that met the SLA:

import http from 'k6/http';
import { Counter } from 'k6/metrics';
export const SLAcounter = new Counter('responseTimeOK');
export default function () {
  const res = http.get('http://example.com/'); // change to URL you want to test
  if (res.timings.duration < 400) {
    SLAcounter.add(1);
  }
}

Notes:

With Jmeter, you have to add a Beanshell listener and provide some code to do what you want, but it may not be performant, or you have to save data as CSV and post-process it with external tools. With k6, the whole thing is performant and 100% configured in code. Note also that the above k6 code example is the complete configuration for the k6 test - it can be saved as e.g. script.js and run using k6 run script.js.

https://stackoverflow.com/questions/38344484/jmeter-how-to-get-sla-metric

8. How to generate sha512 hash

The question concerned making a sha512 hash of a combination of a "salt” string and a password.

...in Jmeter

Add JSR223 Sampler with Java language, pass variables password and salt using vars.get("password") and using this code variable generatedPassword will hold the new hash generated

 import java.security.MessageDigest;
 import java.security.NoSuchAlgorithmException;

 String passwordToHash = vars.get("password");
 String salt= vars.get("salt");
 String generatedPassword = null;
 try {
    MessageDigest md = MessageDigest.getInstance("SHA-512");
    md.update(salt.getBytes("UTF-8"));
    byte[] bytes = md.digest(passwordToHash.getBytes("UTF-8"));
    StringBuilder sb = new StringBuilder();
    for(int i=0; i< bytes.length ;i++){
       sb.append(Integer.toString((bytes[i] & 0xff) + 0x100, 16).substring(1));
    }
    generatedPassword = sb.toString();
    log.info(generatedPassword);
    vars.put("generatedPassword", generatedPassword);
 }
 catch (NoSuchAlgorithmException e){
    e.printStackTrace();
 }

...in k6

Use the following script code:

import crypto from 'k6/crypto';

export default function () {
  const hasher = crypto.createHash('sha512');
  hasher.update(salt);
  hasher.update(password);
  generatedPassword = hasher.digest('base64');
}

Notes:

Like always, with Jmeter you need to add something in the configuration (a JSR223 sampler) before you can start writing any code. In both cases, we need to import the hashing functionality from an external module. The "k6/crypto" module is always available, so no need to check if the import worked or not, with k6. Even discounting this error checking though, the k6 code is shorter, and the k6 API is (IMO) simpler to use.

https://stackoverflow.com/questions/46462210/jmeter-how-to-generate-hash-sha512

9. How to do parallel requests

The question concerns being able to issue more than one request in parallel, like browsers do when fetching web pages and which puts a lot more stress on the server than having each virtual user just issue one request at a time.

...in Jmeter

JMeter provides Synchronizing Timer which allows grouping requests so they could be executed at exactly in the same moment. Just add a Synchronizing Timer to your test plan and make sure that

• it is at the same level with both requests

• number of virtual users in Thread Groupis >= what is set in the Synchronizing Timer
  
  

  

  
  

  See Using the JMeter Synchronizing Timerfor more information on running specific requests at the same time in your JMeter test.

...in k6

Use the http.batch() function to issue multiple requests in parallel:

http.batch([
  'http://test.loadimpact.com',
  'http://test.loadimpact.com/style.css',
  'http://test.loadimpact.com/images/logo.png',
]);

Notes:

The above "solutions" are not 100% comparable. In the Jmeter case, we will still only have each single VU issue requests sequentially. This means that thinking of a VU as "a user" becomes impossible; a single human user will issue multiple requests concurrently, using their web browser, while a Jmeter VU only issues one request at a time. What Jmeter instead does here is synchronizing the VUs so they issue requests at exactly the same time (which to me seems less useful, to be honest - the total throughput will probably go down, given that some VUs have to wait for other VUs to complete requests). k6, on the other hand, will in the above case let each VU open three concurrent TCP connections, and so be able to fetch the three items in parallel, just like a web browser does. This means that in the k6 case, 100 VU may fetch 300 items in parallel. In the Jmeter case 100 VU will fetch 100 items in parallel (and if I am correct, the VUs will spend some time waiting also, instead of issuing requests).

https://stackoverflow.com/questions/33942516/jmeter-how-to-do-parallel-request

10. How to change the font size!

This article got a bit boring, so I decided to throw in a silly question also.

...for Jmeter

1. In order to get src folder you need to download JMeter source code separately, i.e. from here. Unpack the folder and make required changes
2. You will need Apache Ant to build JMeter. Install it and make sure that %ANT_HOME%/bin folder is added to your system PATH variable
3. Build JMeter using next 2 commands:

• ant download_jars
• ant package

Now your should be able to run JMeter with increased font from apache-jmeter-2.13/bin folder like: jmeter -Djmeter.dialog.font=20 or if you want to make font size change permanent add the next line to system.properties file (located in JMeter's "bin" folder) jmeter.dialog.font=20 Meter restart will be required to pick the property up. See Apache JMeter Properties Customization Guide for more information regarding JMeter properties usage

...for k6

• On a Mac, press CMD/+ to increase font size, and CMD/- to decrease it

Notes:

While it can be a hard to avoid accidentally pressing the ALT key when you're aiming for the CMD key on a Mac keyboard, I still maintain that changing the font size in k6 is simpler.

https://stackoverflow.com/questions/36899553/how-to-change-font-size-for-jmeter

Want to know how to do X in k6?

Check out these resources:

• The beautiful https://k6.io documentation site, with tons of info
• The examples directory has lots of useful code snippets there showing how to do things
• Join our k6.io Slack channel and chat with other k6 users and developers
• Ask a k6 question on Stackoverflow (tag it with "k6"!)