We recently released k6, the new load testing tool written in Go and scriptable in Javascript. It has seen a phenomenal reception (over 3,000 stars on Github already!) but there isn’t yet much about k6 on Stackoverflow (please do ask some questions there, if you have any!)
Jmeter, on the other hand, is very well represented on SO. There are tons of questions and answers there about how to do things in Jmeter. We thought: why not respond to some of the SO Jmeter questions, but write about how to do the same thing in k6? And better yet, why not compare how something is done in k6 with how it is done in Jmeter? Based on real Stackoverflow user questions!Obviously, some things that can be done with Jmeter cannot be done with k6, and vice versa. So I have focused mainly on questions that make k6 look good, uh, err, I mean that I have focused on user scenario creation mainly. In my developer-centric view user-scenario creation in k6 is waaay nicer than in Jmeter, of course :)
And, to be clear, the Jmeter answers have not been tested. They have been taken from SO threads so they may contain errors.
So, well, here are all the questions:
- How to save an HTTP response in a variable
- How to test an expected 404 response
- How to reuse custom test code in different tests
- How to implement nested loop
- How to extract first element from JSON array
- How to get random row from a file
- How to get SLA metric
- How to generate sha512 hash
- How to do parallel requests
- How to change font size :)
Let's look at the questions and responses in detail.
1. How to save an HTTP response in a variable
...in Jmeter
- Add a Beanshell Postprocessor as a child of the request which returns the response you’re looking for
- Put the following code into the PostProcessor’s "Script” area: vars.put("response”, new String(data));
- Refer extracted value as ${'{response}'} where required
...in k6
Use the following script code:
Notes:
In both cases, the data ends up in the responsevariable. Main difference is that you have to add a Beanshell PostProcessor to the request in Jmeter, before you can add the code snippet.
https://stackoverflow.com/questions/34463412/how-to-save-response-in-a-variable-in-jmeter
2. How to test an expected 404 response
...in Jmeter
Create a new Response Assertion under the test. In the "Response Field to Test" section of the assertion, make sure to check the box for "Ignore Status".
You can then add other assertions as you'd like, such as setting the radio in "Response Field to Test" to "Response Code" and setting the "Patterns to Test" to 404.
[referral to another SO article - https://jmeter.512774.n5.nabble.com/Making-HTTP-404-a-test-success-tp5713923p5713941.html]
...in k6
Use the following script code:
Notes:
The main difference here is the fact that doing this in Jmeter requires you to click through a GUI and fill in values in entry fields, while with k6 you write a couple of lines of code instead.
https://stackoverflow.com/questions/31317077/how-to-test-an-expected-404-response-with-jmeter
3. How to reuse custom test code in different tests
The question concerned being able to call a logTransaction() function defined in one file, from 150 different test configurations
...in Jmeter
- Add the next line to user.properties file (located in the "bin" folder of your JMeter installation) beanshell.sampler.init=BeanShellSampler.bshrc
- Put your logTransactionfunction in the BeanShellSampler.bshrc file (same location, JMeter's "bin" folder)
- Next time you start JMeter you will be able to call the function from any Beanshell Sampler in any script
...in k6
- Put your logTransaction() function in a file - e.g. "logTransaction.js”
- Use the following statement to import the function in any script: import {'{ logTransaction }'} from "/path/to/logTransaction.js";
Notes:
In k6, any Javascript file can be used directly as an importable module, which allows you to organize files any way you want. You can also import modules directly over the network: import {'{ logTransaction }'} from "s3.amazonaws.com/path/to/logTransaction.js";
https://stackoverflow.com/questions/46470481/jmeter-how-to-reuse-custom-java-function-between-different-jmx-file
4. How to implement nested loop
The question was:
"I am thinking about how to test a servlet with two parameters: X and Y, using JMeter.X and Y are random numbers from 0 to 100.I am thinking of implement a nested loop which is something like":
...in Jmeter
Your schema may be like the following below:
- Thread Group
- User Defined Variables
- maxX = 100 maxY = 100
- Loop Controller X
- Loop Count: ${__BeanShell(Integer.parseInt(vars.get("maxX"))+1)}
- Counter X
- Start: 0
- Increment: 1
- Maximum: ${maxX}
- Reference Name: loopX
- Loop Controller Y
- Loop Count: ${__BeanShell(Integer.parseInt(vars.get("maxY"))+1)}
- Counter Y
- Start: 0
- Increment: 1
- Maximum: ${maxY}
- Reference Name: loopY
- YOUR HTTP Request servlet?param1=${loopX}&param2=${loopY} . . .
- User Defined Variables
...in k6
Use the following script code:
Notes:
Doing more complex logical branching is of course where a GUI-based approach often becomes quite unwieldy compared to writing the logic in plain code. The k6 solution is very similar to the pseudo-code used in the question to describe the problem.
https://stackoverflow.com/questions/12993754/how-to-implement-nested-loop-in-jmeter
5. How to extract first element from JSON array
The question was how to extract the "srcId” value from the first element of a JSON array looking something like this:
...in Jmeter
Please follow the below steps to retrieve srcId.
- Add a JSON Path Extractor to your request and configure below values.Destination Variable Name - myVarJSON Path Expression - \$..cohortDefinition.srcId - this will extract all the srcIDs from the JSON.Default Value - Not Found or Err
- Add a Debug Sampler and View Results Tree to your test plan.
- Save it and execute.
- In Debug Sampler, you can view all the srcId as shown below.
...in k6
Use the following script code to get "3526” stored in the myVar variable:
Notes:
https://stackoverflow.com/questions/43901398/jmeter-how-to-extract-first-element-from-json-array
6. How to get a random row from a file
...in Jmeter
You can do it with the Beanshell sampler using the following code:
You will be able to access the random line as {'${randomLine}'} where required
...in k6
Use the following script code:
Notes:
In both cases, the data ends up in the randomLine variable. Main differences is that with Jmeter you have to configure a Beanshell sampler connected to the request, and use somewhat more code.
https://stackoverflow.com/questions/34463412/how-to-save-response-in-a-variable-in-jmeter
7. How to get SLA metric
The question was how to get Jmeter to report the number of requests that were completed within 400 ms, which was a threshold set by an SLA.
...in Jmeter
Simplest solution is to use Simple Data Writer to save Label, Elapsed Time and / or Latency to a CSV file, which will generate raw output like this:
And from here you can take it to any other tool (awk, Excel, etc.) to filter results you want.
Another option is to use BeanShell Listener to generate such report on the fly. Something like this:
This method, though, may not be performant enough if you are planning to run a stress test with many (more than 200-300) users and many operations that "fit" the filter.
...in k6
Use the following script code to make k6 print the total # of requests that met the SLA:
Notes:
With Jmeter, you have to add a Beanshell listener and provide some code to do what you want, but it may not be performant, or you have to save data as CSV and post-process it with external tools. With k6, the whole thing is performant and 100% configured in code. Note also that the above k6 code example is the complete configuration for the k6 test - it can be saved as e.g. script.js and run using k6 run script.js.
https://stackoverflow.com/questions/38344484/jmeter-how-to-get-sla-metric
8. How to generate sha512 hash
The question concerned making a sha512 hash of a combination of a "salt” string and a password.
...in Jmeter
Add JSR223 Sampler with Java language, pass variables password and salt using vars.get("password") and using this code variable generatedPassword will hold the new hash generated
...in k6
Use the following script code:
Notes:
Like always, with Jmeter you need to add something in the configuration (a JSR223 sampler) before you can start writing any code. In both cases, we need to import the hashing functionality from an external module. The "k6/crypto" module is always available, so no need to check if the import worked or not, with k6. Even discounting this error checking though, the k6 code is shorter, and the k6 API is (IMO) simpler to use.
https://stackoverflow.com/questions/46462210/jmeter-how-to-generate-hash-sha512
9. How to do parallel requests
The question concerns being able to issue more than one request in parallel, like browsers do when fetching web pages and which puts a lot more stress on the server than having each virtual user just issue one request at a time.
...in Jmeter
JMeter provides Synchronizing Timer which allows grouping requests so they could be executed at exactly in the same moment. Just add a Synchronizing Timer to your test plan and make sure that
it is at the same level with both requests
number of virtual users in Thread Groupis >= what is set in the Synchronizing Timer
See Using the JMeter Synchronizing Timerfor more information on running specific requests at the same time in your JMeter test.
...in k6
Use the http.batch() function to issue multiple requests in parallel:
Notes:
The above "solutions" are not 100% comparable. In the Jmeter case, we will still only have each single VU issue requests sequentially. This means that thinking of a VU as "a user" becomes impossible; a single human user will issue multiple requests concurrently, using their web browser, while a Jmeter VU only issues one request at a time. What Jmeter instead does here is synchronizing the VUs so they issue requests at exactly the same time (which to me seems less useful, to be honest - the total throughput will probably go down, given that some VUs have to wait for other VUs to complete requests). k6, on the other hand, will in the above case let each VU open three concurrent TCP connections, and so be able to fetch the three items in parallel, just like a web browser does. This means that in the k6 case, 100 VU may fetch 300 items in parallel. In the Jmeter case 100 VU will fetch 100 items in parallel (and if I am correct, the VUs will spend some time waiting also, instead of issuing requests).
https://stackoverflow.com/questions/33942516/jmeter-how-to-do-parallel-request
10. How to change the font size!
This article got a bit boring, so I decided to throw in a silly question also.
...for Jmeter
- In order to get src folder you need to download JMeter source code separately, i.e. from here. Unpack the folder and make required changes
- You will need Apache Ant to build JMeter. Install it and make sure that %ANT_HOME%/bin folder is added to your system PATH variable
- Build JMeter using next 2 commands:
- ant download_jars
- ant package
Now your should be able to run JMeter with increased font from apache-jmeter-2.13/bin folder like: jmeter -Djmeter.dialog.font=20 or if you want to make font size change permanent add the next line to system.properties file (located in JMeter's "bin" folder) jmeter.dialog.font=20 Meter restart will be required to pick the property up. See Apache JMeter Properties Customization Guide for more information regarding JMeter properties usage
...for k6
- On a Mac, press CMD/+ to increase font size, and CMD/- to decrease it
Notes:
While it can be a hard to avoid accidentally pressing the ALT key when you're aiming for the CMD key on a Mac keyboard, I still maintain that changing the font size in k6 is simpler.
https://stackoverflow.com/questions/36899553/how-to-change-font-size-for-jmeter
Want to know how to do X in k6?
Check out these resources:
- The beautiful https://k6.io documentation site, with tons of info
- The examples directory has lots of useful code snippets there showing how to do things
- Join our k6.io Slack channel and chat with other k6 users and developers
- Ask a k6 question on Stackoverflow (tag it with "k6"!)