When you execute a load test, your System Under Test (SUT) may often become over saturated and start responding with errors. In this case, you need to consider what the iteration execution should do:
- to embrace the system error and continue the execution of the scenario
- or to exit
It's not uncommon for performance testers to forget about these cases. We often write fragile test code that assumes our system's response will always succeed and contain the expected data.
This code will work fine when the SUT returns correct responses. But, when the SUT starts to fail, r.json().length will throw an exception:
This might not be ideal because:
- system errors are propagated as exceptions that are not reported on the test results
- you might want to embrace these errors - as normal - and continue the execution
It's possible to rewrite this test to be less fragile, but it can make our test code longer and less readable.
Sometimes it's hard to predict how a SUT might fail. For those cases, describe catches any internal exceptions and:
- records them as failed assertions
- continues the execution (outside of its describe() function)