TL;DR — More Load Impact customers than ever are continuously running load tests. In response, we’ve just shipped our "Performance Thresholds” feature. Now, engineers set Pass/Fail metrics for their scheduled tests in order to hone in on the data that’s most important to them. Here’s how we think that helps you:
In the spirit of Continuous Delivery and automation, there’s a wave of new Load Impact users scheduling load tests at all times of day and night. With an increased number of performance tests in the software development lifecycle, reliable notifications and triggers when things go wrong become extremely important.
That’s why we’re excited to announce our new Performance Thresholds feature.
The idea here is for testers to use their performance goals to set Pass/Fail criteria for their load tests. For example:
Thresholds can be created for VU Load Time, Failure Rate, page metrics and any of your defined custom metrics.
Users have the option to either immediately abort a test when a metric has failed or allow the test to complete. There are really two simple reasons for this.
Use our Notifications feature (which was released in April to rave reviews) to know exactly when your load tests have completed and whether or not they’ve failed.
For Load Impact users who are already using notifications, you’ll see when a test has "failed by threshold” in the status code, which is already included in the data we send in every notification.
As the CD, CI and DevOps methodologies have picked up substantial steam the last few years, more engineers using Load Impact started regularly scheduling load tests. Here’s an example of what a bunch of our great users are doing: