11 July 2015

Remain CALMS in Your DevOps Approach

Peter Cannell

Defining what DevOps is and what it means to different organizations has been in flux for several years. The vendor community has pushed the methodology's innovation even further by jumping on the DevOps product/marketing bandwagon.

DevOps can be traced back to the 1980s when ITIL (Information Technology Infrastructure Library) became popular. Because many of our readers were either too busy jamming to Flock of Seagulls or weren't born yet, so we will catch you up. ITIL is a set of practices designed to align IT services with the needs of a business, often credited to W. Edwards Deming.

On first blush ITIL sounds very much like DevOps. It is, however, much more focused on processes, procedures and copious documentation. DevOps is more about rapid change, testing, measuring and changing again. ITIL relies more on planning, documentation and strategy.

So if DevOps is a more modern iteration of ITIL in today's web focused shops what does CALMS stand for?

In 2010 Damon Edwards (@damonedwards) coined the acronym CAMS, which stands for Culture, Automation, Measurement and Sharing. Jez Humble (@jezhumble) added Lean to the mix, giving us CALMS.

Then, our friends at Rackspace authored a great blog post about CALMS, and we've decided to take a crack at it ourselves. Here are the aspects we find important and use to further optimize our performance and load testing solution.


This plays a huge role in the success of technology companies and their ability to quickly turn out high quality applications. A culture that embraces all the facets of DevOps shouldn't be limited to only the developers and application owners. The culture shift must come from the top down.


Bringing a high level of automation to development cycles will provide enormous benefits. This is a big part of what Load Impact offers developers. Here are the three different mechanisms to automate testing using our software:

  • Plugins for TeamCity and Jenkins: These downloadable plugins are available at no extra cost and allow for a fully automated pass/fail testing framework. Every time new code is deployed it can easily be tested against pre-determined pass/fail criteria such as requests per second, bandwidth, user load time, etc. Being able to programmatically determine when new code is causing performance issues is what automated testing is all about.
  • SDKs: We offer both a Python and Java SDK to tie into the Load Impact testing platform. Most developers will be familiar with either Python or Java allowing for rapid integration at the API level. Being able to configure, control and run performance tests from virtually any continuous integration platform will ensure testing is run early, often and in an automated fashion.
  • Platform API: We also offer a fully documented platform API in addition to the packaged SDK's above. Being able to not only control and run tests using the API but also pull results and integrate those data into custom systems is powerful — and our customers have been quick to tell us all about it!


Closely coupled with Automation, lean testing cycles can remove barriers to frequent, effective testing. From a performance perspective, you want higher frequency testing with less overhead and friction to configure, run and interpret results. This is a core tenant of DevOps and something we are highly focused on with respect to performance.


Of course, what a SaaS measures is important to the user. That's why we give developers a deep look into their application's performance — far beyond just user load time. But there's something else we've noticed over the last six years.

Really great technologies do reporting and dashboards correctly.

The user interface and user experience is something that some dismiss as fluff or not core functionality, but we have witnessed first hand how important it is. That's why we revamped our frontend using Ember.js, and the facelift has already resulted in even more happy customers!

Generating reports and metrics that can be consumed by non-technical members of the team — especially C-level execs —is powerful not only for the teams using the product but also for the vendor responsible for the tools being used.

Why? If the C-level exec understands the importance and power that testing provides, then budget and culture issues can melt away. Having developers that are able to relate valuable and relevant data to the business is a core benefit of DevOps culture.


Collaboration between developers and operations is core to a DevOps culture. Being able to share what works, what's causing problems and specifics of overall performance is key. Our testing results are easily accessible over the web and can be integrated into daily or weekly progress reports that can be disseminated internally.

Additionally, we have built in social media sharing capabilities which many will appreciate. Whether it's sharing among industry peers or just using the convenience of social media for internal dialogs it's easy to do. On the test summary page just click "share these results":

screenshoot001 shared results tweet

Sharing via Social Media made easy

DevOps will continue to evolve rapidly as organizations adopt, leverage and internalize the concepts embodied by the CALMS approach. DevOps, just like ITIL, is not something that is achieved and checked off a list. It's a process, a journey that organizations will move through at varying speeds and levels of adoption.

A DevOps approach will reward organizations with the ability to change quickly with a nimble response to business inputs. Being able to explain how DevOps is rooted in ITIL may very well prove valuable with executives with development experience and ITIL.

It is certainly here to stay and built upon a solid foundation.

— Here's a wink of the Load Impact eye to Rackspace for inspiring this post, and check us out on the Rackspace marketplace.

< Back to all posts