In the days of old, not very long ago, release cycles were measured in years —organizations were using “on-time” and “on-budget" as the mantra for project efficacy. Business today is compelled to deliver business technology in cycles of hours, or days. Faster cycles render not only tradition “waterfall” processes and silo based IT obsolete, it also renders traditional metrics ineffective! These arcane metrics no longer deliver the visibility and granularity tech pros need to fine-tune their delivery capability. The mission has transitioned to rapidly deliver high quality, high value solutions. For all, this is a significant shift from the past, when the main points of focus were schedule, cost, and efficiency. Modern software metrics — speed, quality, and value — are based on continuous feedback from business partners and customers.

To transition the business, organizations are transitioning to Agile and DevOps practices. Organizations including Capital One[1], Target[2], and KeyBank[3] – to name a few – are leveraging DevOps practices to transition their businesses. Each shared their journey to high velocity at the DevOps Enterprise Summit in San Francisco last November and all remarked on the transitioning of metrics from what were traditional metrics. Diego Lo Giudiceand I share insight into these evolving metrics in our report, “Build The Right Software Better And Faster With Agile And DevOps Metrics.” It addresses the practices and tools that help organizations collect, report, and respond to them. In my research on the subject, I found three key takeaways, listed and briefly described below.

Delivery cycle metrics provide real-time insights into progress and quality

With the introduction of delivery cycle metrics, lapses in quality are no longer excusable. Due to the granular view that these metrics provide, leaders can see exact instances where their delivery cycle needs improvement. The pioneers in this space are doing everything they can to eliminate these deficiencies, to drive greater quality in the future.

Automated testing drives continuous visibility in release readiness

Automated testing is critical to assure that code is fit for purpose and use. Automated testing including static code analysis, functional testing, load and performance testing and user accepting testing. Additionally automated testing can be used to simulate attacks or replay previous intrusion events and validate compliance to to ensure that developers have not made changes that introduced new vulnerabilities or reintroduced old ones. This process makes it easier both to ensure that the release version of a deliverable is error free, as well as to have a granular view of where things may have gone wrong.

Post-production customer experience and user metrics are critical

The granular problem solving doesn’t stop at release, though, and agile DevOps enables you to examine the functionality of a release after it gets installed in the customer’s environment. This late stage situational awareness ensures that future releases are addressing actual problems experienced by actual customers under actual conditions.

Metrics must evolve over time

As you mature in your Agile and DevOps initiatives, your measurements will differ based on the role and responsibility of the reporting level. For instance, CEO and business leaders will want their metrics aligned directly to their business services, while the CIO will be looking for individual productivity measures.

Dashboards are pivotal to success

The typical DevOps will consist of a number of tools across the tool chain, including Continuous IntegrationContinuous Deployment (CICD), testing, staging and release pipeline, typically each with its own reporting features.  To understand the complete picture, organizations need to aggregate and synthesize the information into a single dashboard. An example of one of these dashboards is available from Capital One’s, open source Hygieia project. Hygieia, uses an open plug-in architecture to aggregate data from different application delivery life-cycle tools allowing you to understand their delivery processes from multiple perspectives[4].

If you are investing in DevOps and are a Forrester client, our report;“Build The Right Software Better And Faster With Agile And DevOps Metrics” is available in the Forrester App or you can download it here.

Finally, if you have not completed the Forrester DevOps 2017 please do so!  The survey closes February 13.

As always, I welcome your observations, comments and experiences. You can contact me here and don’t forget to follow me on Twitter (@RobertEStroud).