February 19, 2013
Early this year, on January 15, I published our first research on testing for the Agile and Lean playbook. Connected to that research, my colleague Margo Visitacion and I also published a self-assessment testing toolkit. The toolkit helps app-dev and testing leaders understand how mature their current testing practices and organization are for Agile and Lean development.
The Agile Testing Self-Assessment Toolkit
So what are the necessary elements to assess Agile testing maturity? Looking to compromise between simplicity and comprehensiveness, we focused on the following:
- Testing team behavior. Some of the questions we ask here look at collaboration around testing among all roles in the Scrum teams. We also ask about unit testing: Is it a mandatory task for developers? Are all of the repeititive tests that can be run over and over at each regression testing automated?
- Organization. In our earlier Agile testing research, we noticed a change in the way testing gets organized when Agile is being adopted. So here we look at the role test managers are playing: Are they focusing more on being coaches and change agents to accelerate adoption of the new Agile testing practices? Or are managers still operating in a command-and-control regime? Is the number of manual testers decreasing? Are testing centers of excellence (TCOEs) shifting to become testing practice centers of excellence (TPCOEs)?
- Agile testing practices. In this assessment category, we ask questions about testers being embedded in project teams to coordinate testing activities and help increase levels of test automation. Are development teams testing early on and continuously in sprints? Are manual testers focusing more and more on selective testing approaches like exploratory testing? Are practices like unit testing a must? Are TDD and pair code review practices being suggested or enforced in project teams?
- Automation. Automation is one key area for Agile testing, as it enables testing to really happen during short release cycles — which would otherwise be impossible. So here we’re looking for automation of functional and nonfunctional regression tests and if the coverage of tests that can be automated actually are automated. We also question if teams have improved their code and test design practices, which we know increases automation levels. Do developers own automation? Is the build process automated, and is test automation integrated in a continuous integration process?
- Agile maturity. You can’t have Agile testing unless the organization is broadly transforming to Agile. The point of this assessment category is to understand how testers are integrated in the overall change process to Agile: Are testers familiar with the Agile manifesto and have they bought into the values and principles of it? Have they been trained in Agile practices? What level of Agile maturity do teams have?
The best approach to leverage the Agile testing self assessment maturity toolkit is by first reading our report, Consistent Performance In Agile Teams Must Include Testing. You will also find the assessment toolkit under the Resources tab of this report.
But remember: This is only a starting point for your own assessment of what needs to change or improve to become more Agile in your testing practices. The toolkit is customizable and extensible and offers the option of comparing multiple internal teams against one or more target maturity levels. If you are a Forrester client and subscribe to our inquiry service, give it a shot: Run the self-assessment, send me the results, and then schedule an inquiry. I’ll be happy to discuss your results with you!