Digital disruptors and customer-obsessed organizations are improving customer experience (CX) by shortening their software delivery cycles, delivering features in smaller increments, and scaling their existing Agile processes with DevOps. Traditional testing services don’t cut it for these organisations. In fact in my recent Continuous Testing Services Providers, Q3 2017 wave research, 38 business leaders representing the top tier customers of the vendors evaluated, claim their most important testing services requirements to satisfy within an Agile and DevOps context are:

  1. Functional test execution automation (UI, Data-Driven, API)
  2. Performance testing (Left-shifted)
  3. Application Mobile testing (front-end device testing).

In the wave we evaluated how and which vendors are doing best in shifting their services from manual to automation, are scaling their Agile testing services to provide continuous testing, retraining and hiring new talent. Have they done enough ? Not really, and as allways some are better than others and have been working longer at the transformation. The process has been a long one, and after filtering out from almost 50 vendors in my previous continuous testing (CT) services vendor landscape research  I additionally filtered down to 11 vendors that count based on a list of inclusion  criteria based on revenue, types of services, availability of reference customers to survey on continuous testing and finally vendors that sparked client inquiries or services that put the vendor on Forrester’s radar.  So the end list of the vendors I evaluated fall in the mid market range and are: Atos, Cigniti Technologies, EPAM Systems, HCL Technologies, Hexaware Technologies, Larsen & Toubro Infotech (LTI), Mindtree, NIIT Technologies, QualitTest, Syntel and Tech Mahindra.

The wave analyses in depth: overall strategy, vision, the vendors CT roadmap and market share and the services current offering, i.e. the types and the breadth of the CT consulting services each vendor offers, its adaptability to integrate with development teams, and its ability to operate independently in client Agile and DevOps teams. I scrutinized its testing processes and practices; test case design automation; automation engineering; test data management; services virtualization; test environment provisioning; and even more advanced testing-in-production capabilities. I also looked at accelerators, but put the biggest emphasis on CT talent, taking a look at practices, scale, and referenceable work. The result ? All of the vendors are on a journey themselves to transform their services into continuous ones, as they start delivering these to clients. Some are ahead than others, but all have some unique capabilities. Some interesting examples? Some offer interesting approaches in developing their workforce, creating AI services for CX testing, last but not at least production testing capabilities. So, want to know who the leaders, challengers and contenders are ? If you are Forrester client you can check out the full document on the forrester website, download the interactive wave tool and customise the evaluation to your needs, have an inquiry with me to dig out further findings and get any necessary clarification.
Feedback is allways welcome dlogiudice@forrester.com or just leave your comment !.