There are always leaders and laggards in comparing vendor technology products, but being among the top 15 players in the market in a Forrester Wave™ evaluation is always a win. Why? Because the vendors are being compared with a peer group of test automation vendors that form, based on Forrester’s research, the top 15 performers among all.

The biggest takeaway from our 2020 evaluation can be summarized by the following statements:

  • Seventy-two percent of the 59 reference clients participating in this Wave research opted for an integrated test automation suite over best-of-breed test automation tools.
  • Customer references want a smart, consistent, and integrated test experience to help support end-to-end testing across different architectures, technologies, and even different types of automation (e.g., robotic process automation [RPA], digital process automation [DPA], low code, or AI-infused applications).

In fact, in “The Forrester Wave™: Continuous Functional Test Automation [CFTA] Suites, Q2 2020,” smart, integrated suites of testing tools outperform the less effective, siloed, best-of-breed ones. On top of the typical integration and maintenance functions, capabilities such as AI and machine learning infused in test automation design and execution, combined heterogenous channels, and API testing dictate which providers will lead the pack. Vendors that can provide a smart, integrated suite with seamless access to an extended set of capabilities position themselves to successfully deliver faster, lower-cost, and smarter testing to their customers.

Who participated in our CFTA Wave? We included ACCELQ, Broadcom, Cyara, Eggplant, Experitest, IBM, mabl, Micro Focus, Parasoft, Perforce Software, Ranorex, Sauce Labs, SmartBear Software, Tricentis, and Worksoft — and researched, analyzed, and evaluated them against 26 criteria. They all have their cool capabilities, but four vendors excel. “The Forrester Wave™: Continuous Functional Test Automation Suites, Q2 2020” report shows how each provider measures up and helps application development and testing professionals select the right provider for their needs. In the list, you can see the usual suspects, some that are returning with reenergized offerings, and a few that are new strong entrants. Some consolidation is happening, with many acquisitions in the past year; for example, just during the writing of this Wave report, Experitest was acquired by Digital.ai. Unfortunately, this acquisition happened too late in the Wave process to fully consider the impact.

Are you curious to know who the Leaders, Strong Performers, Contenders, and Challengers in CFTA landscape? You’ll find the answer here. And in case you wonder why these 15 players were picked to be included in this Wave, it’s because we filtered them through a previous research step out of 31 vendors that participated in our “Now Tech: Continuous Functional Test Automation Suites, Q1 2020,” a market landscape study on CFTA that we published earlier in the year.

The Wave and other recent research we’ve published, such as “RPA And Test Automation Are More Friends Than Foes,” as well as research on testing of AI-infused applications and testing in low-code platforms, all show how testing has to keep evolving and innovating to keep up with clients’ growing challenges. Testing is catching up with modern development with Agile and DevOps; growing adoption of automation with RPA, DPA, low code, and AI-infused apps; and examining smart business applications with embedded bots. All these need to be tested to succeed. Watch out for more research coming, and if you have any questions on the above, reach out for an inquiry or send me an email at dlogiudice@forrester.com.