Digital Experience Review Methodology

The Forrester Digital Experience Review is Forrester’s expert evaluation of the digital experiences that firms in a particular industry deliver to consumers. This research evaluates both the functionality integrated into each firm’s digital touchpoint, and the user experience (UX) that the touchpoint provides its users. Our analysis focuses on a digital touchpoint in one industry per report, be it the desktop website, the mobile site, or the mobile app.

Our Digital Experience Review research helps business, marketing, and technology leaders understand the best practices and digital experiences that drive growth. Forrester clients can review these best practices and recommendations for their own industry, but also look across industries to understand how consumer experiences and expectations are changing in other markets. In each report, we provide an overview of our research findings, as well as best practices examples and detailed recommendations for companies seeking to set their own digital experience priorities. Our evaluation enables all companies – participants in the research and non-participants — to make well-informed decisions on what enables exceptional digital interactions.

A Forrester Digital Experience Review builds upon the participation of the following key players:

• Research Leaders. Two research leaders conduct the Forrester DXR research and oversee its conclusions. One leads the review of the digital functionality, the other leads the review of user experience. Through tight coordination, these two research leaders determine the inclusion criteria to select participants, build the persona and scenarios, and establish the criteria for the functionality and user experience scorecards. The two research leaders also oversee the scoring and report writing.
• Project Manager/Researcher. The project manager is the process lead on the Forrester Digital Experience Review, driving and maintaining the project schedule and serving as the main point of contact with participants throughout the process. The project manager also supports the evaluation process, which is conducted by research leaders and researchers.
• Brand response teams. The brand response team comprises the key contacts we work with at the companies we evaluate. In some cases, Forrester relies on the team’s participation to provide access to any non-public portions of the digital experience. The team also aids in verifying information that we collect in our functionality review.

The Forrester DXR process includes:

1. Pre-research planning. The research leaders conduct their primary research for the industry sector. We use our own industry expertise and Forrester’s consumer insights to identify brands for inclusion, pinpoint the most important customer scenarios, identify the digital touchpoints that consumers use, and identify the functionality that helps customers achieve their goals. We target a select number of brands in each industry, and determine unique inclusion criteria on every review. The inclusion or exclusion of a brand in a DXR is exclusively Forrester’s editorial decision. Brands may not be added or removed to the research at their request.
2. Research kickoff. The research leaders and researcher reach out to contacts at the brands we choose to evaluate, to gain participation from brand response teams. This communication provides an overview of the research focus. It also includes the general timeline, methodology, persona, scenarios, and research focus. Finally, it provides an overview of information we may request from participants. If Forrester is unable to find appropriate contacts, we may proceed to the next step without direct involvement from the brand.
3. Expert reviews and user observation. The research leaders, using the identified scenarios and criteria, evaluate each brand’s digital experience. The research leaders evaluate what is currently available to the brand’s prospects and customers. The outcome is a two-part score for each brand — one score each for functionality and user experience. We use these scores to produce supporting scorecards, which brands see during Courtesy Preview, but are not published. Our approach to the UX review varies by the user scenarios and industry sectors we’re evaluating. It may include a heuristic review, usability testing, or both.
4. Fact-checking. The Researcher shares individual evaluation scorecards with each brand we evaluate, if we have been able to reach contacts at those brands. Brands can provide factual feedback and supporting evidence to address any factual errors regarding the brand’s functionality during the fact-checking time period. User experience findings are not subject to fact-checking.
5. Courtesy preview and publishing of the report. Once the team has developed and reviewed all research deliverables, it’s time to share the research with participants. The team sends out a courtesy preview report to all firms we evaluate, including a separate attachment of their place ranking among competitors, usually two days prior to publication of the report on Forrester’s website. This is a courtesy preview period only, and we consider the content of the Forrester DXR report final.

A company participating in a Forrester Digital Experience Review may request an escalation after attempting to resolve a disagreement with the analyst conducting the review. Escalations can happen during the fact-check and final scorecard periods only — not during the courtesy preview period just prior to publication. The initial participants in an escalation are the analyst conducting the research and the research director (RD). If disagreement remains after the initial escalation, a company may request an escalation call with the group director (GD). The GD can decide whether to take the escalation call. The company may not object to scores of other companies, scope, weightings, or scale explanations during the escalation. Forrester reserves the right to decline any escalation based on claims that conflict with the methodology.