Facebook made an announcement yesterday titled “Making Ad Metrics Clearer” that says, in part:
We’ve heard that businesses want more insight into our measurement tools and metrics. So today, we’re introducing new labels on some of our metrics to clearly show how they are calculated . . .
The centerpiece of its announcement is new labels identifying metrics as “estimated” if they are based on a sampled data set and “in development” if the methodology is new and it anticipates further evolution. It also announced an education initiative called “Measure What Matters.”
This announcement fails to answer the questions that advertisers have on three fronts:
- No independent testing. Facebook’s testing process appears to be managed completely internally, rather than partnering with objective, recognized research firms such as Kantar Millward Brown or Nielsen. The majority of adtech platforms such as DataXu, MediaMath, and Viant have partnerships that enable their customers to run brand lift tests directly with independent research firms.
- No transparency into its methodology. It doesn’t describe its sampling methodology and how it ensures that the sample is representative of the segment it is trying to measure. Additionally, the “estimated” label is a headscratcher: Surveys based on sampling and executed according to best practices are well-established and accepted as valid, so it is mystifying why Facebook calls this “estimated.”
- No collaboration with other industry stakeholders. It appears to be flying solo with its measurement education rather than aligning with work that the IAB, ANA, ARF, and other industry organizations have been doing for years. Instead of going it alone, it should follow the example of Neustar, which partnered with the ANA to launch the Analytics Center of Excellence to raise the marketing industry’s skills in using data.
If Facebook truly wants to answer its advertisers’ questions about metrics and be an industry leader, it needs to demonstrate that it follows industry-accepted practices and collaborates with other industry organizations that represent key stakeholders. Instead, Facebook continues to stand aloof from the industry and provide only the smallest of glimpses into its operations, hoping that the gravitational pull of its scale will pull the industry into its orbit.
Thank you to Erna Alfred Liousas, Jessie Liu, and Joanna O’Connell for collaborating with me on this post!