Finding a marketing partner that aligns with your business objectives is harder than ever. Thousands of marketing technology vendors claim to be the best for your company, but how do you know which are telling the truth and which are just, well, marketing?
Performing a head-to-head comparison test (H2H), if done correctly, is an important way to determine which performance marketing partner is best equipped to efficiently drive revenue at a meaningful scale. Additionally, if you’re evaluating new partners, H2H testing can help you decide whether to go with the new partner or stay with your current partner.
But running tests can be expensive and require a lot of resources, so sticking to head-to-head testing best practices is crucial for choosing a partner quickly and accurately. The right methodology mainly consists of proper measurement and setup that eliminates of any outside factors that could skew the results. Here, we’ll help explain how you can implement a testing methodology that will yield the clearest results.
Use the same budget.
You already know that budget size and pricing methods affect the performance of your retargeting campaigns. So it’s important to ensure that the vendors you’re testing are being evaluated on the same budget to ensure both competitors are being evaluated on the same criteria.
Split your audience evenly and test both vendors at once.
To accurately gauge H2H performance, your audience should be evenly and randomly divided amongst the vendors. This way, each vendor can target each audience separately, with no overlap. This will avoid any issues with one shopper seeing ads from multiple partners, which makes it difficult to understand which partner actually drove a single conversion. Additionally, it will ensure that each vendor has a large enough audience to prove statistically significant results.
Lastly, vendors should be tested at the same time rather than sequentially, since factors such as seasonality and ad prices will change over time. This provides the clearest path for evaluation and enables marketers to truly understand which technology provider will deliver the best results.
Determine The Best Testing Period.
Determine how long will it take to understand the true performance of each vendor. We typically recommend at least 30 days for statistically significant results. Allocating a few days for both partners to ramp-up will be important as well. It takes time for a retargeting technology to start understanding shopper behavior, so both partners should be running for at least a few days before the test begins. Talk to your account rep on both ends to determine the best time frame for testing.
Make sure you’re taking full advantage of each vendors’ solutions.
Creative can hugely affect the performance of any campaign, as can different promotions – obviously a 20% off CTA is more likely to get clicked on than a 10% CTA. By ensuring both vendors are utilizing the same CTAs and creatives, you’ll know that it’s not your creative that’s skewing the results one way or another.
Use the same success metrics to evaluate each partner.
Advertisers should do an apples-to-apples comparison when evaluating performance of both partners, especially since the partners you’re evaluating could be using a different attribution model in their reporting dashboard. For example, one may rely on a 30-day last click model, and one may use a 7-day last click model. If you’re using your own attribution tool, ensure that tracking is setup correctly and both vendors know how success is being measured. Setting this expectation is key, so that both partners know what their target is and then can optimize accordingly.
We strongly recommend using one primary performance metric, as well as one secondary volume metric. A performance metric could be return on ad spend, or cost-per-sale. A volume metric could be number of sales and total revenue. This is so that you can measure both performance and ability to scale. Simply looking at performance means nothing if the vendor only drove one sale over the course of 30 days, while the other partner drove 100.
(Learn more: Attribution and what it means).
Head-to-Head Testing for Choosing the Right Partner
Head-to-head testing can be expensive and time-consuming, but it’s a worthwhile pursuit. And it’s in your organization’s best interest to develop the internal know-how to effectively evaluate the tests. How you approach the tests will not only impact the validity of your results, but also how you leverage your internal resources.
A fair and effective test will reveal who the best partner is. And a good partner should not only help you maximize your campaign reach but also pave the way for more successful future campaigns to grow your business!
To learn more, download the ebook below!
