Champion/Challenger vs. A/B Testing

Marketo and other systems provide options for determining the most effective marketing content. Companies can use a multitude of tools, including A/B testing and Champion/Challenger testing, to optimize such content.

During A/B testing, a company tests two different versions of a product, which often consist of emails or web pages, to see which one has more success. The “A” version is that which is expected to heed the best results, and the “B” version is expected to be the next best option. A/B testing employs each of the two versions to see which receives a better response, with half of the test subjects receiving the “A” version and the other half receiving the “B” version. A/B testing consists of a single test to determine the winner.

Champion/Challenger testing is continuous until a company determines a winner (1). With Champion/Challenger testing, the “Champion” version is the version thought to be likely to produce the best results, and the “Challenger” version is the version thought to be the second choice option. Thus, after Champion/Challenger testing is conducted, the “Champion” that is decided upon is utilized in a company’s marketing efforts.

Within the Marketo marketing automation system, companies can conduct both A/B testing and Champion/Challenger testing by sending the “A” or the “Champion” email to one group of people, and sending the “B” or “Challenger” email to another group. Companies can choose which criteria decide the winner of this testing, which can include “Opens,” “Clicks,” “Click to Open %,” “Engagement Score,” and “Customer Conversion” (2). Marketo makes testing straightforward, so that, through Marketo, a company can input the two versions it wants to test, send the emails, and find the results that they need.

When interpreting the results of A/B testing and Champion/Challenger testing, the winner is chosen if the results are statistically significant. If the “A” or the “Champion” option produces similar results to the “B” or “Challenger” option, causing the results to be statistically insignificant, then a company may need to test more campaigns to attain better results. If the testing consistently has statistically insignificant results, then the two options may simply produce similar outcomes. There are many free online statistical calculators available, including the AB Testguide, that companies can utilize to detect whether their testing had results that were statistically significant (3).

RightWave provides expertise and guidance to its customers for marketing strategies and tactics. For more information, contact RightWave.

 

Sources:

  1. Marketo, “Champion/Challenger vs A/B Test in Engagement Program”, 25 September 2019, https://nation.marketo.com/t5/Product-Discussions/Champion-Challenger-vs-A-B-Test-in-Engagement-Program/td-p/193849
  2. Marketo, “Creating an A/B Test program in Marketo”, Undated, https://nation.marketo.com/t5/Product-Documents/Creating-an-A-B-Test-program-in-Marketo/ta-p/248391
  3. AB Testguide, “AB Testguide”, Copyright 2018, https://abtestguide.com/calc/

More articles

On-Demand Webinar: 5 Actions to Increase Marketing Ops Impact on Business Value

As marketing organizations respond to the changing environment, B2B companies must adapt quickly. Results-oriented marketing continues to be challenging as top influencers are more engaged in the decision-making, and as prospective buyers develop more insights, the market’s expectations continue to rise. With every marketing dollar buying less in this time of inflation and difficulties in…

Prestigious ISO awards confirm RightWave’s Quality and Security

CEO Anurag Khemka is extraordinarily proud of this achievement. He stated that “The certifications provide external confirmation of the high standards of quality and information security that RightWave has always pursued. Our customers expect and deserve the highest quality of consistent and dependable services along with rigorous safeguards of their information. ISO 9001 and ISO…