A/B testing allows you to test the hybrid mediation model (bidding + waterfall) against pure waterfall mediation. You’re also able to test the performance of different partner networks or test different settings on line items to find the most optimal settings.
A/B tests are available at each placement level. Make sure bidding and waterfall are already set up for those placements. Refer to Manage Networks for guides to connect each network and setup placements. Review Manage Placements for additional details on placement settings.
A/B Testing Bidding vs Waterfall 🔗
- Under the Mediation placement, setup the bidding and waterfall networks you wish to work with.
- Enable A/B test in Advanced Settings section of the placement. The A/B Test Settings section will automatically be created.
- Control Group (A) is the existing bidding and waterfall setup for the Mediation placement.
- Test Group (B) is a copy of the existing setup. Deactivate all the bidding networks and add the additional line items for the bidding networks in the ad groups.
Mediation automatically allocates 50% of ad requests to Control Group (A) and 50% of ad requests to Test Group (B).
- Each user is randomly assigned to one of the groups which ensures the efficiency of A/B tests.
A/B Test Performance Data 🔗
Once your A/B test has been created, the same A/B Test Settings area will include the following information. You will need to scroll horizontally to view all the information.
- Bid Rate
- Fill Rate
- Show Rate
The Analytics > A/B tests page will include the following information breakdown by all partner networks (sources) or per partner network with a visual graph across a set amount of time. The default time range is set on the last 7 days. This information can be filtered by target countries, network type (bidding vs waterfall/mediation), and by date.
- Estimated Earnings
- Fill Rate
Make adjustments to your tests over time to find your most optimal setup.
Disabling & Re-activating A/B Testing 🔗
If you find one of the groups is having better performance after reviewing A/B test analytics, you can disable the A/B test by disabling the worse performance group in the Mediation placement.
- All ad requests will automatically start going to the remaining active group once one group is being disabled.
- You can not disable both groups. At least one group has to be active.
If you want to re-activate the A/B test, you can directly edit any bidding and waterfall settings in the disabled group first and then turn on the Active toggle.
Last modified November 27, 2023