Mediation

A/B Tests

A/B testing allows you to test the hybrid mediation model (bidding + waterfall) against pure waterfall mediation. You’re also able to test the performance of different partner networks or test different settings on line items to find the most optimal settings.

Any changes to line items shared between Control Group (A) and Test Group (B) will affect both groups. To make adjustments on a line item that does not affect both groups, create a duplicate line item setup then assign the original line item to Control Group (A) and the duplicate line item to Test Group (B). This will then allow changes to a line item that affects only one group.

A/B tests are available at each placement level. Make sure bidding and waterfall are already set up for those placements. Refer to Manage Networks for guides to connect each network and setup placements. Review Manage Placements for additional details on placement settings.

A/B Testing Bidding vs Waterfall πŸ”—

  1. Under the Mediation placement, set up the bidding and waterfall networks you wish to work with.
  2. Enable A/B test in Advanced Settings section of the placement. The A/B Test Settings section will automatically be created.
  3. Control Group (A) is the existing bidding and waterfall setup for the Mediation placement.
  4. Test Group (B) is a copy of the existing setup. Deactivate all the bidding networks and add the additional line items for the bidding networks in the ad groups.

Mediation automatically allocates 50% of ad requests to Control Group (A) and 50% of ad requests to Test Group (B).

  • Each user is randomly assigned to one of the groups which ensures the efficiency of A/B tests.

A/B Test Performance Data πŸ”—

Once your A/B test has been created, the same A/B Test Settings area will include the following information. You will need to scroll horizontally to view all the information.

  • Active
  • Network
  • Earnings
  • Attempts
  • Bids
  • Fills
  • Impressions
  • eCPM
  • Bid Rate
  • Fill Rate
  • Show Rate
  • Clicks
  • CTR

The Analytics > A/B tests page will include the following information breakdown by all partner networks (sources) or per partner network with a visual graph across a set amount of time. The default time range is set on the last 7 days. This information can be filtered by target countries, network type (bidding vs waterfall/mediation), and by date.

  • Estimated Earnings
  • Impressions
  • Fill Rate
  • eCPM

Make adjustments to your tests over time to find your most optimal setup.

Disabling & Re-activating A/B Testing πŸ”—

If you find one of the groups is having better performance after reviewing A/B test analytics, you can disable the A/B test by disabling the worse performance group in the Mediation placement.

  • All ad requests will automatically start going to the remaining active group once one group is being disabled.
  • You can not disable both groups. At least one group has to be active.

If you want to re-activate the A/B test, you can directly edit any bidding and waterfall settings in the disabled group first and then turn on the Active toggle.