# A/B Tests

A/B testing allows you to test the hybrid mediation model (bidding + waterfall) against pure waterfall mediation. You're also able to test the performance of different partner networks or test different settings on line items to find the most optimal settings.

**Any changes to line items shared between Control Group (A) and Test Group (B) will affect both groups.** To make adjustments on a line item that does not affect both groups, create a duplicate line item setup then assign the original line item to Control Group (A) and the duplicate line item to Test Group (B). This will then allow changes to a line item that affects only one group.

A/B tests are available at each placement level. Make sure bidding and waterfall are already set up for those placements. Refer to [Manage Networks](/en/mediation/manage-networks/) for guides to connect each network and setup placements. Review [Manage Placements](/en/mediation/manage-placements/) for additional details on placement settings.

## A/B Testing Bidding vs Waterfall

   1. Under the Mediation placement, set up the bidding and waterfall networks you wish to work with. 
   2. Enable A/B test in **Advanced Settings** section of the placement. The **A/B Test Settings** section will automatically be created.
   3. **Control Group (A)** is the existing bidding and waterfall setup for the Mediation placement.
   4. **Test Group (B)** is a copy of the existing setup. Deactivate all the bidding networks and add the additional line items for the bidding networks in the ad groups.

Mediation automatically allocates 50% of ad requests to Control Group (A) and 50% of ad requests to Test Group (B).
   - Each user is randomly assigned to one of the groups which ensures the efficiency of A/B tests.


## A/B Test Performance Data
Once your A/B test has been created, the same A/B Test Settings area will include the following information. You will need to scroll horizontally to view all the information.
- Active
- Network
- Earnings
- Attempts
- Bids
- Fills
- Impressions
- eCPM
- Bid Rate
- Fill Rate
- Show Rate
- Clicks
- CTR

The **Analytics** > [**A/B tests**](https://helium.chartboost.com/analytics/abtest) page will include the following information breakdown by all partner networks (sources) or per partner network with a visual graph across a set amount of time. The default time range is set on the last 7 days. This information can be filtered by target countries, network type (bidding vs waterfall/mediation), and by date.
- Estimated Earnings
- Impressions
- Fill Rate
- eCPM

Make adjustments to your tests over time to find your most optimal setup.


## Disabling & Re-activating A/B Testing

If you find one of the groups is having better performance after reviewing A/B test analytics, you can disable the A/B test by disabling the worse performance group in the Mediation placement.
   - All ad requests will automatically start going to the remaining active group once one group is being disabled.
   - You can not disable both groups. At least one group has to be active.

If you want to re-activate the A/B test, you can directly edit any bidding and waterfall settings in the disabled group first and then turn on the **Active** toggle.