Skip to main content
All CollectionsMicrosoft AdsAd Management
AB Testing for Ads (Microsoft Ads) - User Guide
AB Testing for Ads (Microsoft Ads) - User Guide
Geetanjali Tyagi avatar
Written by Geetanjali Tyagi
Updated over 8 months ago

On this page:


What is it?

The AB Testing for Ads optimization is a data-driven tool that helps you compare ads with statistically significant data in the same ad group. It helps you find ads that meet your business goals or contribute to the success of your advertising campaign, pause underperforming ads and create new ones to continue testing.

How does it work?

The system analyzes and tabulates the ads for comparison under each ad group, filtering only the underperforming ads based on the selected criteria. The analysis takes into consideration data pulled in real-time from your Microsoft Ads account, and it can look up to 180 days of data.

The optimization makes the recommendations based on the selected parameters and considers only ads with statistically significant data. This optimization supports copying tracking templates.

Why should I use it?

The optimization identifies the best and worst-performing ads in active ad groups, allowing the analysis of the suggestions by comparing clicks and conversions parameters side by side before pausing the lowest performance ads.

The system also recommends high-performing headlines, description lines, and display URLs (from your best-performing ads) to create new ads in the ad groups where you are pausing ads with a lower performance.

Getting Started

Select the Ad type you'd like to test. You'll be able to choose from different types of text ads, and dynamic search ads.

Then you can select the metric by which you'd like to find winning ads.

If you wish to, you can also filter the ads you'd like to test by labels for easier analysis.

Basic Settings

  • Scope:

Select the campaigns or labels for which you want to see results. You can also see any labels that have been assigned to the ads.

  • Date range:

Select the date range you want to take into consideration for the statistical analysis. By default, the system looks at the last 30 days.

You can also choose to select a different date range of your choice using the custom date range option.

  • Network:

Search, Display, or both.

  • Turbo Mode:

The Turbo Mode lowers the traffic threshold required to show optimization suggestions and gives you access to a larger, less-refined data set. When you turn on the Turbo Mode in AB Testing, you'll see all active ad groups with all active ads with more than one impression in them.

Advanced Settings

These settings let you select the parameters you are going to use to find the ads.

  • Test Required Confidence: This setting lets you choose the confidence level you need the system to have (mathematical accuracy) for the results. You can select one of the three available settings (90%, 95%, and 99%).

For example, when you choose 99% it means that the system is 99% sure of the results it shows. Usually, as the confidence percentage increases, the number of suggestions decreases. We recommend using a 95% confidence setting in most cases.

  • Mobile Preferred: Select Yes to compare only mobile preferred ads, No if you want to exclude mobile preferred ads in the comparison, or any to include both mobile preferred and desktop ads.

  • Consider Ads with Impression count at least: You can choose between 100, 500, and 1000 impressions.

  • Number of ad groups to test: The maximum number of ad groups shown at a given time.

  • Consider ads with impression count at least: This can be 100, 200, 500, 1000.

  • Max difference in impressions between ads being compared: This can be 'any', 10%, 25% or 50%.

  • Min active ads to leave in adgroup after pausing suggested ads: This can be either 1 or 2.

  • Find winner by: You can choose between CTR, Conversion Rate, and Conversion/Impression (Combination of CTR and Conversion Rate). We recommend using CTR because the purpose of the ad is to get the user to click and go to the site.

What Do the Green/Red Highlights on Metrics Mean?

The green and red highlights on metrics indicate whether an ad copy is performing positively or negatively in relation to that specific metric.

For instance, if an ad copy's Conv/Impr metric is highlighted in green, it signifies that the ad is winning in terms of Conversion per Impression. In other words, it's generating more conversions for every impression compared to other ad copies. Conversely, the other ad copy will simultaneously have its Conv/Impr metric highlighted as Red.

Additionally, it's possible for an ad copy to be winning (green) in one metric, such as Conv/Impr, while simultaneously losing (red) in another metric, such as CTR.

Get Keywords

Here you'll get a list of keywords associated with the ad group, sorted by impressions. This list of keywords is available for Microsoft Ads campaigns.

You'll also see this option when creating new ads

Create New Ad

Here you can create new Responsive Search Ads or Call Ads with Top Suggestions for each component and preview how it would look. When you are satisfied, just press the "Create Ad" button.

You'll find the ad count for each ad group in suggestions by default as soon as the tool loads. You can also create a filter based on ad count to view only those ad groups that have more or less than x ads or specify a range.

For example, view only those ad groups which have an ad count between 4-20. The filter can be saved for future use too.

Suggestions from Existing Ads

These are based on the clickthrough rate and from other ads in the same campaign.

All the different components of the ads in the same ad group or campaign are considered individually. The suggestions are made based on the aggregated performance of each component, where the components with comparatively high performance make up the suggestion list.

AI Suggested Variations

With this option selected, you'll be recommended potentially high-performing ad assets by our AI model, allowing you to get quality suggestions and more choices. This is only available for headlines and descriptions for RSAs.

The tool scrapes your website to extract relevant content and data using the domain (URL) associated with the winning ad copy. This extracted data is then filtered and processed through an AI model (Open AI), resulting in the suggested ad assets.

Note: If the tool encounters difficulties in accessing the domain or scraping data for any reason, the AI will then generate asset suggestions based on the existing ads. In such cases, the tool will explicitly mention the source from which the AI obtained the data for generating these suggestions.

You are also able to load more suggestions if needed.

Creating Similar Ads

For every ad suggestion, you'll notice a duplicate icon that lets you create a similar ad. This will prompt the ad creation window, where the information will be pre-loaded with the ad's data for you to modify.

Note: This option is not available for Expanded Text Ads as these Ads can no longer be created.

Campaign Projected Gain

The Campaign Projected Gain feature gives you the projected gain for a selected metric using the 'Find winners by' filter. This is Optmyzr's estimation of if an ad copy is optimized. The metrics will be based on the filter you select. For example, a ROAS filter will show the projected gain of revenue.

Demo Video

Did this answer your question?