Don’t be blinded by the ad strength: an experiment on the impact of ad strength

Now that Responsive Search Ads (RSAs) have become the default ad type, ad strength is becoming more important. Therefore, we set up an experiment to understand the influence of this metric on performance. Surprisingly, the results revealed that ad strength is actually less important than you might think. 

No correlation between quality score and ad strength

At first, we investigated if there is a correlation between quality score and ad strength by comparing those two metrics among 5.000 ad groups. The results reveal that there is no correlation.

The experiment

We wanted to do a more thorough experiment to further investigate what the impact is of ad strength on performance. To do this, we set up a drafts and experiment for Albert Heijn, a big Dutch retail client, in which we manipulated the ad strength.

How did we manipulate the ad strength?

To tweak ad strength and discover the possible differences in performance, we made use of pinning and unpinning headlines and descriptions. In the control group, we pinned all headlines and descriptions which resulted in good ad strength. In the experiment, all the headlines and descriptions were unpinned which resulted in excellent ad strength. 

How did we limit the influence of external factors?

To get a solid and clean experiment, and to make sure the only thing that could affect the outcome was ad strength, all conditions were kept the same such as:

• Fixed bids
• Equal keyword scope
• Same extensions
• Same content of the RSAs
• No ETAs
• Distribution of impressions: the data was cleaned by removing ad groups where the difference in impressions between the control and experimental group were >+10%

The results

Finding 1: a higher ad strength doesn’t mean better performance (!) 

For us this came as a big surprise, as we thought that a higher ad strength would indicate higher quality ads and thus better performance. However, we saw that an excellent ad strength showed a lower click-through-rate (CTR), a lower conversion rate and a lower quality score compared to a good ad strength. 

Results experiment

Possible explanation for finding 1

What we’ve experienced in the past is that unpinning is a really important factor for Google to determine the ad strength. Google states that by unpinning, different combinations can be tested which will potentially increase performance. However, we think this will also lead to less relevant combinations which will negatively influence performance. This is supported by the fact that in our experiment the lower quality score was mostly a result of the ad relevance (92%) and the expected CTR (8%). So, unpinning does lead to a higher ad strength, but the less relevant combinations ultimately lead to a lower CTR, conversion rate and quality score. 

Finding 2: time did not positively contribute to better performance

Even though Google will also test less relevant combinations, we expected it would learn from the less relevant combinations causing the quality score to improve over time. However, further analysis showed that this was not the case. 

Quality score over time

Next steps

Don’t be blinded by the ad strength! A higher ad strength doesn’t mean: better CTR, better conversion rate or better quality score. Keep the focus on performance, even though it might mean pinning your headlines and descriptions and therefore having a lower ad strength. 

Keep in mind that RSAs and ad strength are still in development, so the results we presented might be different in one or two years. For this reason, we recommend to not only conduct a similar experiment yourself, but to also leave this experiment running so you can keep a close eye on the impact of the ad strength.