What is Optimization?

When optimizing a campaign - reallocating budget or traffic between media tactics or creative tactics - it's important to use the right metric. Upwave provides access to Outperformance Indicators for every tactic in a campaign. Outperformance Indicators are simple - they show the probability that a tactic is helping a campaign. A tactic is helping the campaign if it pulls the overall campaign lift up. 

A publisher with an 80% Outperformance Indicator, for example, has an 80% probability of helping the campaign, while a publisher with a 30% Outperformance Indicator has a 30% probability of helping the campaign (or, conversely, a 70% probability of bringing the overall campaign lift down.)

If you are looking for a tactic to put additional budget or traffic behind, choosing the top tactic by Outperformance Indicator gives you the highest likelihood of lifting overall campaign performance. Conversely, if you are looking to reallocate budget away from a tactic such as a publisher or audience, choosing the tactic with the lowest Outperformance Indicator gives you the highest likelihood of lifting overall campaign performance.

Historically, advertisers waited for statistical significance indicators before making optimization decisions. This is too late in most instances. That's by design - statistical significance isn't intended to be evaluated until a campaign is over. And its comparison to zero lift doesn't guarantee practically significant improvements in performance, which makes it of limited use for mid-campaign optimizations. (For more on this, and other differences between Statistical Significance and Optimization metrics, see our article comparing the two metrics.)

The Outperformance Indicator reflects the probability that a given tactic is contributing positively to overall campaign performance. It is best understood as a measure of confidence rather than a binary signal. For example, a tactic with an 80% Outperformance Indicator still carries a 20% probability of finishing the campaign below 50%.

For this reason, we recommend against waiting for the Indicator to reach a single "reliable" threshold. A more useful approach is to consider whether the current level of confidence is sufficient for the specific decision under consideration.

In making that assessment, it is helpful to weigh two distinct factors:

• Confidence: The further an Indicator sits from 50%, the greater the likelihood that acting on it will improve performance.

• Stakes: The potential cost to the campaign or to partner relationships if the decision proves incorrect.

Lower-stakes decisions are those that shift impressions without incurring additional cost or affecting business relationships, such as increasing the rotation of a stronger-performing creative within the same environment or tightening frequency caps. Because the downside of these adjustments is limited, and because leaving a campaign unchanged is itself a decision, it is generally reasonable to act on them at a lower level of confidence.

Higher-stakes decisions involve reallocating budget to or from a publisher, data provider, or other partner. Since an incorrect decision in these cases can result in wasted spend, we recommend continuing to monitor performance and acting once the Indicator shows a stronger signal.

Recommended guidelines (mid-campaign, with a sample size greater than 100):

• Between 40% and 60%: No recommendation for lower-stakes optimizations; proceed if you are comfortable.

• Above 60% or below 40%: There is generally sufficient confidence to proceed with lower-stakes optimizations.

• Above 70% or below 30%: There is generally sufficient confidence to proceed with higher-stakes optimizations.