When optimizing a campaign - reallocating budget or traffic between media tactics or creative tactics - it's important to use the right metric. Upwave provides access to Outperformance Indicators for every tactic in a campaign. Outperformance Indicators are simple - they show the probability that a tactic is helping a campaign. A tactic is helping the campaign if it pulls the overall campaign lift up. 

A publisher with an 80% Outperformance Indicator, for example, has an 80% probability of helping the campaign, while a publisher with a 30% Outperformance Indicator has a 30% probability of helping the campaign (or, conversely, a 70% probability of bringing the overall campaign lift down.)

If you are looking for a tactic to put additional budget or traffic behind, choosing the top tactic by Outperformance Indicator gives you the highest likelihood of lifting overall campaign performance. Conversely, if you are looking to reallocate budget away from a tactic such as a publisher or audience, choosing the tactic with the lowest Outperformance Indicator gives you the highest likelihood of lifting overall campaign performance.

Historically, advertisers waited for statistical significance indicators before making optimization decisions. This is too late in most instances. That's by design - statistical significance isn't intended to be evaluated until a campaign is over. And its comparison to zero lift doesn't guarantee practically significant improvements in performance, which makes it of limited use for mid-campaign optimizations. (For more on this, and other differences between Statistical Significance and Optimization metrics, see our article comparing the two metrics.)

The Optimization Indicator states the probability that a tactic is helping the overall campaign. As a result, this Indicator is also an Indicator of stability.

For example: When a publisher has an 80% Optimization Indicator, that means there is a 20% probability the Indicator will end the campaign under 50%. 

So, the important question isn't whether a tactic's Optimization Indicator has reached a certain threshold at which point it becomes reliable, but whether this level of confidence that a tactic is helping a campaign is sufficient to make a particular optimization decision. 

Many optimizations, such as shifting creative rotation weights in an ad server or shifting budget between programmatic line items, are lower-stakes decisions that are best made on a regular (e.g. weekly) basis and for which a lower threshold can suffice (e.g. when the Optimization Indicator is lower then 30% or higher than 70%). 

Other optimization decisions, such as removing a media partner from a media plan, are higher stakes decisions about which advertisers require a higher threshold of confidence.