What are Norms?

Upwave provides various norm types to help customers contextualize Brand Lift performance in alignment with their campaign KPIs. Upwave Norms take into account all campaigns ensuring reliable estimates and serving as a precise indicator of expected lift. Plus, our AI-agent, Bayes, enhances the accuracy of our classification methods.

Upwave Norms are available directly in the Upwave dashboard and as a downloadable spreadsheet–making it easy to access, compare, and evaluate results on a multitude of levels.

Evaluated over campaigns that measure this KPI in the Upwave platform within the last 30 days.

Platform Norms are dynamic, and help account for tentpole events or periods (i.e. Super Bowl, or periods of unrest.)

Evaluated over campaigns in Upwave platform within the last 30 days, accounting for brand baselines.

Brand Maturity Norms set better expectations, and are useful for new entrants and dominant incumbents alike.

Evaluated over campaigns in Upwave platform by industry vertical. 

Industry Norms contextualize performance in terms of the average for an industry vertical.

Evaluated over campaigns in Upwave platform measuring a given tactic.

Tactic Norms can be used to compare your brand’s performance in that specific tactic, classified by Bayes or set by you.

Norms are available right in Upwave’s platform. You can access them in a variety of ways convenient to your different workflows.

1. View norms in lift tables (Open Beta)

   Upwave Norms can be viewed in lift tables, for faster, easier interpretation of your brand lift performance.  Click the norms toggle above the upper right corner of a lift table to select which norms to display in a norms column:

   • Platform Norms for the overall cut and Tactic Norms for other relevant cuts.

   • Industry Norms for the overall cut (no norms for other cuts).

   • Brand Maturity Norms for the overall cut (no norms for other cuts).

   

   Upwave’s AI agent Bayes recommends Tactic Norms, for Channel and Environment, for many cuts so you don’t need to.  However, you can set, change, or remove the Tactic Norm applied to a cut.  

   Hover over a cut in the sidebar, click the gear icon, select your preferred norm in the drop-down, and click save.  Bayes’ recommendations for cuts are visible in the drop-down even after you make a change.

   

2. Quick reference using the norms button

   For a table that’s easy to reference right in the platform, navigate to your campaign’s dashboard tab and click the norms button.

In the modal, you can see Platform Norms, Industry Norms, and Brand Maturity Norms for available KPIs measured on your campaign, alongside your campaign’s lift.  Hover over the norms to see the confidence interval.

Campaigns without an associated industry do not display Industry Norms.  If you don’t see Industry Norms for your campaign, follow these instructions to confirm that an industry is associated with your campaign.

3. Download a norms spreadsheet

As with option (2) above, navigate to your campaign’s dashboard tab and click the norms button.  Within that same modal, Click the download link to download an XLSX spreadsheet.  In the first tab you’ll find Platform Norms and Brand Maturity Norms for available KPIs measured on your campaign, your campaign’s lift for each, and 95% confidence intervals.

In the second tab you’ll find Industry Norms for available KPIs measured on your campaign, your campaign’s lift, and 95% confidence intervals.  You’ll also find the percentage of campaigns beaten within a given industry.

Campaigns without an associated industry do not display Industry Norms.  If you don’t see Industry Norms for your campaign, follow these instructions to confirm that an industry is associated with your campaign.

In the subsequent tabs you’ll find Tactic Norms for available KPIs measured on your campaign along with 95% confidence intervals for each tactic.

• The third tab houses Tactic Norms for Media Channels.

• The fourth tab houses Tactic Norms for Ad Environments.

Upwave provides you with different types of norms so you can contextualize the brand lift performance in ways that work for you.  

When contextualizing lift for your whole campaign use Platform Norms, Brand Maturity Norms, or Industry Norms.

Platform Norms account for the most recent performance of all campaigns live during the last 30 days, enabling you to easily account for tentpole events or periods (i.e. Super Bowl or the holiday season).

Brand Maturity Norms also factor in the maturity of your brand with respect to its category. These norms reflect the empirical reality that driving lift from very low or high baselines is relatively harder to do than for brands in the middle, and that difficulty varies by brand KPI. Brand Maturity Norms help you set better expectations for incremental lift for all brands, but are especially useful for new entrants and dominant incumbents alike.

Industry Norms allow you to contextualize performance in terms of the average for an industry vertical. They can be handy when you want to account for dynamics particular to your industry not reflected in baselines. They are based on final lift estimates of campaigns, classified by industry.

Tactic Norms are what to use to benchmark specific tactics’ performance in the context of the broader marketplace. They allow you to maximize performance signals and potentially include competitors, providing comparison points beyond the performance of the campaigns in your own portfolio. They are based on final lift estimates for tactics classified by Bayes on campaigns.

When assessing brand lift for a KPI on your campaign against a norm, first compare whether it’s higher, lower, or about the same. If your lift is about equal to the norm, then you’re doing as well as the average campaign in the comparison set, correcting for the scale of your campaign using the standard error. If higher, it’s better than average; and if lower, it’s less than average.

Upwave norms include a mean and a confidence interval. In the example below, the lift for the campaign for Aided Awareness is +2.4% points, the norm for Aided Awareness is +1.2% points, and the 95% confidence interval is +0.8% to +1.5%. This means:  

• The campaign’s Aided Awareness lift is beating the average.

• In 95% of cases we’d expect the actual average performance to fall between the upper and lower bounds of the interval.

• Given the lift in Aided Awareness is greater than the upper bound of the confidence interval, we are confident that the norm for this KPI is below the lift estimated for your campaign

  

For Industry Norms, Upwave offers a mean, confidence interval, and a % Campaigns Beaten within Industry metric. This metric provides a sense of the distribution of results, and tells you simply what proportion of campaigns in a given industry your campaign is doing better than. In the example below, the % Campaigns Beaten metric for Favorability is 73%, which can be read as: your campaign performed better than 73% of hypothetical campaigns of the same size in the same industry.

Upwave reports norms for the following Brand KPIs for any type of norm - Platform Norms, Brand Maturity Norms, Industry Norms, Tactic Norms - when there is sufficient data: 

• Unaided Awareness

• Aided Awareness

• Ad Recall

• Brand Preference

• Consideration

• Purchase Intent

• Purchase Propensity

• Familiarity

• Favorability

• Recommendation Intent

Campaigns without an associated industry do not display Industry Norms.  If you don’t see Industry Norms for your campaign, navigate to the campaign information page to confirm an industry has been set for your campaign.  

You can set or update the industry field for pre-launch or live campaigns (for completed campaigns, please contact Upwave Support).

20+ verticals are supported for Industry Norms - click here to see them all.

We support nearly a dozen Tactic Norms across Media Channels and Ad Environments - click here to see them.

Platform and Brand Maturity Norms are updated on a daily basis, while Industry Norms and Tactic Norms are updated on a quarterly basis (typically several weeks into each quarter in order to include campaigns that ended in the preceding quarter).

Platform and Brand Maturity Norms typically show minimal variation from day to day, usually around  .1% points.

In the case of Industry Norms and Tactic Norms, for most industries or tactics we would expect typical changes quarter to quarter to be less than 0.1% points. For industries measuring  fewer advertising campaigns or for tactics less commonly employed and mapped, slightly larger changes on the order of 0.2% to 0.3% are possible.

When calculating any type of norm, Upwave takes a weighted average of the most recent lift results for all campaigns included in the calculation. These weights follow a standard meta-analysis procedure that weights each estimate by the precision of the estimate (roughly proportional to the sample size of the campaign). Ultimately this procedure takes into account the accuracy of the individual lift estimates, while not giving too much weight to large campaigns.