Back to the blog

The conundrum of analysing promotional ROI

Julien - Lucky Cart

By Julien Guitard
Chief Data and Science Officer
On 27 March 2018

This article is an excerpt from our ebook Sales Promotion: How to improve the performance of your investments which you can download for free. 


Nearly 3 out of every 4 shoppers say they are strongly influenced by promotions (source: Nielsen). But in an environment where companies are constantly trying to outdo each other with promotions, their behaviour has changed and consumers have adapted to retailers’ and brands’ strategies. This has resulted in many adverse effects which have widely multiplied in recent years.
To evaluate this observation, we need to define measures for promotional effectiveness that are both convincing for decision makers and scientifically sound to guarantee reliability and reproducibility of results.

Finding a measurement for promotional effectiveness

Coming up with convincing and reliable analysis can prove difficult for manufacturers and retailers. If we consider the promotion as an investment, the ROI is the measure that responds to these criteria and we can use the following formulas:

  • Revenue generated / Investment
  • Profit generated / Investment
  • Extension of customer footprint / Investment

With all of these methods, the denominator is unambiguously defined: this is the promotional budget itself. However, the numerator is not unequivocally defined. There are several possible metrics and each metric can be defined in several different ways. For example, if we are interested in revenue generated, three options can be considered. We could:

  • Use all revenue eligible for the promotion
  • Also take into account revenue generated during and just after the promotion
  • Consider consumers who would have made purchases anyway

Limitations of traditional methods

The first option is the simplest but it has the major disadvantage that it includes opportunism effects of some consumers who haven’t changed their purchasing behaviour while benefiting from the promotion. It also does not take into account factors of storage or anticipation.

The second option takes into account the factors of storage and anticipation, but only as long as we have an idea of the timeframe of these effects (1 week, 1 month, etc.) which generally depends on the products and consumer habits. Furthermore, this second option does not resolve the issue of windfall effects.

The third option allows us to do this but immediately presents a methodological issue: how do we isolate the people who would have purchased in the same way without the promotion? Counting the eligible revenue and volume of promotions is no longer enough, we need to define a counterfactual.

Example 1

A promotional campaign lasted for 10 days with a budget of €100K and an average discount rate of 5%. The budget was fully consumed and no A/B testing was carried out. The consumer base is 1M people.

Over the 10 days prior to the campaign, the total revenue was €9M, over the 10 days of the campaign the revenue was €12M and over the following 10 days €10M.

What is the ROI of the campaign?

If we use the formula ROI = revenue generated/Investment, we already know that the numerator is equal to the budget consumed which is €100K. Calculating the numerator is more complicated:

  • The revenue has increased by €3M between the pre-campaign and the campaign (€12M – €9M)
  • This increase of €3M cannot be solely attributed to the campaign because we can see that the revenue has increased by €1M after the campaign (€10M – €9M).
  • Finally, the incremental revenue cannot be higher than the revenue eligible for the promotion which is €2M (€100K ÷ 5%).

The only thing we can say about the numerator is that it is a fraction of this €2M. It is clear that without A/B testing it is impossible to calculate the ROI of a promotion.

Econometrics and machine learning come to the rescue

Fortunately this counterfactual notion has been studied a lot in marketing and more generally in social sciences and there are techniques to quantify it. This is particularly the case for A/B testing when it is reliable conducted, econometrics (which generalises the notion of A/B testing), as well as machine learning and predictive analysis which itself generalises A/B testing and econometrics. In all cases the idea is the same. Before the start of the promotion we randomly split the consumers into two groups:

  • The control group who won’t get any promotion
  • The test group who will receive the promotion

Then you check that the two groups behave in the same way before the promotion. In these conditions we can attribute all behavioural differences (revenue per day and per person or choice of products for example) to the promotion. In practice, the random choice of groups requires rather clever technology to reduce bias and irrelevant results of sampling and ensure behavioural equivalence. Behaviour before the promotion or in absence of a promotion can be modelled in various ways:

  • Simply by calculating averages such as in the case of A/B testing or econometrics
  • In a more sophisticated way by using machine learning

Example 2

The same promotional campaign is launched, this time with a control group representing 20% of consumers.

The 2 groups are assembled so that their consumption is practically identical over the 10 days prior to the campaign:

  • €0.9005 / day / consumer in the control group
  • €0.9003 / day / consumer in the test group

During the campaign this consumption increases to:

  • €1.0402 / day / consumer in the control group
  • €1.2402 / day / consumer in the test group

The A/B testing, based on a control group with identical consumer behaviour to the test group, allows us to now calculate the difference in consumption that can be attributed to the promotion: €0.2 / day / consumer (1.2402 – 1.0402) meaning €1.6M over the duration of the campaign (0.2 x 80% x 1M).

We can now therefore calculate the ROI of this promotion: 16, meaning 16 euros of incremental revenue for each euro invested (€1.6M / €100K).

Would you like to find out more?

Discover how Lucky Cart can help you increase the ROI of your promotions.