Models for evaluating advertisement efficiency
Nina Golyandina
St.Petersburg State University
Department of Statistical Modelling
Data Science and Advertising
8 June 2017, London
1/29
Nina Golyandina
Models for evaluating advertisement efficiency
Outline
Part I. A general review of consumer behaviour
Part II. Probabilistic model of consumer behaviour
Part III. TV spot testing
Part IV. The model of switching and the Dirichlet model
Part V. The model of switching and clicks. Discussion
2/29
Nina Golyandina
Models for evaluating advertisement efficiency
Part I
Part I. A general review of consumer behaviour
3/29
Nina Golyandina
Models for evaluating advertisement efficiency
Part I. A general review of consumer behaviour
There are different models of buyer behaviour.
There are different models of advertisement efficiency.
There are different models of on-line advertisement efficiency.
There are many papers and books devoted to these problems.
As a result, there are different qualitative models, case studies, or
complex mathematical models (e.g. the Dirichlet model).
On the other hand, different machine-learning procedures frequently use
complex quantitative models with many (non-interpreted) parameters.
What we want: to consider simple probabilistic models with basic
easy-to-interpreted parameters that help to understand the consumer
behaviour, estimate quantitative characteristics, predict the future
behaviour.
4/29
Nina Golyandina
Models for evaluating advertisement efficiency
Part I. A general review of consumer behaviour
Thus, we want to describe the consumer behaviour and the
advertisement efficiency.
We consider a mathematical probabilistic model with two main
parameters:
the effect of external actions directed to the buyer (we mean
advertisement);
the measure of behavioural loyalty (=conservatism/memory in the
product choices). Behavioural loyalty means to buy the same
product when there are few alternatives available.
The model should describe the process of the product
choice/purchase/click for a random buyer.
Remarks.
(1) We do not discriminate between brands and products.
(2) Assume that buyers have similar behaviour within the same
(sub)category.
5/29
Nina Golyandina
Models for evaluating advertisement efficiency
Part I. How to interpret probabilities
Certainly, each person is unique and has different characteristics.
Statistical or probabilistic model corresponds to transformation of
individual features to probabilities.
Difference:
Individual approach: the first 10 of 100 people were affected by the
advertisement and the next 90 people were not. That is, the first 10
people differ from the other 90 people.
Probabilistic approach: all have the same behaviour, each person
was affected with probability 0.1 and was not affected with
probability 0.9.
The result: the number of affected people is 10% in average —
the same in both approaches.
Advantage of probabilistic model is that each person has the same (or
similar) behaviour.
6/29
Nina Golyandina
Models for evaluating advertisement efficiency
Part II.
Part II. Probabilistic model of consumer behaviour
7/29
Nina Golyandina
Models for evaluating advertisement efficiency
Part II. The model
Let us discuss how to describe the consumer behaviour.
Let a consumer use the product/brand B.
How does she react on the advertisement of Product A? What is her next
choice in the shop?
An example of the behaviour of a random buyer
With probability padv she is affected by the advertisement and
therefore buys Product A.
What are her actions with probability 1 − padv ?.
This depends on the loyalty of the customer to Product B.
The loyalty can be described in probabilistic manner: with probability
pch the customer is ready to change the product/brand; with
probability 1 − pch (loyalty) she is not ready to change her
preferences.
8/29
Nina Golyandina
Models for evaluating advertisement efficiency
Part II. The model
It is clear that a loyal-to-B customer of Product B, who is not affected by
advertisement of A, will buy Product B.
Consider the case, when a customer of Product B is not loyal to B and is
not affected by advertisement.
Since we discuss a probabilistic model, we presume that this happens
with some probabilities (with pch a customer is not loyal and with
1 − padv she does not affected by advertisement).
It is natural to assume that the customer, who is not loyal to a specific
product/brand, will buy a random product from the given (sub)category;
that is, she ‘forgets’ the previous choice.
We assume that the market shares are not changed and therefore the
customer buys Product A with probability pA , Product B with probability
pB , and so on, where pA is the market share of A, pB is the market share
of B, ...
9/29
Nina Golyandina
Models for evaluating advertisement efficiency
Part II. The simple model
Let us show how to construct the models and how to use the results.
We start with the simple case without advertising.
‘Product A’ or ‘Product B’ means that the buyer purchases the
corresponding product/brand.
Product A
pA
.
prev
ts’
e
g
r
‘fo
p ch
pB
Product B 1 − p
loya
l
10/29
Product B
pC
ch
Product B
Nina Golyandina
Product C
Models for evaluating advertisement efficiency
Part II. The simple model
Let pB % be the share of buyers, who purchased Product B at the first
step.
Then, at the second step, the share of Product B is calculated as
P(B, B) + P(A, B):
pB · ((1 − pch ) + pch pB )
+
(1 − pB ) · pch pB = pB .
We see that the model describes the balanced market (purchases do not
change market shares).
A
A
pA
pA
B,
pB
ts’
rge
‘fo
p ch
1−
loy
al
pB
B
pc
h
1−
loy
al
B
11/29
A,
1 − pB
ts’
rge
‘fo
p ch
Nina Golyandina
pB
B
pc
h
A
Models for evaluating advertisement efficiency
Part II. Simple model
Thus, we consider a simple model with
the parameters pA , pB , ..., which reflect the market shares of
products and
the parameter pch , which reflects the share of customers that are
ready to switch between products/brands.
On the base of changes of choices, we can estimate the loyalty 1 − pch :
pch = (P(A, B) + P(B, A)/(2pA pB ), where P(A, B) and P(B, A) are the
probabilities of switches between different products.
Example:
Model: A
B
Data:
A
B
A
P(A, A) = 0.5
P(A, B) = 0.1
B
P(B, A) = 0.1
P(B, B) = 0.3
A
50 purchases of (A,A)
10 purchases of (A,B)
B
10 purchases of (B,A)
30 purchases of (B,B)
Here pA = 0.6, pB = 0.4, pch = (0.1 + 0.1)/(2 · 0.6 · 0.4) ≈ 0.42.
12/29
Nina Golyandina
Models for evaluating advertisement efficiency
Part II. The model with advertising, version 1
Let us include the advertising of Product A in the model. There are two
ways how to do it.
The first version: the advertising effect does not depend on loyalty.
Product B
t
ffec
ot a
n
s
oe
ad d − p adv
1
Product B
l
loya
p ch
1−
pch
‘for
gets
’ pr
ev.
Product A
pA
pB
pad
Product B
v
ad a
ffec
ts
Product
A
pC
Product C
13/29
Nina Golyandina
Models for evaluating advertisement efficiency
Part II. The model with advertising, version 2
The advertising of Product A.
The second version: the advertising affects only non-loyal customers.
Product A
’
gets
‘for
p ch
Product B 1 − p
loya
l
ts
ffec
ad a
v
p ad
1−
pad
ad d
v
oes
not
affe
ct
Product A
pA
pB
Product B
ch
Product B
pC
Product C
14/29
Nina Golyandina
Models for evaluating advertisement efficiency
Part II. The models with advertising, summary
Parameters:
pA is the share of the advertised product, pB = 1 − pA ;
padv characterizes the strength of the advertising;
pch characterizes the probability that the customer is ready to
change his choice.
The first model: the advertising effect does not depend on loyalty.
Then the increase of buyers of Product A is equal to (1 − pA )padv .
The second model: the advertising affects only non-loyal customers.
Then the increase of buyers of Product A is equal to (1 − pA )pch padv .
15/29
Nina Golyandina
Models for evaluating advertisement efficiency
Part III
Part III. TV spot testing
16/29
Nina Golyandina
Models for evaluating advertisement efficiency
Part III. TV spot testing
We analysed the TV spot tests provided by P&G company.
A typical experiment of Copy Testing is directed to measure the efficiency
of an advertisement for a certain Product A. The corresponding data set
is given in the form of switching matrices (the numbers of respondents
switching their preferences from the product i to the product j).
As a measure of the TV spot effect, the expected increment of the
market share of Product A called TPM (trial potential measure) is used.
The first version of the model: TPM = (1 − pA )padv .
The second version of the model: TPM = pch (1 − pA )padv .
This means that TMP depends on different characteristics: the share of
other products (1 − pA ), the TV spot strength padv , and, maybe, pch .
17/29
Nina Golyandina
Models for evaluating advertisement efficiency
Part III. TV spot testing
An example of the switching matrix:
Post - choice Total P.01 P.02 P.03 P.04 P.05 P.06 P.07 P.08 P.09 P.10 P.11 P.12 P.13 P.14 P.15 P.16 P.17 other
AOK
10
9
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
1
0
Avon
6
0
4
1
0
0
0
0
0
0
0
0
0
0
0
0
0
1
0
Bebe
13
1
0
9
0
0
0
0
0
0
0
1
1
0
0
1
0
0
0
Ellen Betrix
37
0
0
0
23
3
1
0
2
0
0
1
0
1
0
1
1
2
2
Jade
17
0
1
1
0
10
0
0
0
0
0
1
2
0
0
0
0
1
1
Juvena
9
0
0
1
0
0
6
0
0
1
1
0
0
0
0
0
0
0
0
Kamill
4
0
0
0
0
0
0
3
0
0
0
0
0
1
0
0
0
0
0
Lancome
27
0
0
0
2
1
2
0
17
1
0
1
1
2
0
0
0
0
0
Marbert
8
0
0
0
0
0
0
0
0
3
0
2
0
1
0
2
0
0
0
Mouson
5
1
1
0
0
0
0
0
0
1
2
0
0
0
0
0
0
0
0
Nivea
39
0
1
1
0
0
1
1
1
0
1
28
1
0
0
3
0
0
1
Oil of Olaz
29
0
0
0
0
0
1
0
1
0
0
2
22
1
0
1
1
0
0
Plenitude
15
0
0
0
0
0
0
0
2
0
0
0
0
10
1
1
1
0
0
Pond's
2
0
0
0
0
0
0
0
0
1
0
0
0
0
1
0
0
0
0
Quenty
22
0
0
0
0
0
0
0
2
1
0
4
1
0
1
10
3
0
0
Vichy
21
0
0
0
0
0
0
0
1
0
1
0
0
0
0
1
18
0
0
Yves Rocher
11
0
0
0
1
0
0
0
1
0
0
0
0
0
1
0
0
7
1
other/not any brand
14 no answer
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
14
Total
289
11
7
13
26
14
11
4
27
8
5
40
28
16
4
20
24
12
Before: 26 consumers.
After: 37 consumers.
TPM = (37 − 26)/289 = 0.038, the expected increment is 3.8%.
The estimated parameters: pA = 0.1, padv = TPM/(1 − pA ) = 0.042,
that is, 4.2% of all customers will be affected by the advertising
pch = 0.35, that is, 65% are loyal to the chosen product/brand.
18/29
Nina Golyandina
Models for evaluating advertisement efficiency
19
Part III. TV spot testing
The expected growth of buyers of Product A is TPM = 3.8%.
Model 1 (advertising affects all buyers):
strength of TV spot = 4.2%
Model 2 (advertising affects only non-loyal buyers):
strength of TV spot = 12%.
Probably, in the real-life conditions, the model 2 is proper.
Then:
For categories with higher loyalty, the advertising should be much more
stronger.
19/29
Nina Golyandina
Models for evaluating advertisement efficiency
Part III. TV spot testing
Products are labelled by category and subcategory. Loyalty depends on
subcategories.
PP_DP — Category ‘Paper’, Subcategory ‘Diapers’ —
a stronger loyalty, since pch ≈ 0.1
PP_TP — Category ‘Paper’, Subcategory ‘Toilet Paper’ —
a weaker loyalty, since pch ≈ 0.3
20/29
Nina Golyandina
Models for evaluating advertisement efficiency
Part III. TV spot testing
The use of the model in TV spot testing:
1
Understanding of relation between the advertisement strength and
the impact on the market shares
2
More precise statistical estimation of parameters.
It is important to distinguish between random switches and driven
switches.
The model helps to understand how many people is enough to
decide correctly if the TV spot is successful.
The accuracy depends on loyalty. The smaller is loyalty, the worse is
the accuracy.
Thus, for (sub)categories with large random switches, larger sample
sizes of buyers for ad testing should be taken.
21/29
Nina Golyandina
Models for evaluating advertisement efficiency
Part IV
Part IV. The model of switching and Dirichlet model
22/29
Nina Golyandina
Models for evaluating advertisement efficiency
Part IV. The model of switching and Dirichlet model
There is a connection between
the Dirichlet model of Buyer Behaviour with the parameter S and
the model with the probability of random switches pch .
It appears that for two products S = pch /(1 − pch ), pch = S/(S + 1).
Two citations:
‘Bound (2009): S parameter has in practice quite a simple intuitively
attractive meaning as a measure of brand purchasing diversity. It is
closely associated with the average number of brands bought.’
‘The S parameter has been used to calculate φ, a measure of
polarisation, which has been defined as a brand loyalty measure.
Jarvis, Rungie and Lockshin (2007) define it as φ = 1/(1 + S). In
that paper φ is used to compare the brand metrics of product
categories of repertoire and subscription type categories.’
Thus, in these terms, polarisation = loyalty: φ = 1 − pch .
23/29
Nina Golyandina
Models for evaluating advertisement efficiency
Part IV. The model of switching and Dirichlet model
If there are several brands, then the loyalty for each brand can be
calculated on the base of switches between the given brand and the other
brands (Li, Habel, Rungie (2009)).
(j)
Then we obtain different polarisations φj = 1 − pch as measures of
jth-brand loyalty.
The Dirichlet model: all φj are equal.
Deviation from the Dirichlet model: e.g., brands with higher market share
(within the same category) have larger loyalty.
Generally, we can consider φ (or 1 − pch ) as a weighted average of brand
loyalties as a measure of behavioural loyalty.
However, for some categories the loyalty excess for strong products
cannot be ignored. E.g., the credit cards market.
Thus, our simple model is connected to the known market models; in
particular, the parameter estimates are comparable between different
models.
24/29
Nina Golyandina
Models for evaluating advertisement efficiency
Part V
Part V. The model of switching and clicks. Discussion
25/29
Nina Golyandina
Models for evaluating advertisement efficiency
Part V. The model of switching and clicks
The previous model helped to predict the advertisement potential to
increase the market share.
How to construct the model for clicks
with the same parameters pch and padv ?
Let an advertisement of Product A be shown to the consumer, who uses
Product B.
Then the model may be:
ready
Product B
ects
ad aff
p adv
1−p
ange
to ch
p ch
ad do
es
1−p
Click
adv
not a
ffect
No click
ch
loyal
26/29
No click
Nina Golyandina
Models for evaluating advertisement efficiency
Part V. The model of switching and clicks
Let an advertisement of Product A be shown to the consumer, who uses
Product A.
Then the model may be:
ready
Product A
Click
ects
ad aff
p adv
1−p
ange
to ch
p ch
ad do
es
adv
not a
ffec
t
1−p
No click
ch
loyal
27/29
Click?
Nina Golyandina
Models for evaluating advertisement efficiency
Part V. The model of switching and clicks. Purchase
frequency
The scheme with clicks is working if the buyer needs the product at the
moment of the ad show.
It is connected to the purchase frequency.
In turn, the purchase frequency depends on the time frame.
- If, say, the product is daily used/purchased, then the probability that
the buyer needs the product pneed = 1.
- If, say, the product is quarterly used/purchased, then pneed = 1/90 in
the framework of one day and pneed = 1/3 in the framework of one
month.
In any case, the model for clicks has an additional parameter pneed , which
depends on the purchase frequency in the (sub)category.
28/29
Nina Golyandina
Models for evaluating advertisement efficiency
Part V. The model of switching and clicks. Discussion
Product A is advertised:
Click
B
pB
p adv
1−
p
p ch
1−
adv
No click
pch
No click
A
pA
p adv
1−
p
p ch
1−
Click
adv
pch
No click
Click?
Thus, in our simple model, the average number of clicks is
N · pneed · (padv · pch + pA · (1 − pch )), where N is the number of shows.
We obtain the natural conclusion: the loyalty is the driver for clicks for
people who use the advertised product, while for those who are ready to
change their choice, clicks are based on the advertisement strength.
Here pA is the share of Product A within the people that look at the
advertisement.
Definitely, to click does not mean to buy. E.g. the probability
pA · (1 − pch ) corresponds to clicks, which probably do not lead to the
increase of the market share.
Certainly, different models should be considered.
29/29
Nina Golyandina
Models for evaluating advertisement efficiency