Competition and Crowd-out for Brand Keywords in
Sponsored Search∗
Andrey Simonov
University of Chicago (Booth)
Chris Nosko
University of Chicago (Booth)
Justin M. Rao
Microsoft Research
September 30, 2015
Abstract
On search keywords that include trademarked terms, the brand owner (focal brand)
and other relevant firms compete for consumers. For the focal brand, paid clicks have
a direct substitute in the organic link that appears directly below the advertisements.
This crowd-out can create a large wedge between the real causal effect and commonlyreported “nominal measures,” such as click-through rates. We quantify this wedge
using a large-scale, fully randomized experiment on Bing, which allows us to study
2,500 brands with a sufficient sample size. We find a positive, statistically significant
impact of brand ads, with a modest size, on the order of 1–4%. There is significant
heterogeneity, with more well-known brands showing a smaller causal effect. Competitors greatly impact the market. First, if the focal brand is not present in the top slot,
competitors “steal” 10-20% of clicks on average. Second, when the focal brand is in
the top slot, competitors steal only a modest fraction of clicks, 1-5%, but dramatically
shift the focal brand’s traffic from the organic link to the paid link, raising the focal
brand’s costs. Taken together, the results indicate the primary value of branded paid
links is defensive in nature.
∗
We thank Christian Perez, Matthew Goldman and Giorgos Zervas for helpful comments. Most of this
work was completed when Simonov was an intern at Microsoft Research. All opinions represent our own and
not those of our employers.
1
Introduction
In sponsored search advertising, brand keywords, queries that include a company’s trademarked name, are a substantial portion of advertising expenditure. At first glance this
practice seems sensible: Consumers learn about products through a variety of channels
and seek them out online using a search engine. Entering a specific brand signals product
awareness. Click-through rates (CTR) tend to be higher than similar non-brand keywords.
Further, since bidding on another firm’s trademarked term is legal, competing firms can step
in and siphon off traffic from the focal brand (as in theoretical model of Sayedi et al. (2014)).
The focal brand might want to bid aggressively to minimize “traffic stealing.” On the other
hand, the focal brand also occupies the first organic result, which is shown just below the
paid link(s), creating the possibility that the paid link simply crowds-out free clicks. Indeed
a recent paper, Blake, Nosko and Tadelis (2015), find almost complete crowd-out for a single, well-known brand, eBay. Using a controlled experiment, they document that when eBay
stopped bidding on its own keywords 99.5% of traffic was retained via the organic link.
We are thus left with a bit of a mystery. Advertising on brand keywords is substantial, yet
the presence of an almost complete substitute (the organic link) suggests that it is practically
worthless for the focal brand. Moreover, there may be no need to continue to advertise to
users who intentionally seek out a particular brand – these are folks that will find their way to
your product regardless of search advertising. Competing explanations of why firms continue
to bid on brand keywords have fundamentally different implications for the marketplace and
our understanding of firm behavior. The first class of explanations posits that advertisers
focus on “nominal measures” of ad performance, such as CTR, rather than the causal impact
on customer visits, which leads to an irrational interest in keywords that appear attractive
through this lens. The second class of explanations points to eBay as being unrepresentative
in some way. It is the second largest e-retailer with a very established brand and thus may
be immune to competitive effects among customers using “eBay” in a search query. Under
this explanation, other firms very well could be making rational choices.
In this paper we arbitrate between these explanations with a large-scale, fully randomized
experiment run on Bing. On both Bing and Google, the maximum allowable number of ads
placed above the organic search results is 4. In the experiment this cap was exogenously
reduced to 0, 1, 2 or 3. For example, if a query had a single ad that cleared the auction
1
reserve price, comparing conditions “Cap 0” (0 ads allowed) and “Cap 1” (only 1 ad allowed)
effectively replicates the eBay experiment. We have such experiments for tens of thousands
of brands, but focus our analysis on the top 2,500 brands, which account for 98.7% of brand
searches. The included firms have widely varying brand quality metrics.
We start by measuring the extent to which organic search results serve as substitutes
for paid search links when no competing ads are present (matching the setting of Blake et
al.). We find that the results in Blake et al. are a bit of an outlier. In our sample of firms,
sponsored links on branded search queries drives significant incremental traffic—total clicks
to a firm’s website increases by 2-3 percentage points on average. This effect is significantly
larger for lesser known brands, while the strongest brands in our sample show effects closest
to that of eBay. But while incremental traffic is relatively modest, “crowd-out” of free clicks
on the brand’s link in the top organic slot tends to be quite large.1 When we experimentally
introduce the brand ad about half the traffic, or 40 percentage points of CTR, is shifted
from the organic link to the paid ad. Since crowd-out exceeds the causal effect by more than
a factor of ten, the nominal “cost per click” (CPC), the standard pricing metric reported
by online advertising platforms, strongly diverges from the real “cost per incremental click”
(CPIC).
For the case where brands do not face competing ads we measure the wedge between
CPIC and CPC for a broad cross-section of firms. For firms that show a significant ad effect,
CPIC is on average 11 times larger than the CPC. For all firms, we establish a lower bound
ratio of 22.2 While these multipliers might seem too high to rationalize, we must keep in mind
that the focal brand always pays a much lower CPC than competitors because the pricing
rule of the generalized second price (GSP) auction rewards link relevancy and clickability. To
put our CPIC estimates in context we compare them to two quantities that should capture
the real costs of customer acquisition on the platform: 1) the CPC focal brands pay on
keywords when they do not occupy a high organic position 2) the CPC competitors pay on
brand keywords. CPIC estimates exceed these comparison measures in the vast majority of
cases.3
1
All the brands in our sample always occupied the top slot.
We cannot reliably compute CPIC at the firm level when the estimated causal effect is close to zero.
As such, we rely on the convexity of the underlying function and use Jensen’s equality to produce a lower
bound by estimating the function at the average value of the arguments.
3
We cannot observe the potential differences in the value of an ad click on different search queries or click
type, nor the overall value of a click (given bid shading in the GSP Gomes and Sweeney (2014)), so these
comparisons are by no means perfect. We note that more than 90% of firms have the same “landing page”
2
2
We have thus far focused on the case of focal brands not facing aggressively bidding rival
firms. If competitors are present—a situation commonly observed in our data—the focal
brand can choose to 1) either “defend” and bid aggressively to occupy the top slot or 2)
not bid and allow competitors to fill the top of the page, pushing the brand’s organic link
further down.4 In the first case, the focal brand’s ad in the top slot provides a largely effective
defense—competitors are able to steal only 1-5% of clicks, which depends on the number of
competitors present and brand attributes. It turns out that in this case competing ads have
a much larger impact on crowd-out rates. When facing no competitors (Cap 1 condition),
about 50-60% of clicks to the focal brand’s website go through the paid link and the remaining
half or so are free. When we randomly add in competitors, the fraction of paid clicks increases
by 10% for the first competitor added, 9% for the second and 5% for the third, reaching 84%
for a full slate. Thus increased competitors both lower the effectiveness (relatively small
effect) and raise the costs (relatively large effect) of the focal brand’s defensive position.
Now we look at the case where the focal brand does not advertise but competitors do.
These firms come in two varieties. The first, which we’ll call “reseller” types, are firms
that produce a product and sell it themselves, but allow other licensed retailers to sell it
too (e.g. laptop manufacturers). These firms typically get only about 30% of total clicks
on brand searches. The second variety is the exclusive seller, such as a travel website or
car insurance company. These firms typically get more than 80% of total clicks. Using
the experimental variation to forcibly remove competitors’ ads reveals that, in both cases,
competing advertisers get 20% of total clicks—far higher than the 1-5% we observed earlier
when the focal brand was present in the top slot. For reseller types, only about one-third
of this 20% comes from substitution from the focal brand’s links, with the rest coming from
clicks that would have gone to other firms present in the organic links. In contrast, for
exclusive types, nearly all the competing ad clicks come from the focal brand. Based on
observed click metrics, exclusive types are a much better match to the brands that choose
to advertise, indicating that competitors pose a real threat to traffic acquisition if the brand
is not present.
Our results indicate that the returns to advertising for the focal brand hinge on presence of
competitors. In the absence of competitors, ads have a positive and statistically significant
(where a user goes after clicking on both.)
4
It is exceedingly rare that a brand bids and does not occupy the top slot. Conditional on bidding, they
bid to win.
3
effect, but associated crowd-out produces CPIC/CPC ratios (which give a multiplier on
nominal costs to get real costs) that are suspiciously high: 11 for firms with a significant ad
effect and 22 overall. Computing CPIC when both the brand and competitors are present is
more difficult because it requires us to estimate a counterfactual we cannot directly observe at
the firm level. Nonetheless, using average estimates from other firms indicates that exclusive
type firms would lose a substantial fraction of traffic—roughly 25% of total clicks—if they
discontinued their ad, while the impact on reseller type firms is more muted.5 This translates
to CPIC/CPC ratios of 3–5 for exclusive types, with the exact figure depending on the
number of competitors present and brand attributes. Given the relatively low CPC paid by
focal brands, these multipliers generally place real costs below our comparison measures and
thus are rationalizable.
How does firm behavior align with our findings? We saw that without the defensive value
induced by competition, the real costs of brand ad clicks are quite high and the overall returns
are, at the very least, questionable. In this case, a slight majority, 54%, of firms choose not
to advertise and only a small fraction of the firms that do advertise show ad effects and
crowd-out rates that rationalize their choices. The remaining firms appear to excessively
focus on average rather than marginal costs, as has been previously been documented in
other contexts (de Bartolome, 1995; Ito, 2014), or suffer from an incentives failure of some
sort. When competitors are present, the real-to-nominal cost ratios are generally easy to
rationalize by the data, and here we observe that the vast majority exclusive-type brands
choose to advertise. Finally, advertising on a rival’s keywords produces a double-dividend: it
necessitates a relatively costly defense, which increases with the number of competitors, and
generates a modest amount of stolen clicks. Consistent with these observations, we observe
the practice is widespread. Taken together, the majority of firms exhibit behavior that is
broadly consistent with our findings, but a non-negligible fraction make choices that do not
appear to be supported by the data.
5
The returns to these firms is more complex, since “competing firms” are advertising the product the
focal brand produce, although they almost always sell other brands on their website.
4
2
Background and Context
We define brand keywords as queries that consist of a trademarked term and where the
trademark holder occupies the top organic slot. Competitors using a trademarked term to
guide their bidding is a contentious practice as the focal brand dislikes the fact that their
competitors can target a user that has expressed a particular interest in them. Indeed the
brand firm may use other forms of advertising to generate searches for their products, as
shown in Lewis and Nguyen (2014), and not want this traffic prone to competition. Although
there have been trademark infringement lawsuits on the matter courts have consistently
upheld the practice. The use of trademarked terms within a competitors ad is not allowed6 ,
an exception was granted in 2009 to licensed resellers, since a reasonable consumer would
not be confused by the practice (the good being sold be the reseller is genuine). Chiou and
Tucker (2012) study this change and find that this change did not damage the focal brand
because it made the competing resellers less distinct.
In empirical studies of brand search, early experimental evidence comes from Reiley
et al. (2010), which showed that organic links and ads are substitutes for each other. This
substitution pattern is overwhelmingly present in our experimental data as well, and has also
been found in structural work (Jeziorski and Segal, 2014).7 Reiley et al. (2010) further show
that more ads can increase total CTR for the ad placed in the top slot because organic links
act as slightly better substitutes for ads in their study. As discussed in the introduction,
Blake et al. (2015) ran a large experiment with eBay and the results of the experiments
led the firm to discontinue brand search ads. In the experimental period eBay did not face
competing ads.
The final relevant strand of literature involves user behavior in selecting options on the
search results page and how this relates to bidding incentives. The first set of papers in this
literature help explain why the position of a competitor’s ad on the page causally influences
a user. Granka et al. (2004) presents eye-tracking evidence that most users use a “top-down
cascade” approach to searching the page. Craswell et al. (2008) takes this result to the
field and shows that “position effects” can be large near the top of the page—the same link
6
For example, while travel website Expedia is free to bid on “priceline,” it cannot include “priceline” in
their ad text.
7
One paper, Yang and Ghose (2010), sharply diverges from this result. Using a structural model, they
assert that there is a positive interdependence between organic ranking and search click-through rate.
5
gets substantially more clicks based on position alone. Later work has shown that search
is more complex than simple top-down traversals (Dupret and Piwowarski, 2008), but still
supports the finding that being pushed further down the page tends to lower CTR. In a
(mostly) theory paper, Desai et al. (2014) model interactions between competitors with
brand keywords. They discuss the prisoner’s dilemma nature of branded keyword bidding.
Jerath et al. (2011) present a related theoretical model where there are low and high quality
firms and discuss why inferior firms have more incentive to locate at the top of the page
because otherwise no one will find them. In contrast, a superior firm may get more clicks
despite being lower down the page. In our setting this could correspond to the brand’s
organic link still getting the majority of clicks even if there are rival ads above it. One could
broadly read our empirical evidence as consistent with many of these theoretical predictions.8
2.1
Experiment and Data Description
The data in our study come from a fully randomized experiment on the Bing search engine.
On Bing, the sponsored listings that appear at the top of the page, above the organic listings,
are known as the “mainline.” A maximum of four mainline ads could be shown on a given
query—the same practice employed by Google. Absent an experiment, the number of ads
and their composition is endogenously determined by firms’ bids. A cross sectional regression
that looked at differences in the number of advertisements by keyword would conflate true
effectiveness of advertisements with differing environments across keywords. We use our
experimental variation to control for these confounding factors.
The experiment was conducted on a small fraction of U.S.-located users over nine days
in January of 2014. Users’ searches were randomized into one of five conditions. Four of
these conditions had some treatment, and corresponded to limiting the maximum number
of mainline ads at 0, 1, 2, and 3. The fifth condition was a control group, and corresponded
to the maximum of 4 mainline ads, the typical production setting.9 The treatment limited
the number of ads that could be shown, but often this cap did not bind. For instance, in
the treatment group that limited mainline ads to a maximum of 3 (“cap 3” to employ the
8
We do not, however, observe brands occupying the lower adverting slots. The context of brand search is
somewhat different than generic product search, so we would not expect all the predictions to be borne out.
9
The reason “all the rest” of the traffic is not the control group is that other experiments are running
concurrently, so a “clean control” is preferred.
6
terminology we will use throughout), if there were not enough bidders that met the reserve
price to fill the 3 slots, then fewer than 3 ads were shown. We carefully control for this
issue by selecting only queries that matched into bidding data where an ad would have been
shown in the absence of the experiment. See the Appendix A for more detail on this process.
To identify brands, we extracted 87,000 retailer and brand names from the Open Directory Project.10 A search is characterized as a brand query if and only if (1) the query
is in this list, meaning it is a verified firm brand, and (2) the query matches the domain
name in the first organic position. We focus only on brands that are in the first organic
link because this selects true brand keywords. Queries that generate brands that are not in
the first organic position might be searches of a different nature, perhaps not meant to get
directly to the brand page, but to a broader set of sites.
Figure 1: Brand search example
This example has two mainline ads: own brand ad in mainline 1 and competitor’s ad in mainline
2.
10
dmoz.org, the project uses volunteer annotators to “classify the web.”
7
Figure 1 provides an example of a brand query. Queries are simplified using standard
techniques, e.g. we treat “Macy’s,” “macys.com,” “macys,” and “macy’s” as the same query.
We focus on searches with 0 or 1 clicks on the page, ignoring rare instances of 2 or more
clicks.11 The majority of brands have very few exposures in the dataset. Table 1 presents
the number of brands binned by the number of observations for those brands. 64.7% of all
brands in the control group have less than 10 exposures but represent only 0.19% of all traffic,
whereas 96% of traffic comes from the 1045 brands that have 1000 or more exposures. The
starting point for our sample selection will be the 2517 companies with over 350 exposures.12
These firms offer us relatively precise estimates and cover 98.7% of market activity.
Number of
exposures
in Control
1
2
3
4 - 10
11 - 100
101 - 1000
> 1000
Total
Table 1: Ad coverage in the control condition
Number of Percentage Percentage Percentage
Percentage
brands
of brands
of traffic
of own ads of competitor’s ads
in Control
(%)
(%)
in ML1 (%)
in ML1 (%)
4869
23.1
0.02
3
30.6
2773
13.1
0.02
4.1
32.4
1686
8
0.02
6.3
30.8
4315
20.5
0.12
10.2
34.5
4200
19.9
0.64
19.8
34.6
2202
10.4
3.6
42.64
28.5
1045
5
95.6
43.8
13.6
21090
100
100
14.4
31.4
Percentage of ads is computed across companies. For example, companies with 4 exposures and
companies with 10 exposures are given the same weight in group 4-10. Total frequency is also
computed across companies, unweighted.
In our working sample of firms, 32.7% of companies advertise on their own brand keywords
more than 90% of the time. 39% of brands do not advertise on their own keyword at all.
The remaining brands advertise selectively, turning brand advertising on and off, or (more
likely) advertising in some geographic regions but not others. More detailed information is
given in the Appendix A, Figure 12.
11
In these occurrences, the searcher often visits all advertisers, making it less interesting to study. Further,
search engines often refund clicks from such patterns.
12
With this selection rule we are balancing the number of firms against the inclusion of brands that don’t
provide meaningful information because they are so small. We have done substantial robustness around this
threshold and the results are not materially impacted.
8
3
Results: Causal Effect of Brand Advertisements
In this section we measure how effective advertising is in driving traffic to the focal brand’s
website.
3.1
Ad Effect without Competitors Present
We begin by looking at firms that chose to advertise on their own keyword over 90% of the
time during our sample period. 32.7% of the 2,517 high-exposure firms are in this set.13
For 42.7% of this set (352) a competitor occupied the second paid slot less than 20% of the
time. This represents the most extreme form of potential substitution between paid and
organic search links because the link directly below the mainline advertisement goes to the
brand’s website with limited-to-no competition to stand in the way of the firm getting the
click. To estimate ad effectiveness for this case, we compare the cap 1 treatment group for
cases when the top ad went to the brand website to the cap 0 treatment condition where no
advertisements were shown for the same set of queries because the experiment forcibly (and
randomly) removed all paid search links.
Our primary outcome of interest is the probability that an individual arrives at the
website of the searched brand. We estimate this probability using the server logs that
separately record clicks on a brand’s organic link and those on the paid link. If a user does
not click one of these links it is either because she chose to visit a competitor or did not
click any link. Figure 2 plots the probability that a user arrived at a brand’s website by
the cap 0 vs. cap 1 groups. The figure shows that advertising on one’s one keyword drives
an incremental 2.27% of traffic to a brand’s website. This average effect is overwhelmingly
statistically significant, but we note that the y-axis is “zoomed in” and the overall magnitude
of the effect is perhaps economically small.
Next, we look at whether the ad effect varies by observable features in the data, focusing on a sub-sample of 493 companies with a sufficient amount of traffic to get reliable
company-specific estimates of the probability to get a click (the results are not sensitive to
13
If we focus on companies that advertise on their own keyword at least once, these companies correspond
to 53.6% of companies and 19.6% of traffic
9
Figure 2: The Effect of Advertising on One’s Own Keyword
this choice).14 The first step is to check if there is more variance in our estimates than would
be expected from sampling variation alone. Figure 3 presents an estimate of the density
across firms. In this sub-sample, the average effect is 0.021 (this is consistent with a smaller
effect for larger firms), and the standard deviation is 0.048. We plot a fitted normal density
over the distribution of effects, which shows the empirical distribution has heavier tails. We
formally test and readily reject the hypothesis that there is no heterogeneity.15
Having verified there is real heterogeneity, what is the source? In our comparison of
experimental conditions cap 0 vs. cap 1, we use all firms with a sufficient sample size in the
top position—this includes both those firms that would have faced competing ads in slots 2–4
and those that are typically alone on the page. The presence of competitors is presumably
endogenous to many unobservable factors such as the brand capital (as in Simon and Sullivan
(1993)) of the focal firm. A natural hypothesis is that “stronger” brands are less susceptible
to competition and thus competing bidders is a signal of a weaker brand. If weaker brands
benefit more from advertising, then we expect a larger effect for those brands that face
competing bidders in the normal state of affairs.
14
We keep companies with more than 80 exposures in each condition.
We perform a series of standard tests for normality, including Shapiro-Wilk, Jarque-Bera, DÁgostino
and other tests. All of them reject normality of the distribution.
15
10
Figure 3: There is heterogeneity in the effect of own ad in mainline 1
Table 2 divides the sample into groups depending on the probability (across queries and
time) that a competitor would have occupied the 2nd slot in the absence of the experiment.
For firms whose competitors never clear the reserve price on their keyword, the incremental
effect is 1.4% on average, whereas for firms who almost always face competition, the effect
is 4.9% on average. These differences are significant well beyond conventional levels and the
results are also confirmed in a regression framework given in Appendix B, Table 10. We can
also cut the data by the number of other competitive bidders for a given query, as given in
Table 9 in Appendix B. The table shows shows that greater competition, in terms of bidders
that clear the reserve price, is associated with a large experimental estimate.
We now examine if the effect size varies with a firm’s “brand capital.” Blake et al. showed
that eBay’s brand ads had almost no impact on website traffic. As the authors note, eBay
is a very strong brand—it is the second largest e-retailer. Our broader sample allows us to
look at a wide range of firms, from those that are quite similar to eBay to those that the vast
majority of consumers would not recognize by name. We proxy for brand capital with data
collected from Alexa.com, a widely used website ranking service. In particular, we collect
US and global rankings, “bounce rate” (the probability of a visit less than 30 seconds), the
fraction of traffic from search (well-known firms get more direct navigation via the URL
window), pages viewed per day and time spent per day. Table 8 (Appendix B) provides a
11
Table 2: Effect is higher for brands with more competitors on their keywords
Total
Competitors’ ads frequency in mainline 2
0-0.1 0.1 - 0.25 0.25 - 0.75 0.75 - 0.9 0.9 - 1
Ncomp
493
192
99
125
34
43
pnoad
pad
pad − pnoad
0.779
(0.001)
0.800
(0.001)
0.811
(0.002)
0.825
(0.001)
0.790
(0.003)
0.808
(0.002)
0.756
(0.003)
0.776
(0.002)
0.739
(0.005)
0.783
(0.004)
0.708
(0.005)
0.757
(0.003)
0.021
(0.002)
0.014
(0.002)
0.018
(0.003)
0.020
(0.004)
0.044
(0.007)
0.049
(0.006)
where
Ncomp - number of companies
pnoad - probability to get to brand’s website when there are no ads
pad - probability to get to brand’s website when there is own ad in mainline 1 and no competitor’s
ads
summary of brand capital measures.
Table 3 presents the results of four regressions of own brand ad effect on brand capital
measures. Specification (1) shows that companies with higher rankings tend to have a lower
incremental effect of one’s own ad in mainline 1. On average, the effect of an own brand ad
for a very well-known company (log(US) ranking around 2.5) is 3 percentage points lower
than for an unknown brand (log(US) ranking around 11). As we add more controls for our
brand capital measure in specification (2), the coefficient on log ranking is still significant. In
specification (3) we add controls for how much space the brand links have on the webpage:
the number of “deep links” (sub links below the main link), the number of “detail cards”
(detailed informational panels including things like maps and ratings), and the number of
organic links of one’s own brand on the first page.16 The coefficient falls by roughly 30% and
is no longer statistically significant. This drop combined with the negative coefficients on
the features that indicate richer and larger content for the focal brand indicate that some of
the effect was coming through stronger brands getting better visual representations. Finally,
in specification (4) we add in the frequency of times competing bidders occupy slot 2. The
coefficients do not change much and we can use this specification to confirm the larger effect
when the ad typically faces competition.
16
For a description and examples of deeplinks and dcard please see Appendix C
12
Table 3: Companies with higher brand capital have lower own ad effect
Dependent variable:
Effect of own ad in Position 1
(1)
(2)
0.004∗∗
(0.001)
0.009∗
(0.005)
0.006
(0.005)
−0.002∗
(0.001)
−0.002∗∗∗
(0.001)
−0.001
(0.002)
Other brand capital controls
−0.008
(0.012)
No
0.024
(0.020)
Yes
0.062∗∗∗
(0.024)
Yes
0.006
(0.005)
−0.002
(0.001)
−0.001∗∗
(0.001)
−0.0002
(0.002)
0.017∗∗
(0.008)
0.052∗∗
(0.024)
Yes
Observations
R2
Adjusted R2
492
0.013
0.011
492
0.038
0.026
492
0.079
0.062
492
0.089
0.070
Log US Website Traffic
(3)
Deep links
Details card
# brand organic links
(4)
Freq. competitor in Slot 2
Constant
∗
Note:
13
p<0.1;
∗∗
p<0.05;
∗∗∗
p<0.01
We conclude that brands with higher brand capital gain less from occupying the top
adverting slot. This appears to be both a combination of consumers being less likely to
substitute away from going to a firm’s website when that firm has higher brand capital and
because stronger brands get more visually appealing organic search results. Regardless of
the precise causal channel the implication for managers remains the same—firms with higher
brand capital benefit less from bidding on their own brand keywords.
3.2
Ad Effect with Competitors Present
First, we look at situations where competitors are the only bidder that clears the reserve
price. In this case, competing links are above the focal brand’s first link (the organic result)
on the search results page. This is clearly a selected sample because the brand’s decision
not to advertise is a choice. In particular firms that choose to not “protect” their own brand
keywords are presumably different than those that do.17 However, for firms that chose not
to advertise on their own keyword the experiment gives us clean variation to estimate the
causal competitive effect. We start by comparing cap 1 to cap 0, where the competing ad
has been removed at random.
In order to mitigate selection effects we break up the set of firms that do not advertise on
their own keywords but face competition into two sets. To do so we use the probability, in
the absence of any advertisements (a case created by the cap 0 condition in our experiment),
that consumers click on the firm’s organic unpaid link. The case where consumers click
relatively infrequently on the brand’s organic link indicates that many firms sell a genuine
version of the intended good. For example, one can buy a Dell laptop directly from Dell or
through many e-retailers. We will call this the “reseller” case. In contrast, consider a search
for a travel booking tool like Expedia.com. For these types of searches there is only one
“genuine version” of the good and the branded firm garners a high fraction of clicks. We
thus refer to such cases as the “exclusive” type.
We assign types using the probability of arrival distribution, which is given in Figure 4
for the 49 firms that do not bid on their own keywords and have a competitor in mainline
1 above 90% of the time in the control group for our experiment. The rates here are much
17
Recall that one of the downsides of the experiment is that we were not able to place ads in situations
where they wouldn’t have existed otherwise – we were only able to subtract them.
14
Figure 4: Heterogeneity in firms that do not bid on their own brand keywords
lower as compared to the average arrival rate of about 80% observed in the last section.
There is also substantially more more heterogeneity, with a large mass of firms below 0.30
(Figure 14 in Appendix E shows a similar histogram for 493 companies which advertise on
their keyword more than 90% of the time). We divide the firms with the two vertical lines
shown at arrival rates of 50% and 70%. Based on click behavior, the exclusive types form a
more suitable comparison group for the firms that typically advertise on their keywords.
Figure 5 graphs the difference in the number of clicks on the brand’s own organic link and
the competitor’s ads across the cap 0 and cap 1 conditions. The left-panel gives the reseller
type and the right panel the exclusive type. When we experimentally add the competitor’s
advertisement two effects occur. First, the percentage of traffic to the brand’s website drops.
The drop is 8 percentage points for the reseller case and 17 percentage points for the exclusive
case. These drops represent “traffic stealing.” Second, the competitor gains a large number
of clicks—around 20% in both cases. This means that for the exclusive case, almost all the
clicks garnered by the competing ad come at the expense of the focal brand, whereas for the
reseller case less than half of the clicks came from the focal brand, with the rest coming from
other resellers with links further down the page.
We now move to the case where the focal firm wins the auction and competitors occupy
15
Figure 5: The incremental effect of bidding on one’s competitors’ keywords for “reseller
type” (left panel) and “exclusive type” (right panel) firms.
slots 2, 3, and 4. We narrow our sample down to firms where, in the absence of the experiment, 4 mainline ads would be shown.18 This holds the set of firms constant and the
experimental conditions systematically remove advertisements. Figure 6 displays the causal
effect of occupying each of the mainline slots. Starting with the left panel, the first point
displays the total traffic that goes through both the organic and paid link to the brand’s
website when only the brand ad is present. The second through fourth points display the
traffic to the brand’s website adding in competitors into the mainline slots 2 through 4,
respectively. The relationship is downward sloping, as expected and the full slate of competitive ads reduces the traffic to a brand’s own website by 4.3 percentage points on average.
In the right panel we separate advertisers by a median split of website traffic. The lower line
represents below-median firms. We can see the smaller firms have a lower baseline CTR and
a slightly greater loss of clicks when facing 3 or 4 competitors.
Figure 7 breaks down the stolen traffic by competitor position. Two points are worth
noting. First, the amount of traffic garnered by the second and third place bidders is not
18
Again, we deal with a bit of a selected sample – keywords where both the brand and competitors are
already bidding. We have already shown these firms tend to be weaker brands, for instance. However these
are presumably the firms for which these estimates matter the most.
16
Figure 6: The effect of competitive ads in mainline slots 2-4
(a) Average effect
(b) Upper and Lower 50% by Brand Capital (US
Ranking)
significantly affected by the number of additional ads below them, though there is slight
evidence that the second ad prefers to face more ads below it, consistent with Reiley et al.
(2010). Second, the higher placed bidders secure more clicks. This is a natural consequence
of the auction logic and should not be interpreted as a causal effect of position, though some
of the effect is likely due “position effects,” Goldman and Rao (2015).
We started by examining the impact of the focal brand ad in isolation and observed a
statistically significant effect of about 2.5 percentage points. This effect tended to be larger
for weaker or more susceptible brands. Within the sample of firms that we observe that do
not bid on their own keyword and have a competitor in mainline 1, this competitor receives
around 20% of clicks for those keywords.
Taken together, our estimates provide some justification for the notion that firms should
bid on their own keyword in order to protect their brand keywords from competitors. In the
absence of a focal brand, ad clicks to the brand’s website decrease by an average of between
8 and 17 percentage points depending on the type of firm. We showed that such “defensive
positions” are relatively effective. When the focal brand is present, competitors in slots 2–4
17
Figure 7: The effect of competitive ads in mainline slots 2-4
steal far fewer clicks, around 4 percentage points, or one-fourth of the effect for the exclusive
types (the appropriate comparison group) when the brand ad is not present. This evidence
indicates that the marginal effect of the brand ad is quite high in this case—on the order
of 15 percentage points. In contrast when competitors are not present the effect is around
2 percentage points. This comparison suggests the primary value of a focal brand’s ad is
defensive.
4
Results: Crowd-out and Cost Per Incremental Click
Up until now we have combined paid and organic clicks together in order to measure the
causal impact of the ads. Of course the firm is not agnostic to the source of traffic: If the
traffic goes through the paid link, they must pay the search engine for each click; if it goes
through the organic link, it is free. When crowd out occurs, for every marginal click there
are inframarginal clicks that the firm must pay for as well. In this section we define “cost
per incremental click,” which accounts for the real cost of acquiring traffic.
18
4.1
Crowd-out and CPIC without Competitors Present
Figure 8 shows the percentage of traffic going to one’s own brand website through the paid
link in mainline 1 for the sample of focal firms consistently advertising on their keywords.
On average, 46.3% of clicks go through the paid link and most of the firms fall between 30%
and 70%.
Figure 8: Histogram of the fractions of total clicks going through paid links
Figure 9 visually shows how many of these clicks are “cannabilizing” organic clicks by
using the experimental variation induced by the cap 0 vs. cap 1 conditions. The difference
in the overall height of the bars represents incremental clicks to a brand’s website, i.e. the
average effect of 2.27% we previously documented. The drop in the organic segment (pink)
shows the degree to which the paid link cannibalizes organic clicks—roughly half of the
traffic now goes through the paid link. Putting both figures together, on average for every
incremental click the firm pays for 19 inframarginal ones. It is important to note, however,
that this calculation implicitly sets the counterfactual of no competetive ads. Since our
interest is to compute an accurate costs per incremental click at the firm level, we need to
choose the appropriate counter-factual: what would happen if the firm did not advertise.
19
For firms that typically do not face competing ads, the simple Cap 0 to Cap 1 comparison
is valid and we use this as a starting point.
Figure 9: Organic click cannibalization compared to incremental clicks
We compute CPIC as:
CP IC =
xp1
CP C
(p1 − p0 )
where x is the fraction of clicks going through an ad, p1 is the probability of getting to
one’s own website with the own brand ad in mainline 1, and p0 is the probability of getting
to one’s own website without the own brand ad in mainline 1. The CPIC/CPC ratio is
a natural measure of crowdout—it gives how many clicks a firm has to pay for to garner
one incremental click. However, a challenge in computing this ratio is that CPIC is only
a coherent object if there is a positive effect of own brand ad (otherwise it is infinite or
negative). In our sample, the estimate is significantly positive for 16% of companies. We
start by computing CPIC for these 43 firms. The median is 9.1, the mean is 11.07 and the
distribution is shown in Figure 10.
To estimate this ratio for all firms we rely on the assumption of a non-negative ad effect
and the convexity of the relationship between CPIC and effect size. This convexity is due
20
to effect size showing up in the denominator of CPIC. Using Jensen’s inequality, evaluating
the CPIC function at the expected value of it’s arguments will be a lower bound for the
expectation of the function.19 For these firms, the average average effect of an own brand
advertisement is 1.6,%20 and the average amount of traffic going to the own brand website
through the paid link is 36%. The resulting estimate of the CPIC to CPC ratio is 22.5.
Figure 10: Histogram of CPIC/CPC ratio for the estimates for the 43 firms that do not
typically face competing ads and show a significantly positive ad effect.
Table 4 presents a detailed summary of the costs paid per click by the focal brand and
competing firms. The focal brand’s nominal CPC is low, between $0.06–0.15 depending on
the sample and the weighting. But given the amount of traffic that goes through these links,
the amount of money involved is far from trivial. Competitors typically pay many multiples
of the brand’s price, between $0.47–0.86 on average, despite occupying lower positions. This
disparity is a function of the pricing rule of the GSP auction—more relevant and clickable ads
get a lower per click price. Real costs, given by CPIC, are much higher. We can estimate
CPIC at the firm level for those with a significant ad effect. It comes in at $1.12 in the
unweighted average and $1.42 when we weight the average by searches. To etimate the
average across all firms, we use the lower bound approach previously described and estimate
$3.50 and $2.52 for the unweighted and weighted figure respectively. All of these averages
19
We provide more details on this in the Appendix D.
Recall these are only firms that do not face competition. Effects for these companies are presented in
columns 1 and 2 of Table 2.
20
21
are higher than the CPC competitors are paying, both at the average and firm level, which is
especially true when weighting by impressions. A second natural comparison point for CPIC
is the CPC focal brands pay on keywords when they do not occupy high organic position and
thus nearly all clicks are marginal. Even for the firms with a significant effect, CPIC exceeds
this figure for 93% (51% unweighted) of the companies that show a significantly positive ad
effect. We are unable to make firm level comparisons when the effect size is not significant
and positive, but note that the lower bound given by evaluating the CPIC at average values
well exceeds the comparison measures of CPC.
Table 4: Cost of Clicks for Focal Brands and Competitors
Unweighted average
Measure
All brands Significant
¯
CP C brand ($)
0.15
0.15
CP¯ C compet ($)
0.86
0.78
‡
CP¯ C brand−other ($)
1.36
1.43
∗
¯
CP ICbrand ($)
3.50
1.12
CP ICbrand > CP Ccompet (%)
51.2
CP ICbrand > CP Cown−other (%)
55.9
N
268
43
Weighted average†
All brands Significant
0.09
0.06
0.61
0.47
1.14
0.92
∗
2.52
1.42
92.7
90.6
268
43
† Weights
‡ CPC
are number of exposures
paid by the brand on other keywords when they are not high in the organic listing. It is
computed for 85% of companies in the sample where a match is possible.
∗ Lower bound computed with average values as described in the text.
Taken together, the evidence indicates that these firms, which did not face competition in
our sample period, are spending a substantial amount of money on brand keyword advertising
when the CPIC calculations indicate that they could acquire clicks for a lower real cost via
other keywords. Similarly, firms are paying more to attract these customers than their
competitors are.
What explains this pattern of results? One way to rationalize behavior is to posit a higher
valuation for clicks for brand searches as compared to more generic “product searches.” For
any given effect size there is a valuation premium that rationalizes the implicit CPIC. While
we cannot rule such valuations out, it seems questionable that users already familiar enough
with a brand to enter a targeted query are substantially more valuable than users acquired
through more generic keywords or those specifically searching for competitors. Indeed, since
22
brand searchers are already aware of the focal brand, the “brand awareness” channel of
advertising effectiveness is shut down (Laurent et al., 1995), which would point to a prior that
these users are less valuable. The second class of explanations points to bounded rationality
on the part of the decision makers, such as the online media buyer at firms.21 If decision
makers focus on vanilla CPC, the familiar and widely reported metric, as opposed to CPIC,
which must be computed via experimentation, then the low CPC, often ten times less than
what the brand pays on generic keywords, makes brand searches look quite attractive. This
explanation is supported by past work showing that people struggle to “think marginally,”
sub-optimally focusing on average as opposed marginal measures (de Bartolome, 1995).22
Finally, it could be that the proximate decision makers know the CPIC but choose to report
CPC to their less-savvy superiors in order to frame the purchase as a “good deal” (potentially
by adding them to a larger portfolio to reduce the average (nominal) “customer acquisition
cost” of online advertising).
While we cannot conclusively reject the first explanation, the scales seem to tip against
it. We have no way to differentiate between the bounded rationality and principle-agent
explanations and thus leave this unpacking too future work. Finally we note that among
the set of firms in our sample that do not face competition, 54.4% choose not to advertise
and some that do advertise have observable behavior that is easily rationalizable. This
indicates that if there are indeed biases (or incentive problems) at play they are by no means
universally present.
4.2
Crowd-out with Competitors Present
We now shift our focus to the case when one or more competitors clears the auction’s reserve
price and the brand’s ad still occupies the top slot.23 Competitors impact crowd-out because
their ads appear at the top of the page, shifting the brand’s organic link further down. Given
that top-down visual search is common, this pushes more traffic through the paid search link
for the brand.
21
At smaller firms, this individual would buy the advertising through the platforms online portal. For
larger firms, they likely handle a relationship with a “search engine management” firm.
22
We wouldn’t argue that firms don’t intuitively understand that some of this cannibalization is happening.
The question is to what extent firms measure and adjust their bids based on it.
23
It is exceedingly rare for the brand to occupy slots 2–4. Bids reveal that if they choose to bid, they win
the auction easily.
23
To examine the magnitude of this effect, we extract the set of keywords that, in the
absence of the experiment, have 4 ads present. We then use experimental variation to study
the effect of ads in slot 1 (focal brand) and slots 2-4 (competitors). In each case we compare
the total traffic that goes to the brand’s own website and the fraction of that traffic that
goes through the organic versus the paid link.
Figure 11: The effect of competition in increasing cannibalization
Figure 11 plots these effects. The relative uniformity in the bar height confirms what we
previously documented in terms of a modest brand ad effect and minimial traffic stealing
when the brand occupies the top slot. Far more dramatic differences emerge when we focus
on the paid/organic traffic split. When only the brand ad is present, 60% of clicks go through
the paid link. (This is higher than our previous estimates because the sample of firms here
is restricted to those that face 3 competing ads in the control condition.) This increases to
70%, 78% and 84% with one, two and three competitors present respectively.
Table 5 extends the sample of firms to include those that typically face one (first column)
two (second column) and three (corresponding to Figure 11) competing ads. The effect is
consistent across these samples of firms. A competitor’s ad in mainline 2 causally increases
24
the fraction of ads going through the focal brand’s paid link by 10-12 percentage points, an
additional 8–9 percentage points for the third slot and an additional 6 percentage points for
slot 4. When facing a full slate of competitors, the ad gets 84.3% of brand clicks.
Table 5: Crowd-out by the number of ads a brand faced in the control and number of
competitors present
Number of competitors in Control
Condition
1
2
3
Cap 1
53.9% 56.6%
59.5%
(0.29) (0.42)
(1.33)
Cap 2
65.5% 68.0%
69.9%
(0.32) (0.39)
(1.30)
Cap 3
77.3%
78.2%
(0.38)
(1.18)
Cap 4
84.3%
(0.91)
N
784
682
564
Standard errors in parentheses. In Cap 0, not shown, is 0% in all cases.
It is clear why brand’s have fought to ban competing firms from bidding on their trademarked terms. If competitors do not advertise, a slim majority of brands choose not to
advertise, meaning all clicks are free. Those that do advertisers have crowd-out levels of
roughly 54–60%. If competitors bid aggressively, the vast majority of companies choose to
advertise in order to “defend their turf.” This practice can be explained by the large click
stealing we observed if no defense is present. Such defense is largely effective, but, as competition increases, would-be free clicks are causally pushed to the paid link, raising costs for
the focal brand. A competitor can “tax” a rival through this channel, while stealing a few
clicks along the way. It is not surprising, then, that this is a commonly observed practice.24
4.3
CPIC with Competitors Present
To compute CPIC, we need the relevant counterfactual of what would happen if the focal
brand did not advertise. When competitors are absent, if the firm stopped bidding, then no
ads would be present and thus we can measure CPIC at the firm level by comparing Cap 0 to
24
One could imagine that either collusion or repeated game dynamics lead to a “don’t bid on mine and I
won’t bid on yours” situation. To the extent that this occurs, it does not appear to be common.
25
Cap 1. In contrast, when competitors are present, if the focal brand stopped bidding, then a
competitor would occupy the top slot. There is not a comparison of experimental conditions
that provides us with this counterfactual for the set of firms that choose to advertise. Instead,
for a different set for firms, namely those that do not advertise, we can estimate the impact
of competitors by experimentally varying the level of competition, which was presented in
the second part of the last section. Although we are unable to estimate CPIC at the firm
level, we can use the analysis to give back-of-the-envelope estimates.
In Section 3.2 we noted that there were two types of firms that chose not to advertise,
resellers and exclusive types, and that only the exclusive types seemed to match firms that
typically do advertise. For exclusive types, competitors were able to steal about 20 percentage
points of clicks, though there was heterogeneity in the effect. By advertising, the causal effect
of the ad roughly cancels out click stealing, meaning the relevant “effect size” is about 20
percentage points. The faction of traffic going through the paid link is 65–70%, 77–78%
and 84% with 1–3 competitors present respectively. This gives average CPIC/CPC ratios
ranging from 3.2 to 4.2—far lower than the ratios we observed in the absence of competition.
Returning to Table 4, brands are paying about $0.06–$0.15 cents per click on their own
keywords, while competitors are paying $0.47–$0.78. Further, focal brands tend to pay
a bit more than competitors on generic keywords. Multipliers in the range of 4 are thus
easy to justify. If we halve the ad effect, then the ratio enters close call territory, meaning
conditional on competitors advertising, many brands “enjoy” a surplus, even though they
of course prefer competitors to be banned outright. In contrast, for very strong brands that
have a low causal effect, defense may not be warranted.
5
Conclusion
In this paper, we’ve described and documented the competitive environment for brand keyword advertising. This market has the interesting property that a free substitute exists in
close proximity to the paid advertisement. While this is a particularly strong substitute in
the case of brand keywords, the dynamic is generally present in sponsored search. Consider,
for example, the decision of a large firm such as Geico of whether or not to advertise on
keywords such as “car insurance.” Based on their size and relevance, Geico enjoys a high
organic ranking. While not as dramatic as the case of brand search, the organic channel still
26
creates a wedge between the commonly reported CPC and the real cost of a marginal click,
given by CPIC. Since the largest firms tend to advertise on these types of generic keywords,
this wedge will commonly exist, albeit with varying size.
The importance of this wedge depends critically on the level of competition. In the
absence of competition, advertising on one’s own keyword drives a statistically significant but
economically modest amount of traffic to the brand’s website, with a larger effect observed
for weaker brands. The modest increase, however, comes at a relatively large cost: roughly
half of would-be free clicks are shifted to the paid link, which creates a large wedge between
nominal and real costs. We argued that the observed CPIC/CPC ratios for many firms that
choose to advertise in the absence of competition are hard to explain with fully rational agents
and complete incentives. Although we note that 54% of firms elect not to advertise in this
case. When competitors bid aggressively on a brand’s keyword the situation fundamentally
changes. “Defensive advertising” by the brand is an effective way to prevent click stealing
and the implicit causal effect of these ads indicates it is an easily rationalizable expenditure.
Consistent with this observation, the vast majority of brands that face competition choose to
advertise. Conditional on the brand occupying the top slot, the level of competition matters:
adding competitors gives rise to a modest increase in click stealing and a substantial increase
in crowd-out. This makes the defense simultaneously less effective and more expensive, and
it is thus unsurprising that we commonly observe firms advertising on their rivals’ keywords.
Taken together, these factors substantially complicate estimating the returns to search
advertising. Although CPC is widely used and easy to understand, it often does not capture
true marginal costs of traffic. In contrast, CPIC is an economically sound metric but it
is complicated because it depends on the relevant counterfactual, which may not be easily
observable, can change over time or is an unfamiliar concept to decision makers. We do,
however, observe evidence that many firms broadly understand these complexities. For
example, advertising on brand keywords is far more common when competitors are present.
Nonetheless, when we have the resolution to study behavior at the firm level, there is evidence
that the depth of these complexities is not fully understood or captured by within-firm
incentives.
27
References
Chiou, L. and Tucker, C. (2012). How does the use of trademarks by third-party sellers
affect online search? Marketing Science, 31(5):819–837.
Craswell, N., Zoeter, O., Taylor, M., and Ramsey, B. (2008). An experimental comparison
of click position-bias models. In Proceedings of the 2008 International Conference on Web
Search and Data Mining, pages 87–94. ACM.
de Bartolome, C. A. (1995). Which tax rate do people use: Average or marginal? Journal
of Public Economics, 56(1):79–96.
Desai, P. S., Shin, W., and Staelin, R. (2014). The company that you keep: When to buy a
competitor’s keyword. Marketing Science, 33(4):485–508.
Dupret, G. E. and Piwowarski, B. (2008). A user browsing model to predict search engine
click data from past observations. In Proceedings of the 31st annual international ACM
SIGIR conference on Research and development in information retrieval, pages 331–338.
ACM.
Goldman, M. and Rao, J. M. (2015). Experiments as instruments: Heterogenous position
effects in sponsored search. In Available on SSRN.
Gomes, R. and Sweeney, K. (2014). Bayes–nash equilibria of the generalized second-price
auction. Games and Economic Behavior, 86:421–437.
Granka, L. A., Joachims, T., and Gay, G. (2004). Eye-tracking analysis of user behavior in
www search. In Proceedings of the 27th annual international ACM SIGIR conference on
Research and development in information retrieval, pages 478–479. ACM.
Ito, K. (2014). Do consumers respond to marginal or average price? evidence from nonlinear
electricity pricing. THE AMERICAN ECONOMIC REVIEW, 104(2):537–563.
Jerath, K., Ma, L., Park, Y.-H., and Srinivasan, K. (2011). A “position paradox” in sponsored
search auctions. Marketing Science, 30(4):612–627.
Jeziorski, P. and Segal, I. (2014). What makes them click: Empirical analysis of consumer
demand for search advertising. Technical report, Working papers, Haas School of Business.
28
Laurent, G., Kapferer, J.-N., and Roussel, F. (1995). The underlying structure of brand
awareness scores. Marketing Science, 14(3 supplement):G170–G179.
Lewis, R. A. and Nguyen, D. T. (2014).
play advertising’s competitive spillovers
http://dx.doi.org/10.2139/ssrn.2374414.
A samsung ad for the ipad?
disto search.
Available at SSRN
Reiley, D. H., Li, S.-M., and Lewis, R. A. (2010). Northern exposure: A field experiment
measuring externalities between search advertisements. In Proceedings of the 11th ACM
conference on Electronic commerce, pages 297–304. ACM.
Sayedi, A., Jerath, K., and Srinivasan, K. (2014). Competitive poaching in sponsored
search advertising and its strategic impact on traditional advertising. Marketing Science,
33(4):586–608.
Simon, C. J. and Sullivan, M. W. (1993). The measurement and determinants of brand
equity: a financial approach. Marketing science, 12(1):28–52.
Yang, S. and Ghose, A. (2010). Analyzing the relationship between organic and sponsored search advertising: Positive, negative, or zero interdependence? Marketing Science,
29(4):602–623.
29
6
Appendices
6.1
Appendix A: Matching with Auction Data
Experiment we use randomly restricts the number of possible paid links on top of the search
page. The control group corresponds do the default, which is a maximum of 4 advertisements
in the mainline (Cap 4). There are 4 experimental conditions: Cap 0, 1, 2 and 3. The idea
is similar to the control: e.g. Cap 0 does not allow any advertisements in the mainline, and
Cap 3 allows at most 3 advertisement in the mainline.
This design of the experiment restricts us to studying only the cases where advertisement
is eligible to be shown in the mainline, which means that in the absence of experiment
advertisement will be shown in the mainline. E.g. we cannot study the effect of advertisement
for a company that does not advertise on its own keyword: there will be no own brand
advertisements in both Cap 0 and Cap 1 condition.
Thus, we restrict our attention to cases where companies advertise. We are still facing a
challenge: if a company advertises only 50% of the time on a given query and selects search
traffic where the effect will be higher25 , we cannot compare occasions with advertisement to
the treatment condition where the ad will be removed. E.g. if we would like to estimate
the effect of own brand advertisement in mainline 1 when in 50% of the cases company
advertises, and in 50% of the cases there is no advertisement, comparing occasions in Cap 1
conditions with a paid link shown to the entire Cap 0 conditions will bias the estimates.
To find the right treatment group, we need to allocate the occasions where ad was actually
removed from the mainline. In the example above, we would like to compare occasions with
own brand paid link in Cap 1 to occasions in Cap 0 when own brand paid link would have
been shown. To find such occasions, we collect the auction data for the search queries in the
experiment. The allocation of positions in mainline follows the standard GSP auction rules:
players submit the bids for price of a click, platform computes the ”rankscore” of a given
player, and players are allocated the positions in mainline based on their rankscores. Given
that the reservation level is cleared, company with the highest rankscore gets position 1,
company with the second highest rankscore gets position 2, etc. Rankscore is proportional
25
E.g. using geo-targeting
30
to the bid and a probability of click on the ad as computed by the platform
RSj ∝ bj pclickjα
(1)
where bj is a bid of company j, pclickj - probability of company j to get a click, and α is
the tuning parameter.
This implies that knowing the rankscores of bidders and reservation level for a search
query allows to say which advertisement would be shown in the mainline in the absence of
the experiment. To get this information, we exploit auction data collected by the advertising
team. Experiment that we use was designed by removing the potential advertising slots from
mainline, but bidding data was still collected.
Table 6: Summary of matching of the experimental and auction data
Condition
Cap 0
Cap 1
Cap 2
Cap 3
Control
Searches
Searches
%
Total
Matched Matched, % ML 1
3162615 1506827
47.6
30.6
6342073 3568054
56.3
41.8
6338914 3568918
56.3
41.8
6348311 3577819
56.4
41.9
22209220 12506083
56.3
41.9
of eligible ads in
ML 2 ML 3 ML 4
9.2
4.6
2.4
11.2
4.7
2.9
10.9
6.1
3.2
11
5.7
4.1
11
5.7
3.5
Table 6 presents the summary of matching experimental data and collected auction data
for Cap 0-4. For Cap 1, 2, 3 and 4, around 56.3% of search queries in experimental data
where matched with the auction data. A search query will not be recored in the auction
data if no advertiser submitted a non-trivial bid26 , so the unmatched data can correspond to
queries with no bidders.The majority of unmatched queries correspond to occasions where
no advertisements were shown, which supports this explanation.
Cap 0 condition has a higher percentage of unmatched search queries. This indicates
the problem with the matching, given that the experiment was constructed to be balanced
between the treatment and control groups. We further find that percent of advertisement
eligible for the mainline 1 position in Cap 0 is substantially different from the percent of
advertisements eligible for mainline 1 position in Cap 1, 2, 3 and 4.
26
As defined by the platform
31
This creates a potential problem for using the occasions with eligible brand advertisements for Cap 0 condition. Consider the case of estimating the effect of own brand advertisement in mainline 1. Using the matched data, we would like to compare occasions
with own brand advertisement from Cap 1 condition to occasions with eligible own brand
advertisement from Cap 0 conditions. We know that some occasions with eligible own brand
advertisement are missing from Cap 0. If this mismatch is correlated with the probability
of a click on own brand weblink, our estimate of advertisement effect will be biased.
To check if there is a selection problem in Cap 0 matching, we estimate the effect of own
brand advertisement in mainline 1 for companies which always advertise in mainline 1 on
their own keyword27 . For these companies, comparison of Cap 0 to Cap 1 provides the casual
effect of own brand advertisement: we know that, if not the experiment, search results in
Cap 0 will have own brand advertisement in mainline 1. We also can estimate the effect
using only eligible own advertisement occasions in Cap 0 and Cap 1. If the estimates of the
effect based on two methods are different, we can confirm that the occasion in Cap 0 which
have eligible own brand advertisement in mainline 1 are correlated with the probability of
click on own brand website.
Table 7: Effect of own brand ad in mainline 1 is significantly underestimated when using
eligible ads
Ncomp
p̂own0
p̂own1
p̂own1 − p̂own0
All queries
391
0.7867
(0.0022)
0.8035
(0.0015)
0.0168
(0.0027)
When own ad is eligible
391
0.8115
(0.0029)
0.8179
(0.0015)
0.0063
(0.0033)
p̂own0 is the probability of a click on own brand link in Cap 0
p̂own1 is a similar probability in Cap 1
Table 7 presents the estimation results. Advertisement effect estimate based on all traffic
is 1.68 percent points. Advertisement effect estimated based only on traffic with eligible
own advertisement is 0.63 percent points. The difference in two estimates is statistically
27
Companies that have own brand advertisement in mainline 1 more than 99% of the time
32
significant28 .
We thus confirm that the occasions of eligible own brand ads in mainline 1 in Cap 0 are
correlated with the probability to get a click on own brand weblink. This restricts us from
using the eligible advertisements occasions to compare Cap 0 and Cap 1. Instead, we focus
only on companies that have a paid link in mainline 1 more than 90% of the time. For these
companies, comparison of Cap 0 and Cap 1 gives casual effect of advertisement.
Figure 12 shows that around 50% of companies advertise at least 10% of the time, with
around 33% advertising more than 90% of the times. Restricting the analysis to the later
group gives us 824 companies which always advertise on their keyword.
28
Difference in estimates is 0.0105, with standard error of the difference being 0.0043, which corresponds
to a t-stat of 2.46
33
Figure 12: Frequency of ads in mainline 1 for 2517 most popular brand queries
(a) Frequency of own ads in ML1
(b) Frequency of competitor’s ads in ML1
(c) Frequency of ads in ML1
34
6.2
Appendix B: Additional Table
Table 8: Brand capital measures
Mean
log(expos)
7.59
log(Rankglobal )
10.52
log(RankU S )
9.05
bounce rate, (%)
35.1
time spent per day (minutes)
4.6
pages viewed per day
4.5
search traffic (%)
19.85
Standard deviation
1.41
2.14
1.96
14.3
2.98
2.76
9.19
Table 9: Effect is higher for brands with more bidders on their keywords
Total
Quantiles of number of bidders
0-0.1 0.1 - 0.25 0.25 - 0.75 0.75 - 0.9 0.9 - 1
Ncomp
493
48
74
246
74
51
# of bidders 11.55
3.7
6.22
10.88
16.72
22.42
pnoad
pad
pad − pnoad
0.779
(0.001)
0.800
(0.001)
0.819
(0.003)
0.833
(0.002)
0.807
(0.003)
0.825
(0.002)
0.780
(0.002)
0.802
(0.001)
0.743
(0.004)
0.766
(0.003)
0.749
(0.005)
0.776
(0.003)
0.021
(0.001)
0.014
(0.004)
0.018
(0.004)
0.022
(0.002)
0.023
(0.005)
0.027
(0.006)
35
Table 10: Companies with higher competition have higher own ad effect
Dependent variable:
Effect of own ad in ML1
(1)
Frequency of competitor in ML2
(2)
(3)
∗∗∗
0.032
(0.007)
Number of bidders
Constant
0.012∗∗∗
(0.003)
0.001
(0.0004)
0.014∗∗∗
(0.005)
Observations
R2
Adjusted R2
493
0.043
0.041
493
0.005
0.003
∗
Note:
6.3
p<0.1;
∗∗
Appendix C: Deeplinks and Dcard Example
36
p<0.05;
0.034∗∗∗
(0.007)
−0.0002
(0.0004)
0.013∗∗∗
(0.005)
493
0.044
0.040
∗∗∗
p<0.01
Figure 13: Deeplinks example
37
6.4
Appendix D: Lower bound on CPIC/CPC
We compute the CPIC/CPC ratio for a company with average effect size. This estimate
gives a lower bound on the average CPIC/CPC ratio as CPIC/CPC is a convex function in
the effect size:
xp1
CP IC/CP C =
p 1 − p0
Fix the xp1 and assume the effect size p1 − p0 > 0. Denote the LHS of the CPIC/CPC
equation as f (∆(p)), where ∆(p) = p1 − p0 Then, by Jensen’s inequality, E(f (∆(p))) >
f (E(∆(p))). Hence, computing CPIC/CPC in the average effect point E(∆(p)) gives a
lower bound on the expected value of CPIC/CPC.
38
6.5
Appendix E
Figure 14: Probability to get a click for firms that bid on their own brand keywords
(a) Cap 0
(b) Cap 0 versus Cap 1
39
6.6
Appendix F: Competition and Brand Capital Relationship
Table 11: Companies with higher brand capital have less competition
Dependent variable:
Frequency of competitor in ML2
(1)
(2)
0.052∗∗∗
(0.009)
Constant
−0.126∗
(0.075)
0.028∗∗∗
(0.009)
−0.023∗∗∗
(0.008)
−0.022∗∗∗
(0.003)
−0.039∗∗∗
(0.012)
0.483∗∗∗
(0.109)
Observations
R2
Adjusted R2
492
0.065
0.063
492
0.216
0.209
Note:
∗
log(us)
Deeplinks
Dcard
Number of own organic links
40
p<0.1;
∗∗
p<0.05;
∗∗∗
p<0.01