Perception and effectiveness of search advertising on
smartphones
Alexa Domachowski
Heise Media Service
Karl-Wiechert-Allee 10,
30625 Hannover, Germany
[email protected]
Joachim Griesbaum
University of Hildesheim
Universitätsplatz 1,
31141 Hildesheim, Germany
[email protected]
ABSTRACT
This paper explores the perception and effectiveness of
mobile search ads from the perspective of users. The study
investigates the attention and interaction of users as well as
their subjective estimation of paid listings within Google
search results on smartphones. During the tests, each of the
20 users has to accomplish four different search tasks. Data
collection methods combine eye-tracking with clickthrough analysis and interviews. Results indicate that there
is no “ad blindness” on mobile search, but similar to
desktop search, users also tend to avoid search advertising
on smartphones. For mobile search, ads appear to cause
higher usability costs than on desktop.
Keywords
Search engine advertising, mobile search behavior, eyetracking analysis.
INTRODUCTION
Search engine advertising (SEA) is the core of the business
model of universal search engines. Google Adwords is the
financial backbone of Google search services. Search ad
related business models are an important conditional factor
of the current state of the web economy as a whole.
Adwords generates revenue for Google and connects its
advertisers with prospective customers.
For search engine users, these ads have the potential both to
be useful and to act as a barrier when searching for
information. In contrast to display advertising, which is
pushed to the user and aims to interrupt his or her current
activities, one can expect a much higher acceptance of
search ads because they are related to the current
information need as represented by the query submitted.
Nevertheless, several studies (Buscher et al., 2010; Jansen
{This is the space reserved for copyright notices.]
ASIST 2015,November 6-10, 2015, St. Louis, MO, USA.
[Author Retains Copyright. Insert personal or institutional
copyright notice here.]
Ben Heuwing
University of Hildesheim
Universitätsplatz 1,
31141 Hildesheim, Germany
[email protected]
& Resnick, 2006) point out that users prefer organic results,
which are generated on the basis of relevance criteria
automatically computed by the search engine´s ranking
algorithms. The diffusion of the mobile web changes the
parameters of search related advertising, both for users and
for the advertising industry. For example in October 2015,
more search queries have been submitted from mobile than
from stationary devices to Google Search (Sullivan, 2015).
As a result, the mobile web (including apps) develops into
the primary distribution channel for search related
advertising. However, search on mobile devices, and
therefore search related advertising, is different from search
on stationary devices. Although mobile devices are
predominantly used in stationary contexts at home and at
work, mobile search also happens on the move on many
different locations, sometimes in the context of social
interaction, and is often location based (Church & Oliver,
2011). In addition to these context specific factors, the
devices themselves have different constraints: Their screens
are much smaller than screens on stationary or portable
devices, leaving significantly less room to present search
results. As a result, according to Schwartz (2014), users are
much more focused on the first result than it is the case for
desktop search. Despite these significant differences, there
is a surprisingly small amount of user oriented research on
the perception and effectiveness of search advertising on
smartphones. Pursuant to a white paper of Marin Software
(2013) that compares mobile and desktop search, mobile
search ads receive higher click rates, lower costs and lower
conversion rates than their desktop counterparts. At first
sight, this analysis indicates that mobile search engine
advertising is accepted but shows comparatively little
effectiveness for advertisers. In an exploratory study
employing eye-tracking, Djamasbi et al. (2013) demonstrate
that mobile search advertising is not subject to “banner
blindness”. Apart from this investigation, studies explicitly
investigation users´ behavior and estimations on mobile
search engine advertising are, at least to our knowledge,
widely missing. This is the starting point of this work. We
aim to take a first step to explore basic aspects of search ad
perception, usage and acceptance from the perspective of
users, pointing to conclusions for the market for mobile
advertising. Preliminary results of this study have been
published previously (Domachowski et. al. 2015, German).
This version ads an extended analysis and a systematic
overview of the state of the art.
RELATED LITERATURE
There is plenty of research on search engine advertising. On
January 13th 2016, a search for the phrase “search engine
advertising” in publications´ titles in Google Scholar
reveals 95 results. Most of these publications are focused on
click stream analysis or auction based click management.
There are comparatively few studies explicitly aimed at the
users´ perception and acceptance of mobile search ads. In
this chapter, we aim to sketch an overview of the state of
the art of relevant research. First, we start with some basic
considerations on the perception of display advertising on
the desktop based internet. Then, we focus on investigations
explicitly covering search related advertising. Following
that, we discuss current research on (search) ads on mobile
devices. Finally, we summarize the central results and
provide an estimation of methodological aspects of these
investigations.
Perceptions of Online-Advertising
Although advertising is the business model behind “free
content” services and websites which users are accustomed
to, internet advertising usually elicits negative connotations
and is rather unpopular. As early as 1998, Benway and
Lane (1998) coined the term “banner blindness”, indicating
that user actively try to avoid ads that disturb them in their
current task. A more recent study of Pagefair and Adobe
(2014) states that the number of users with adblocking
software installed resembles nearly 5% of the internet
population, reinforcing the widespread proliferation of ad
aversion
Higgings et al. (2014) argue that eye movement is a strong
indicator for visual attention and depth of information
processing. They suggest using eye tracking to capture the
details and dynamics of visual attention. They consider it as
a method that seems more objective and less prone to biases
in comparison to individuals´ verbal statements on ad
perception. With regard to display ads on the internet, they
state that ad avoidance is a central behavioral pattern of
users. Users learn to remember typical ad positions and/or
identify ads in their visual periphery. Thus, users often
avoid viewing ads in the first place. Perception is dependent
on visual factors like ad position and animation. Relevancy
and content similarity of display ads do not influence the
probability of fixations and viewing time. Finally, memory
for display ads is rather low.
With regard to users´ minimal cognitive resource allocation
to advertising, Courbet et al. (2014) examine effects of
cognitive fluency and implicit memory of pop-up ads
viewed at low-level attention. In an experiment, while
allegedly given the task to evaluate a prepared website
about diet and health issues, 398 students have been
repeatedly exposed to different pop-up ads. A week after
this exposure, results show a more favorable attitude
towards the exposed brands than towards control brands
participants have not been exposed to during the
experiment. This holds true three months after ad exposure.
Thus, despite their bad reputation from the users’ point of
view, display ads appear to be effective, with regard to the
implicit memory of users, fostering positive attitudes on
brands and purchase intentions of users.
Search ads usually have a different visual presentation than
display ads on websites. Rather than primarily relying on
visuals (logos, images and animations), search ads usually
consist solely of written text comprising a short title, a
concise description and a URL. Owens et al. (2011)
investigate “banner blindness” regarding text ads displayed
in the context of an artificially constructed Website. Results
indicate that users tend to ignore areas that are perceived as
text advertising and that the concept of “banner blindness”
needs to be expanded to text advertising.
Search Advertising
Although to date search ads are primarily displayed as text
ads, one needs to consider that we cannot simply transfer
text ad blindness as investigated by Owens et al. (2011) to
search advertising. One has to keep in mind that the role of
ads in the context of webpages is fundamentally different to
ads on a search engine results page (SERP) for a keyword
based search task, cp. Booth & Koberg (2012: 6-11). The
basic difference between display and search advertising is
the information available for advertisers about the goals and
intentions of users. During search, the advertising message
is pulled by the user who actively states his or her
information need. Display ads can only be targeted towards
contextual factors of the user´s website visit and, if
available, to more general socio-demographic or behavioral
factors of the target audience. Nevertheless, the current
intention of the user is usually less clear when placing the
ad (Booth & Koberg 2012: 6). Whereas display ads are
often employed for brand building purposes, possibly
disturbing or interrupting users´ current activity, search ads
are usually directly addressing the stated information need
and should, theoretically, support the user in his current
task. Search ads are only displayed in explicit search
contexts. Because of these fundamental differences of
display and search advertising one should expect significant
differences with regard to ad perception on part of the
users.
One of the first studies on the perception of search ads
based on a user test is the “examination of searcher's
perceptions of nonsponsored and sponsored links during
ecommerce web searching” conducted by Jansen & Resnick
(2006). In this study, 56 students accomplish six
ecommerce searching tasks. The search task queries are
selected from the logfiles of a commercial search engine.
For each query, the first SERP on Google is retrieved and
prepared for the tests. All information hinting to Google as
the originating search engine is removed and replaced with
a fictitious brand. In total, six SERPs, containing 60 organic
and 30 sponsored links, are collected and prepared. For
each of the six pages, a “switched page” is prepared. For
these, the five sponsored links and the top five organic
results are exchanged. In the experiment students had to
interact with the pre-defined SERP. For all six tasks, half of
the SERPS presented to a participant were “Switched
Pages” and original pages. Results suggest a strong
preference for nonsponsored links. Subjects viewed
nonsponsored links first in more than 82% of all cases.
Although the paid listings are controlled with regard to
position and description on the SERP, organic results get a
higher proportion of positive relevance assessments (52%
vs. 42%). Interestingly, this only holds true for the results
description on the SERP. With regard to relevance
assessments of the results landing pages, there are no
differences between both types of results. Analyzing the
students comments during the tests, the authors conclude
that lack of trust is the primary reason for not examining
paid listings. This study allows interesting conclusions:
First, the aversion of display ads is transferred to search
ads. Second, the general mistrust of users regarding search
ads may be unfounded as the perceived relevance of
sponsored listings does not differ from the relevance of
organic results.
In an eye-tracking study, Buscher et al. (2010) investigate
search ad perception and click behavior in dependence on
ad quality and ad position. They recruit 38 subjects for their
experiment. Each subject has to accomplish 32 pre-defined
search tasks with an underlying commercial information
need. Half of the tasks are navigational and half of them are
informational. Both the initial query and the SERP are
prepared in advance. Buscher et al. (2010) define different
areas of interests on the SERP, among them the top ads, the
organic results and the ads on the right side of the SERP.
Each SERP contains three top ads and 5 side ads. Ad
quality is varied systematically: Each participant is exposed
either to mostly high or low quality ads or a mixture of
both. Low quality ads are generated by using only a subset
of the terms of the initial query, while those ads provided
by the search engine for the actual query of the task are
regarded as high quality ads. For each area of interest,
fixations and click data are collected. Total time on SERP is
also observed. As a first result it is to state, that the right
ads area is widely neglected by the users. It attracts only
1.00 % of the fixations and even less clicks (0.06%). Only
the top ads receive a significant amount of attention
(10.89% fixations) and clicks (4.98%). In addition, visual
attention on ads is dependent on ad quality but not on task
type. The amount of attention on ads is higher when ad
quality is consistently high. In sum, the proportion of visual
attention for ads is higher than the proportion of clicks
generated. Low quality ads got no clicks at all. Therefore,
we can notice that the perception and effectiveness of ads is
dependent on their quality and position. Furthermore,
sequence and thus predictability influences users´
expectations of ad quality.
Investigating the effects of different search ad presentation
styles on desktop based search on Baidu, Liu, Liu, Zhang
and Ma (2014) provide some interesting insights. The
authors compare ads marked with a lightly colored
background (WBC) and ads without such a feature (NBC).
The experiment consists of 24 tasks with 7 navigational and
17 informational/transactional queries. The first query is
predefined and for each of the 24 tasks there were two
versions of SERPs prepared. One version included NBCAds and the other included WBC-Ads. Each SERP
contained three ads. Based on the results of tests with 30
students, the authors conclude that NBC-Ads got a higher
proportion of visual attention (double length of fixations)
and nearly three times higher click rates than WBC-Ads.
The authors assume that the reason for these differences can
be found in the higher visual similarity of WBS-Ads to
organic results. Checking user satisfaction rates with
SERPS containing NBC-Ads against SERPS containing
WBC-Ads showed no significant differences. Based on
these results, NBC-Ads appear to be worthwhile for search
engines and advertisers, reaching a significantly higher
potential for monetarization without annoying their users.
Summarizing current research on the perception and
effectiveness of search ads, it is to state that users exhibit a
certain amount of mistrust to this type of results on SERPs,
although search ads are not necessarily worse than regular
results with regard to relevancy. Nevertheless, paid results
fall behind organic results with regard to fixations and
click-through-rate and appear to unite the advantages of the
information pull context in search (as there is a certain
amount of clicks) with the problems of ad avoidance of
display ads. At least, recent studies indicate that the amount
of views and clicks on paid listings is largely dependent on
the visual style and the position of the ads on the SERP.
Mobile Advertising
The dependence on position and style is of special interest
for search ads on mobile devices. As mentioned in the
introduction, mobile search is different with regard to
search context and device specific factors. At first glance,
device specific factors appear to limit the potential for
mobile search advertising as there is less room to display
ads. In addition, click allocation in search is much more
focused on the first result (Schwartz, 2014). With regard to
the touch paradigm, accidental taps need to be taken into
account as a problem for users and as a restriction for
mobile advertising (Smith 2012). As a counter argument,
one may also take into consideration that the smaller screen
size limits users´ options to avoid ad positions and that
touch based controls allow for a more natural interaction
with advertising, e.g. to directly make a call.
As has been demonstrated, the acceptance and usage of
search ads is also dependent on their relevance for the user.
In the context of mobile search, there exists much more
potential to present ads that are relevant because a larger
amount of user and/or location specific context information
is available, cp. Pelau and Zegreanu (2010) and Krum
(2010).
As already stated, in mobile there are comparatively few
studies explicitly aimed at the perception and acceptance of
mobile search ads by users. O'Donnell and Cramer (2015)
compare the perception of personalized ads on mobile and
desktop. They combine an online survey (n=256) with in
depth-interviews with 13 teens and 11 adults. According to
the survey data, desktop ads are assessed as being more
relevant than ads on mobile. The interviews reveal that
mobile ads are perceived as less personal than ads displayed
on the desktop. Considering the potential of the enriched
context information of the mobile web, this result is rather
surprising. A possible explanation is provided in the study:
One interviewee explicitly mentions that user behavior is
fundamentally different on mobile. Whereas on desktop,
one uses a broad range of websites, on mobile devices the
usage behavior is much more specialized and focused on a
few specific apps. Therefore, on desktop, advertisers are
able to collect a much broader portfolio of information for
personalization.
One study closely related to our research interest is the
work of Djamasbi et al. (2013). In a preliminary
investigation of “SERPs and ads on mobile devices”, the
authors investigate click, scroll and viewing behavior
before the first click. Their test design consists of two
search tasks on Google. The tasks are presented to the test
users in random order. During each task, the participants
have to enter a predefined query into the Google search
mask and examine the search engine result page for
relevant information. Click and scroll behavior as well as
fixations are recorded. 16 users take part in the tests. The
study reveals some interesting insights. On average, the
time until the first action is taken by users is five seconds.
In two thirds of the cases, scrolling is the first action, while
in one third of the cases, users click a result first. Data
indicates that ads influence scroll and click behavior. On
SERPs containing ads, users scroll with a shorter delay as
on SERPs without ads. On the other hand, if a SERP
contained ads there was a longer “click-delay” than on
SERPs without ads. In addition, ads receive attention by the
large majority of users. Four out of five participants focus
on ads if they are displayed. Thus, the authors conclude
there is no “banner blindness” on mobile search. To our
knowledge, this study is the first to actually investigate
users´ perception of mobile ads in search contexts. Due to
the restriction of measuring first click behavior only, its
epistemological “reach” is limited. In addition, none of the
mentioned results were statistically significant. Therefore,
one should assess this investigation as rather tentative.
Nevertheless, it serves as a kind of first point of reference
for our investigation.
To sum up, the central results of current research on ad
perception and effectiveness present an interesting picture.
To start, display advertising has a bad reputation. Users try
to avoid display ads (Benway & Lane 1998) and this is also
true for text ads (Owens et al. 2011). Display ad perception
is dependent on ad position and ad format (Higgins et al.
2014), which also holds true for search ads. In addition, the
perception of search ads depends on ad quality (Buscher et
al. 2010) and on the similarity of ads to regular results (Liu
et al. 2014). The effectiveness of display ads is seen as
rather low (Higgins et al. 2014). Nevertheless, one can
assume that ads induce a prolonged implicit memory effect
(Courbet et al. 2014). Search ads are less effective with
regard to selection probability than regular results although
not necessarily worse with regard to relevancy (Jansen &
Resnick 2006). Top ads get a significant portion of clicks,
especially if ad quality is perceived as high (Buscher et al.
2010) and if they are visually similar to regular results (Lui
et al. 2015). One can assume that search ads combine the
relevancy advantage of the pull context with the problem of
ad avoidance of display ads. Astonishingly, although it
should be easier to target user needs on mobile devices, ads
on mobile are perceived as less relevant than their desktop
counterparts. Investigating mobile search ads, Djamasbi et
al. (2013) detect differences with regard to time to first
action in dependence on the presence or absence of ads.
Although ads affect the distribution of attention of the
displayed results, ads receive the attention of 90% of users.
This hints to the assumption that in mobile search there is
no ad blindness.
Concerning methods of investigation, eye-tracking is often
employed as an indicator of attention and information
processing which is more objective and less prone to
memory biases in comparison to individuals´ verbal
statements on ad perception (Higgins et al. 2014).
Behavioral patterns of ad avoidance involve less fixations
on ads due to users learning typical positions of ads as well
as recognizing them in the periphery field of vision. Click
data analysis can be used as an additional observational
method which provides an approximation of effectiveness
of search results. Both methods are often combined and
data collection can be “enriched” with the subjective views
of users collected by surveys. Research designs often reflect
task based experimental designs. Information seeking and
browsing tasks are at the center of data collection.
Therefore, we assume that the design of the tasks can be
seen as of highest importance when designing an
investigation in this field.
METHODOLOGY
The research design of this investigation employs methods
as described in user centered studies presented in the
chapter above (cp. especially Djamasbi et al. 2013, Owens
et al. 2011, Buscher et al. 2010; Jansen & Resnick 2006).
Our research focuses on three aspects. We state them as the
following research questions:
1) How is attention distributed on the first screen of a
mobile SERP and which actions do users take?
2) How effective are ads on mobile SERPs regarding
the clicks generated as an indicator of relevance?
3) Are users subjectively aware of ads on mobile
SERPs and what is their estimation of ads?
To answer these research questions, we created the
investigation as a combination of user tests and surveys. We
apply objective data collection methods (eye tracking and
click data) with the subjective estimation of the participants
(post-test surveys). Eye tracking delivers insights with
regard to the allocation of attention on the SERP. The
selection of results, especially the click through rate on ads,
indicates perceived ad relevance, which in turn indicates
their effectiveness from the point of view of advertisers to
generate visitors on the respective target properties (usually
a landing page). The surveys show the users´ subjective
awareness and estimation of mobile search ads.
Tasks
Search tasks constitute the core of the test design. Every
test person was given the task to execute a mobile search
scenario on four pre-defined queries and information needs.
For three of the search tasks, the SERP contains ads, while
for one search task (no. 2) the SERP displayed only organic
results. The queries for the search tasks are derived from a
set of 25 informational or transactional queries (Broder,
2002) with a commercial intent, e.g. product search. This
set of 25 queries has been compiled from a list of 1,006
unique queries collected from popular search query lists
(e.g. Google Zeitgeist, Bing category top lists). For each of
the 25 queries, search intents are condensed from the
descriptions of possible information needs surveyed from
several users (up to five). For the test, four queries (and
corresponding intents) out of these 25 are selected and
prepared. A pre-test indicated that the selected queries are
too general. Because of this, the four queries had to be
modified to make them more specific and unambiguous.
For example, the query “iphone 6” was changed to “iphone
6 price”. Table 1 gives an overview of the queries and
#
Query
Search intent
1
iphone 6
price
You search for general information on
the phone and are primarily interested
in the price
2
digital
camera
test
You want to get an overview of the best
cameras and are primarily interested
on test reviews
3
wii games
You want to find an interesting game
for your Wii console
dieting tips
You want to lose weight and are
searching on information about the
best way to change nutrition habits and
about sport disciplines especially
suitable for slimming
4
Table 1. Search tasks. Note: Translated to English, The
tasks were presented in German.
Figure 1: Search Engine Results Pages (SERPs)
prepared for task 1 to 4.
reconstructed information needs of the four test scenarios.
In addition, the pre-test indicated that the participants spend
a significant amount of time on the landing pages. This
extends the duration of the test and does not provide
additional insights. Thus, each task has been restricted to a
maximum duration of three minutes. As an explanation, the
updated task descriptions present a fictitious scenario in
which the test person is asked to imagine executing the task
to bypass waiting time on a bus stop.
Test website
To reflect the most common use case, we choose Google as
the search engine employed in our investigation. As ad
displays are variable when executing the same query
repeatedly, a standardized test website has been prepared to
secure that the SERPS and containing ads are exactly the
same for each user. For that purpose, all queries are
executed on google.de with a Nexus 4 smartphone (screen
size: 4.7 inch, resolution: 1280x768 pixels) in order to
represent a typical smartphone configuration in 2014.
Firefox Mobile is used as the browser with the browser
cache and the browser history cleared while the user
executing the queries is logged out of Google. The first
SERP of the query is saved as a screenshot and the
corresponding links are re-inserted. Figure 1 gives an
illustration of the SERPS on the test website.
Figure 2: Gaze distribution (duration) before first click/scroll for task 1 (n=11), 2 (n=15), 3 (n=15), and 4 (n=14).
P1
1,5
P2
P3
1
0,5
0
1,1
0,9
0,3
0,7
1,2
0,3
0,8
0,7
0,4
0,7
0,6
0,5
0,8
0,7
0,4
Task 1
Task 2 (no ads)
Task 3
Task 4
Tasks 1/3/4
Figure 3: Average total duration of fixations (seconds) before first click/scroll by ranking position P in task 1 (n=15),
2 (n=20), 3 (n=19), 4 (n=18), and 1/3/4 (all tasks with ads). Confidence intervals α=0.05. Organic results marked white.
Test procedure and data collection
The hardware and browser used for the user tests are
identical to the smartphone type and browser app used for
preparing the test website, a Nexus 4 smartphone and the
Android Firefox Browser. A Tobii X2-60 eye-tracking
device is used to record the fixations on the smartphone
display. For that purpose, the smartphone is attached to the
Tobii Mobile Device Stand for X2 which supports a
relatively convenient and natural way of interaction with
the smartphone.1 The eye-tracking device is positioned
1
http://tobiipro.com/product-listing/mobile-device-stand
below the smartphone, which is fixed at a constant distance
in portrait mode. User can interact with the smartphone
using either their thumb or index fingers. A video of the
screen is recorded with a camera positioned above the
smartphone. Eye-movements are recorded relative to this
video. The procedure of each test is as follows. First, each
participant is given a short introduction to the test
environment, comprising hard and software including the
eye-tracker and the tasks. Second, the eye-tracker has to be
calibrated. Third, each participant executes the tasks in the
given order to secure the same ad and results sequence for
each participant, as Buscher et al. (2010) show that ad
sequence influences user behavior. This means that the
perception of the ads in the different tasks is dependent on
the perception of the ads in the preceding tasks.
Consequently, in our explorative research setting we
consider possible effects of task order in the analysis,
because a considerably larger group of participants would
have been necessary to systematically control for task
order. For consistency, each task is presented in written
form. For every task, the test facilitator stops the task after
three minutes. During the tasks, fixations, clicks and time to
first action are recorded. Finally, at the end of each user
test, a short structured interview collects the subjective
awareness of users and their estimation of mobile search
ads.
Participants and test execution
Users have been recruited via direct invitations and e-maillists within the personal and professional environment of
the first author. The test sample can be described as a rather
young (age: 23-34; 26.25 mean value), predominantly
female (12 female; 8 male), highly educated group which is
experienced in the usage of mobile devices. Two users state
that they “always” use the smartphone for information
seeking, 14 users use it “most of the time” and 4 declared to
use it “sometimes” for this purpose. A total of 21 users
participated in the tests. One of the participants does not use
a private smartphone and therefore has to be dropped from
the sample. All tests have been executed between October
31 and November 5 in 2014. The average duration of the
tests is 30 minutes.
ANALYSIS
In the following, the results are structured according to our
research questions.
Attention on ads and interaction on the first screen
We recorded fixations on the SERP before the first scroll
action and compiled the resulting gaze distributions of all
users to measure ad attention. Figure 2 visualizes gaze
distribution for all four scenarios based on the time-frame
between the start of the task and the first action
(scrolling/clicking) of a user (avg. 3.6 seconds). The
number of participants varies for variables because of
recording issues. The proportion of ads on the visible screen
estate is 57% for task 1 and 60% for tasks 3 and 4. The
attention of users during these first seconds after opening a
web-page appears to be fairly evenly among hits. Statistics
on the distributions of the duration of all fixations show,
that results on the two top ranks receive more attention
(Figure 3), regardless of ads being displayed (task 1/3/4) or
not (task 2). From this, we can conclude that users do not
systematically skip over ads positioned at the top of the
result list.
The gaze distribution for task 1 differs from the other tasks.
Here, users focus more explicitly on the hit displayed at the
top (compared to task 3 and 4, respectively). There are two
possible explanations: First, as the sequence of the tasks is
identical for all users, there could have been a learning
effect. Another possible explanation is that the two product
listing ads at the top of the SERP of task 1 accumulated
more attention because these product listings show a picture
of the product. This seems plausible, as Google states that
product listing often receive higher click through rates than
text ads (Google 2015b). In addition, Malheiros et al.
(2012) mention that targeted rich media ads receive a
higher level of attention.
Furthermore, we analyze scrolling behavior to measure the
visibility of ads that are positioned below the organic
results at the end of the first SERP. For tasks 1, 2 and 4,
roughly half of the users scroll down to the end of the page.
For task 3, only one test person scrolls to the end of the
page. This indicates that ads at the bottom of the SERP
have a comparatively low chance of being noticed.
Behavioral patterns can give an estimation of the validity
and reliability of the eye-tracking data. A quick decision to
scroll can be seen as an indication of low attention on the
first results (ads for tasks 1,3,4), while a comparatively long
time to decide on the first click can be seen as a sign of
added cognitive barriers for result selection. As a whole, the
data in table 2 indicates that over all tasks, values are
comparable as there are slight deviations with regard to
absolute values but no fundamental differences with regard
to time to first scroll and time to first click. Thus, we can
assume that behavior is relatively uniform regardless if ads
are present on the SERP or not.
However, it is to state that interaction on the SERPs varies
among users. The time until the first scroll action varies
from 1-2 seconds as the shortest time span to 7-14 seconds
as the longest time span, the mean value accounts to 4.76s
(SD 3.25s). Over all users, the data shows a normal
distribution within the mentioned time spans, but there are
some outliers who scroll after waiting for 10 seconds,
especially with regard to task 1 (three outliers) and task 4
(two outliers). The same is true with regard to time until the
first click. The shortest time span corresponds to 2-5
seconds and the longest to 32-44 seconds. The mean value
with regard to time to first click is 16.7s (SD 9.7s). Again,
over all users, the data shows a normal distribution within
the mentioned time spans. Table 2 presents the mean values
and standard deviations with regard to time to first scroll
and time to first click, grouped by task.
Time to
first scroll
Time to first
click
Task 1: iphone 6 price
5.75 (3.60)
18.25 (8.35)
Task 2: digital camera
test
4.55 (2.93)
14.25 (9.98)
Task 3: wii games
3.95 (1.36)
13.7 (8.68)
Task 4: dieting tips
4.8 (5.05)
20.6 (11.71)
Table 2: Time to first scroll and time to first click. Mean
values (SD) in seconds (n=20).
Effectiveness of ads
We measure the effectiveness of ads on the basis of their
relevance to users and their ability to drive traffic to the
website of the respective advertiser. Although positive
brand related memory effects can be expected based on ad
exposure alone (Courbet et al. 2014), the focus here is on
the aptitude of search ads to induce a direct response.
Overall, tests users click on organic and ad based search
results 176 times. Again, we see a comparable behavior
with regard to overall result selection, at least with regard to
organic results: On average, users selected 2.2 results per
task. The standard deviation is relatively uniform and
accounts to 1.07 results over all tasks. Table 3 gives an
overview on click behavior on the four tasks. Click
behavior on search ads is different between tasks. For tasks
1 and 3, the fraction of clicks on ads is nearly 10% (task 1:
8.7%, task 3: 9.8%). For task 4, there is only one click on
an ad. One can state that there is a clear preference for
organic results, despite the fact that ads take up the far
majority of the space initially visible on the SERPs (on
tasks 1, 3 and 4). For task 4, we can even state a clear
behavior of ad avoidance as there is only one search ad
selected during the tests. The reasons for this are unclear
here. One can suspect a lower ad quality or a stronger
informational aspect of the information need and search
query. This could be an object for further studies. When
considering the order of result selection, the picture
becomes even more interesting. In only two cases, the
results selected first are ads. The seven other occasions on
which ads are clicked occurred after users had already
selected organic results. Taking into account that 70% of
the results initially visible to the users on tasks 1, 3 and 4
are search ads and that only one of the results initially
visible on each SERP is an organic result, the impression is
created that search ads significantly delay users when
accomplishing their search tasks. From this perspective, one
can postulate that search ads present a tremendous usability
problem for mobile search. In addition, ads that are not
initially visible have only a small chance of being selected.
Click distributions on result positions (Figure 4) indicate a
Organic
Ads
Task 1: iphone 6
price
42
4
Task 2: digital
camera test
43
No ads
available
-
Task 3: wii games
37
4
9.8%
Task 4: dieting
tips
45
1
Awareness and estimation of ads
Awareness and estimations of ads by users is collected at
the end of each user test, with the help of a short, structured
interview. First, users are asked if they have consciously
perceived the search ads. 16 participants (80%) answer
affirmatively. Next, they are asked if they feel that ads have
disturbed them during their search task. Only one
participant confirms this statement. This indicates that the
large majority of users may accept ads. However, 12 of the
16 users who perceive search ads consciously state that they
have deliberately avoided the ads. These answers are
widely in accordance with the actual behavior during the
tests: Only one person who states that she deliberately
avoids ads has actually clicked on an ad during the
preceding search tasks. Three out of those four users who
state that they do not avoid ads selected at least one ad
during the preceding tests. Asked, whether they had noticed
that task 2 did not contain any ads, 13 of the 20 test users
answer that they had not noticed that task 2 did not contain
Task 1: iphone 6 price
4
2
0
1
2
20
10
0
3
4
5
4
6
7
8
5
2
9
11
12
1
2
20
3
3
13
4
13
14
15
16
17
Task 2: digital camera test (no ads)
8
6
2
1
10
13
8
2.2%
clear preference for top positions, as long as those positions
are not occupied by ads. Finally, as only five out of the 20
participants select search ads at all, a rather negative picture
presents itself when it comes to the effectiveness of search
ads from the perspective of users.
11
10
8.7%
Table 3: Clicks on organic results and search ads (n=20)
15
20
Fraction
of ad
clicks on
all clicks
5
1
2
6
7
2
8
9
10
11
12
13
14
19
15
16
17
Task 3: wii games
10
2
2
1
2
2
1
8
9
1
2
11
12
1
0
3
4
5
6
5
6
7
10
13
14
20
1
0
1
2
3
16
17
Task 4: dieting tips
10
10
15
4
5
2
3
6
7
5
8
5
1
8
9
10
11
12
13
14
15
Figure 4: Clicks distribution in dependence on ranking (n=20). Ads marked red.
16
17
search advertising. This supports the learning thesis as
stated above. It indicates that at least a part of the users
learned in task 1 that ads are inserted at the top of the page
even if there are no ads displayed as was the case in task 2.
Further questions focused on the general assessment of
search ads. 15 users (75%) mentioned that they click on ads
if they meet certain quality criteria. The criterion mentioned
most often was trust. Several users mentioned amazon as an
example of a trustworthy vendor. Thus, it seems that search
ad utility, seen form a users´ perspective, relies on the
relevance of results and on (brand) trust. This is a finding
that is in conformance with the results of the investigation
of Ur et al. (2012) on the perception of online behavioral
advertising. As a whole, the interviews indicate that there
are different groups of users: de facto ad clickers (the
minority) and de facto ad avoiders (the majority). This fact
is not new, but nevertheless represents bad news for search
engine providers. Nevertheless, users accept to endure ads
as a part of the deal to be able to use search engines free of
charge. In addition, there is good news for search engines
and advertisers as the data also indicates that the majority of
ad avoiders can probably be turned into ad clickers if
certain criteria are met. Of these criteria, trust seems to be
the most important. Therefore, incorporating trust signals
when displaying search ads seems to be a promising idea to
enhance the effectiveness of search ads.
SUMMARY AND DISCUSSION
Compiling the results of our analysis, we can support the
following conclusions. Search ads receive attention if they
are displayed above organic results. Nevertheless, the focus
of attention of most users is on the organic results. This is a
first indicator for ad avoidance in mobile search.
This interpretation is fortified when we observe click
behavior. In spite of the fact that search ads represent 70%
of the results initially visible to the users on tasks 1, 3 and 4
and their screen display amounts to roughly 60% of the
initially visible screen, users are hesitant to click such
results. Although all tasks that include advertising produce
clicks on search ads, and click through rates in tasks 1 and 3
correspond to the higher level in comparison to desktop as
argued in the Marin Software Whitepaper (2013), ads were
the “first choice” to answer a search task in only two out of
60 tasks (20 users a 3 tasks containing ads). This seems to
be rather negligible and emphasizes the huge dimension of
the ad usability problem seen from a perspective of users.
Thus, in contrast to their most prominent screen display, the
effectiveness of ads (measured with the click through rate)
can be assessed as rather low.
However, based on the oral reports collected, one can
postulate that ads are accepted and expected by the majority
of the users in this test. Nevertheless, most of them do not
click on ads. Interestingly, 75% of the users in this test say
that they would click on ads if certain criteria were met.
The statements reveal that apart of relevancy, trust is very
important factor. This is especially interesting as one can
argue that paid search is rather important for unknown
brands and vendors to which users usually not have been
able to develop trust. Well-known brands and vendors may
not need to employ search advertising. According to
Fishman (2013), in such a case search ads may not work
and even cannibalize clicks on organic results. This,
obviously, is an area where there is room for improvement
for search engines and advertisers.
As a whole, the findings here are widely in conformance
with results of research on search ads on the desktop, cp.
Malheiros et al. (2012), Owens et al. (2011), Buscher et al.
(2010), Jansen & Resnick (2006). In comparison to the
study of Djamasbi et al. (2013) on SERPs and ads on
mobile devices, we would be more reserved to state that ads
influence scroll and click behavior. We follow the
conclusion that there is no ad blindness on mobile search,
but find indications of a pattern of ad avoidance for a subset
of users. We would additionally state that with regard to the
effectiveness of ads, the observed perception of top ads by
users does not offer much help for advertisers. In contrast,
we believe that search ads on smartphones have a larger
negative influence on search tasks than in the context of
desktop search, where usually users can at least see more
than one organic result on the first screen.
From a methodological perspective, we estimate this
investigation as a first exploration in the field. Future
studies could include additional factors such as variating the
relevance of ads, taking a more fine grained perspective on
information needs or comparing pattern of eye-movements
of different user types, e.g. ad clickers and de facto ad
avoiders.
REFERENCES
Acquisti, A., & Spiekermann, S. (2011). Do interruptions
pay off? Effects of interruptive ads on consumers'
willingness to pay. Journal of Interactive Marketing,
25(4), 226-240.
Benway, J. P., & Lane, D. M. (1998). Banner blindness:
Web searchers often miss “obvious” links. ITG
Newsletter, 1(3).
Buscher, G., Dumais, S. T., & Cutrell, E. (2010). The good,
the bad, and the random: an eye-tracking study of ad
quality in web search. In F. Crestani, & S. MarchandMaillet (ed.), Proceedings of the 33rd international ACM
SIGIR conference on Research and development in
information retrieval (pp. 42-49). Geneva: ACM.
Booth, D. & Koberg, C. (2012). Display Advertising: An
hour a day. Indianapolis: Wiley.
Broder, A. (2002). A taxonomy of web search. ACM Sigir
forum 36, 2, 3-10.
Church, K., & Oliver, N. (2011, August). Understanding
mobile web and mobile search use in today's dynamic
mobile landscape. In M. Bylund (Ed.), Proceedings of the
13th International Conference on Human Computer
Interaction with Mobile Devices and Services (pp. 6776). Stockholm: ACM.
Courbet, D., Fourquet‐Courbet, M. P., Kazan, R., &
Intartaglia, J. (2014). The Long‐Term Effects of E‐
Advertising: The Influence of Internet Pop‐ups Viewed at
a Low Level of Attention in Implicit Memory. Journal of
Computer‐Mediated Communication, 19,2, 274-293.
Djamasbi, S., Hall-Phillips, A., & Yang, R. R. (2013).
SERPs and ads on mobile devices: An eye tracking study
for generation Y. In C. Stephanidis, & M. Antona (Ed.),
Universal Access in Human-Computer Interaction. User
and Context Diversity (pp. 259-268). Berlin, Heidelberg:
Springer.
Domachowski, A., Griesbaum, J., Heuwing, B. (2015).
Wahrnehmung und Effektivität suchbezogener Werbung
auf Smartphones. In S. Elbeshausen, G. Faaß, J.
Griesbaum, B. Heuwing, & J. Jürgens (Ed.), Proceedings
des
9.
Hildesheimer
Evaluierungsund
Retrievalworkshops (HIER 2015) (pp. 117-128).
Hildesheim: Universitätsverlag Hildesheim.
Fisman, R. (2013). Do Paid Search Ads Work? eBay
doesn’t think so. Harvard Business Review. Paid Search
Ads: Did eBay just prove they don’t work? Retrieved
May
28,
2016
from
http://www.slate.com/articles/business/harvard_business_
review/2013/03/paid_search_ads_did_ebay_just_prove_t
hey_don_t_work.html
Google (2015). Google Inc. Announces Second Quarter
2015 Results. Retrieved May 28, 2016 from
https://abc.xyz/investor/news/earnings/2015/Q2_google_
earnings/
Google (2015b). About Shopping campaigns and Product
Listing Ads. Retrieved May 28, 2016 from
https://support.google.com/adwords/answer/2454022?hl=
en
Higgins, E., Leinenger, M., & Rayner, K. (2014). Eye
movements when viewing advertisements. Frontiers in
psychology, 5.
Jansen, B. J., & Resnick, M. (2006). An examination of
searcher's perceptions of nonsponsored and sponsored
links during ecommerce Web searching. Journal of the
American Society for Information Science and
Technology, 57, 14, 1949-1961.
Krum, C. (2010). Mobile marketing: Finding your
customers no matter where they are. Pearson Education.
Liu, Z., Liu, Y., Zhang, M., & Ma, S. (2014). How Do
Sponsored Search Results Affect User Behavior in Web
Search?. In A. Jaafar, N. M. Ali, S. Azman, M. Noah,
A.F. Smeaton, P. Bruza, Z.A. Bakar, N. Jamil, & T. M. T.
Sembok (Ed.), Information Retrieval Technology, 10th
Asia Information Retrieval Societies Conference, AIRS
(pp. 73-85). Kuching: Springer International Publishing.
Malheiros, M., Jennett C., Patel, S., Brostoff, S., & Sasse,
M.A. (2012). Too Close for Comfort: A Study of the
Effectiveness and Acceptabilitiy of Rich-Media
Personalized Advertising. In J. A. Konstan (Ed.),
Proceedings of SIGCHI Conference on Human Factors
in Computing Systems (pp. 579-588). Austin: ACM.
Marin Software Whitepaper (2013). Mobile Search
Advertising around the Globe. How Smartphones &
Tablets Are Changing Paid Search. 2013 Annual Report.
Retrieved
May
28,
2016
from
http://www.iabaustralia.com.au/uploads/uploads/201310/1382054400_daf59f00d5fda6ce1b6f0cbae937f0ec.pdf
O'Donnell, K., & Cramer, H. (2015). People's Perceptions
of Personalized Ads. In A. Gangemi, S. Leonardi, & A.
Panconesi (Ed.), Proceedings of the 24th International
Conference on World Wide Web Companion (pp. 12931298). Florence: ACM.
Owens, J. W., Palmer, E. M., & Chaparro, B. S. (2014).
The pervasiveness of text advertising blindness. Journal
of Usability Studies, 9, 2, 51-69.
Owens, J. W., Chaparro, B. S., & Palmer, E. M. (2011).
Text advertising blindness: the new banner blindness?
Journal of Usability Studies, 6, 3, 172-197.
Pelau, C., & Zegreanu, P. (2010). Mobile marketing - the
marketing for the next generation. Management &
Marketing, 5, 2, 101.
Pagefair and Adobe (2014). Adblocking goes mainstream.
PageFair and Adobe 2014 report. Retrieved May 28,
2016
from
http://downloads.pagefair.com/reports/adblocking_
goes_mainstream_2014_report.pdf
SeoClarity (2014). If you don’t rank first in mobile you
may as well be on the 2nd page. Retrieved May 28, 2016
from http://www.seoclarity.net/mobile-desktop-ctr-study11302/
Schwartz, B. (2014). Mobile Search Ranking Study: Rank
Number One Or Not Rank At All. Ranking in position
one in mobile is way more important than on desktop
search.
Retrieved
May
28,
2016
from
http://searchengineland.com/mobile-search-rankingstudy-rank-number-one-rank-206510
Smith, S. (2012). Up To 40% Of Mobile Ad Clicks May Be
Accidents Or Fraud? Retrieved May 28, 2016 from
http://www.mediapost.com/publications/article/182029/u
p-to-40-of-mobile-ad-clicks-may-be-accidents-or.html
Sullivan D. (2015). More Than Half Of Google’s Searches
Happen On Mobile, Worldwide. Retrieved May 28, 2016
from http://searchengineland.com/half-of-google-searchis-mobile-232994
Ur, B.; Leon, P. G.; Cranor, L. F.; Shay, R. & Wang, Y.
(2012). Smart, useful, scary, creepy: perceptions of
online behavioral advertising. In L. F. Cranor (Ed.),
Proceedings of the Eighth Symposium on Usable Privacy
and Security (pp. 4:1-4:15). New York: ACM.