Public Understanding
of Science
http://pus.sagepub.com/
Can media monitoring be a proxy for public opinion about technoscientific
controversies? The case of the Italian public debate on nuclear power
Federico Neresini and Andrea Lorenzet
Public Understanding of Science published online 6 October 2014
DOI: 10.1177/0963662514551506
The online version of this article can be found at:
http://pus.sagepub.com/content/early/2014/10/01/0963662514551506
Published by:
http://www.sagepublications.com
Additional services and information for Public Understanding of Science can be found at:
Email Alerts: http://pus.sagepub.com/cgi/alerts
Subscriptions: http://pus.sagepub.com/subscriptions
Reprints: http://www.sagepub.com/journalsReprints.nav
Permissions: http://www.sagepub.com/journalsPermissions.nav
Citations: http://pus.sagepub.com/content/early/2014/10/01/0963662514551506.refs.html
>> OnlineFirst Version of Record - Oct 6, 2014
What is This?
Downloaded from pus.sagepub.com by guest on October 9, 2014
551506
research-article2014
PUS0010.1177/0963662514551506Public Understanding of ScienceNeresini and Lorenzet
P U S
Theoretical/research paper
Can media monitoring be a
proxy for public opinion about
technoscientific controversies?
The case of the Italian
public debate on nuclear power
Public Understanding of Science
1­–15
© The Author(s) 2014
Reprints and permissions:
sagepub.co.uk/journalsPermissions.nav
DOI: 10.1177/0963662514551506
pus.sagepub.com
Federico Neresini and Andrea Lorenzet
University of Padova, Italy
Abstract
Is it possible to infer information about public opinion by looking at how the media discuss controversial
technoscientific public issues? We conduct content analysis on media coverage of nuclear power in Italy in
the years 1992–2012 and compare it with longitudinal public opinion surveys. By treating a large amount of
textual data and applying an innovative methodology based on indicators of the presence of ‘risky terms’, that
is, keywords referring to the issue of risk and danger, very high correlation has been found between media
discourse on risk and opposition within public opinion. The analysis is conducted testing as a preliminary
step Mazur’s hypothesis on quantity of coverage and opposition towards controversial technoscience. Then,
risk content measures are used in order to gain stronger correlations between media and public attitudes
towards nuclear power.
Keywords
mass media coverage, nuclear power, public opinion, big data, technoscientific controversies
This article is focused on a challenging research question for the fields of Public Communication
of Science and Technology (PCST) and Science and Technology Studies (STS), related to the
increased availability of large datasets for media monitoring analyses of science and technology
(S&T). This question, synthesized in the title of our essay, may seem a provocation if we consider
media monitoring as just a substitute of a consolidated research technique such as the survey; our
point becomes instead immediately clearer if we think that technological developments in web
scraping, database construction and text mining allow social researchers to handle large quantities
Corresponding author:
Federico Neresini, Department of Philosophy, Sociology, Education, and Applied Psychology, University of Padova, via
Cesarotti 10/12, 35123, Padova, Italy.
Email: [email protected]
Downloaded from pus.sagepub.com by guest on October 9, 2014
2
Public Understanding of Science 
of data from the perspective of ‘big data’ (Mayer-Schoenberger and Cukier, 2013). We therefore
ask here how the features of media coverage provide data and insights for consolidating our understanding of public attitudes towards controversial technoscience. This also means we go further
into the perspective of expanding the research agenda of PCST (as suggested by Bauer et al., 2007:
89–90) and find possible answers for monitoring long-term trends and cultural aspects related to
S&T media coverage (pp. 89–90).
Our aim is thus not to find an alternative to conducting surveys but rather, to analyse the relationship between public opinion and media coverage. To do so, we will take into consideration in
the first stance one of the few hypotheses regarding this relationship in the case of controversial
technoscientific issues, that is, the Mazur’s hypothesis. Mazur (1981) claims that there is a direct
relationship between the quantity of media attention to a controversial technoscientific issue and
the predominant orientation of public opinion towards it; we will show how this view has several
limitations and that, consequently, it is worth exploring other approaches. The debate on nuclear
power (NP) in Italy will be then analysed as a significant case study.
1. Technoscientific issues and the problem of media effects in the
era of big data
Why should making comparisons between opinion polls and media coverage be relevant when a
technoscientific controversy is publicly debated? What is at stake here are not media effects on
public opinion; they are not the aim of our research, and data considered here cannot be useful for
testing media effects hypotheses. Theories and studies on media effects are typically focused on an
individual research perspective as they base ‘their inferences on observations of individual media
users’ (Valkenburg and Peter, 2013: 222), while we consider media coverage and public opinion in
the context of social processes in their whole. We do not test any model of causal relationship
between media coverage and effects, given that we do not examine data concerning media use or
media exposition and attitudes and/or behaviours at the individual level. We have instead collected
data regarding media coverage directly from newspapers, as well as from blogs and web forums,
and data on public opinion – even if collected, of course, at individual level – are taken into account
as a whole as an expression of a more general social climate about controversial technologies (e.g.
NP). In doing so, we also considered that recent studies on media effects show that only small or
at the best moderate effects can be found (Valkenburg and Peter, 2013: 221) and that the correlation
at macro-level between media coverage and public opinion, where demonstrated, is based on a
spurious link and not a direct/causal effect.
Leaving aside the problem of media effects at the micro-level and preferring a macro-level
perspective, we considered in our research insights from cultivation theory and the analysis of
frames.
The perspective of cultivation theory (Morgan et al., 2012) allows us to look at the connection
between media and public opinion and how they mirror and transmit representations and beliefs
about technoscientific issues circulating within non-experts’ judgements and attitudes.
Cultivation theory has been developed within the media effects perspective, and it maintains
that repeated exposure to certain contents leads to the development of certain perceptions; so, for
example, people exposed to violent contents in the media develop an understanding of the world
as a violent place (Gerbner, 1998). The cultivation hypothesis has also been used for analysing
media effects and attitudes towards science (Brossard and Dudo, 2012; Morgan et al., 2012), and
it implies a long-term approach; for this reason, it can also be considered in non-causal terms, and
it is therefore compatible with the co-evolutionary approach we adopted: media and public opinion
act on each other reinforcing reciprocally. The cultivation perspective is at the same time
Downloaded from pus.sagepub.com by guest on October 9, 2014
3
Neresini and Lorenzet
connected to the concept of framing (Scheufele, 1999), suggesting that the media define and spread
different interpretations of the same issue.
According to Gamson and Modigliani (1987), a media frame is ‘a central organizing idea or
story line that provides meaning to an unfolding strip of events’ (p. 143); according to Entman
(1993), ‘to frame is to select some aspects of a perceived reality and make them more salient in a
communicating text, in such a way as to promote a particular problem definition, causal interpretation, moral evaluation, and/or treatment recommendation’ (p. 52). Still, research assuming the
frames approach is clearly interested in looking for their effects; at the same time, because frames
can be used both as independent and as dependent variables (Scheufele, 1999), the insertion of a
causal direction in the relationship between media and public opinion is due to a decision taken by
the researcher, and it is not an inherent feature of this relationship in itself; as a consequence, there
is no doubt that it can be analysed in terms of correlation. It is clear that correlation becomes even
more manifest in the long term, that is, in periods of time that are long enough to foster and
enhance a cultivation process. Moving in this direction requires the analysis of a controversial
subject that has been present in the public sphere for a long time; this is surely the case for NP, a
public issue that raised a debate lasting decades, even though its presence in the public arena has
not always been consistent, due to events such as incidents, public consultations.
Here, we analyse data related to media coverage of the NP issue in Italian newspapers, blogs
and forums, trying to understand in which terms media coverage could be regarded as highly correlated to the orientation of public opinion.
We can identify two main research questions:
RQ 1. Is there any relationship between media coverage – measured only counting how many
articles or posts are published on a technoscientific issue – and public opinion on the same
issue?
RQ 2. Does it make any difference to take into account the content of texts offered by the media
in order to better understand the aforementioned relationship? In other words, is it possible to
detect a co-evolution path between mass media and public opinion?
If these two questions refer to well-known researches in the social sciences, reflecting the interest of determining the traits, the protagonists and the arguments presented by the development of
knowledge societies, it is possible also to include a third question related to the so-called ‘big data’.
By this we mean the analysis of a quantity of documents larger than those that are usually collected
for a traditional content analysis.
The third research question, therefore, can be formulated as follows:
RQ3. Does it make sense to approach the analysis of the relationship between media coverage
and public opinion in the case of technoscientific controversies from the perspective of ‘big
data’?
Our interest is in considering the potential and the limits in the analysis of large quantities of
texts, with the aim of better understanding the orientation of public opinion. This interest in big
data as a new and promising approach for social sciences can be interpreted as another development of the more general perspective known as ‘digital methods’, in which digital data – mainly
available through the Internet, though not exclusively – are viewed in terms of both a new social
phenomenon to be studied in itself and a new source of information regarding other social processes (Rogers, 2013).
Downloaded from pus.sagepub.com by guest on October 9, 2014
4
Public Understanding of Science 
2. NP controversies in Italy
Debates on the civil use of NP can actually be considered a central element in the strategies of a
country, at the point that this issue – like the ones of the construction of dams, power lines and
other large technological systems – can raise discussions involving cultural traits of public opinion
(Bijker, 2007; Hughes, 1987); moreover, debates on NP are also globally part of the discussions
regarding the development of technology from the perspective of the risk society (Beck, 1990). In
the Italian debate, several shifts happened, both at the level of nuclear public policy and public
opinion’s. Marked enthusiasm for NP was present in Italy after World War II, together with the
economic drive sustaining the post-war ‘rebirth’. But during the 1970s, when the Italian government decided to push for further investments in NP mainly as a response to the oil crisis, the growing societal sensitivity for environmental issues resulted in NP becoming less popular than before.
An opposition to NP within public opinion was already present in March 1979, after the accident
at the Three Mile Island plant. The rising Italian anti-nuclear movement started to call for a suspension of NP, even if the Government, facing increasing energy demands, decided to keep NP as a
priority in energy policies.
The outbreak of Italian NP strategy initiated in 1986 after the Chernobyl disaster. The changing
opinions and the political struggles that followed the accident led to a referendum on 8–9 November
1987. A large part of the population voted for a revision of Italian NP policy, and this was interpreted as a clear opposition towards NP itself. From this moment on, the debate on NP in Italy
remained latent, but in the mid-2000s, the nuclear option started to be reconsidered, following a
more global trend. Thus, the Italian Government launched a new Italian strategy for NP in 2008,
and anti-nuclear environmental movement started a new campaign asking for a new referendum to
be held in June 2011.
While political and civil forces were mobilizing for the referendum, another tragedy radically
influenced the discussion: in March 2011, the accident at the Fukushima plant in Japan brought
again to the attention of global public opinion the risk of the nuclear option; the referendum of
2011 then led to the abandonment of the nuclear option in Italy.1
3. Mass media and public opinion in case of public controversies
To focus on an issue such as NP means to look at public technoscientific issues that are controversial and tend to polarize public opinion (Gaskell and Bauer, 2001; Venturini, 2010, 2012). Previous
studies on the relationship between mass media and public opinion in technoscientific controversies show relevant findings for us. A study by Bauer (2002) inspired by the cultivation approach on
the biotechnology issue over a period of more than 20 years discussed the role of mass media in
framing the public’s perception of ‘green’ biotechnologies (genetically modified (GM) plants and
crops) and ‘red’ biotechnologies (biomedical ones). A previous study by Gamson and Modigliani
(1989) on the relationship between media coverage and public opinion about NP in the United
States showed the role of interpretive packages in organizing communication around a positive
idea of ‘progress’ and a negative ‘runaway’ position.
The studies of Alan Mazur also explore the relationship between media coverage of technoscience and public opinion. His hypothesis can be stated thus: ‘when media coverage of an issue
increases, public opposition to such technology increases; when media coverage decreases, opposition also decreases’ (Mazur, 1981: 109).
Mazur’s hypothesis is quantitative, by counting articles published on a topic at a given time, you
can have insights about public opinion; but we can also look at the problem in different ways and
analyse the content of what the media say about technoscientific issues at a given time. Taking into
Downloaded from pus.sagepub.com by guest on October 9, 2014
5
Neresini and Lorenzet
account the content, the relationship between media and public opinion could become more articulated; it is not the same to say that opposition towards the use of GM organisms depends in large
part on the fact that the media cover this issue widely or that media coverage indirectly indicates
what is going on within public opinion. Therefore, if the majority of articles in newspapers and TV
appearances are oriented towards emphasizing the risky aspects of a technology, will the result of
public opinion surveys show attitudes in line with this tendency? Despite the complex relationship
between media coverage and public opinion, Mazur (1998) formulates a very simple hypothesis:
‘people are influenced more by the quantity of coverage, especially by the repetition of simple
images […], and less by their content’ (p. 459). Moreover, since the public seems affected by a
structural and preconceived ‘conservative’ attitude, for which in uncertain situations it is preferable
to choose what is already known and not what is new (Mazur, 1981: 114), Mazur’s (1987, 1990,
1998) starting hypothesis is reinforced: the more the media cover a technoscientific issue, the more
the public is brought to assume an opposition towards it.
While Mazur’s work has been criticized and called into question with regard to its consistency
(Gutteling, 2005), it nevertheless remains one of the few existing references for our research. The
main criticisms of Mazur include both the theoretical level, since his hypothesis recalls the ‘direct
effects’ approach that media studies have strongly criticized, and the empirical level, suggesting
Mazur does not take into consideration the relationship between media exposure and attitudes.
The few contributions that tested Mazur’s hypothesis directly (Gutteling, 2005; Peters, 2000;
Wiegman et al., 1989) or indirectly (Bauer et al., 2001; Bucchi and Neresini, 2002) have found a
rather weak relationship between media exposure and public opinion.
In order to study the Italian debate on NP, we decided to consider Mazur’s hypothesis, but only
as a benchmark for our analysis, which is instead based on content analysis of newspapers, blogs
and web forums, using data from Internet sources that obviously Mazur was not able to consider
during the 1980s.
4. Methodology
Articles containing both the words ‘nuclear’ and ‘power’ were collected from the archives of three
main Italian newspapers (Il Corriere della Sera, La Repubblica and La Stampa) for the period
ranging from 1992 to 2011, while data for blogs and forums were retrieved starting from 2005
because no consistent coverage of this issue was found before that period on these sources.
Five surveys on the attitudes of Italians towards the adoption of NP were considered. The surveys were conducted on September 2002, September 2005, January 2007, April 2009 and April
2011.2
In the case of newspaper data, a study previously done on the same Italian issue, but with a very
reduced set of data regarding a shorter period of time (Bucchi and Neresini, 2012), followed an
approach very similar to what is formally known as ‘sentiment analysis’, that is, classifying the
articles on the basis of their content as positive or negative about NP.
As a result, this previous attempt found ‘a relatively balanced coverage tone of NP’ characterizing the Italian media, a context in which negative opinions are slightly prevalent, given the
repeated nature of accidents at NP plants and constant references to the nuclear threat in military
affairs. Nevertheless, despite this, public opinion seems to see NP in rather positive terms (Bucchi
and Neresini, 2012: 459) at least for the considered period, from 2002 to 2009.
The presence of contradictory results in the above-mentioned study suggested the opportunity
to investigate further the relationship between public opinion and media coverage, exploring alternative routes. Instead of classifying the articles on the basis of a positive/negative evaluation, their
Downloaded from pus.sagepub.com by guest on October 9, 2014
6
Public Understanding of Science 
content has been assessed making reference to the presence of ‘risky terms’, that is, words that
belong to the semantic area of risk and danger.
We made the hypothesis that by following this approach, we could obtain more reliable data
and therefore less contradictory results. We are quite familiar with the problems connected to
sentiment analysis, realized automatically in large amounts of textual data. The application of this
approach seems to be quite stabilized in short documents that have a well-defined topic – such as
product reviews – but many questions remain open about how to treat more complex content such
as newspaper articles or blog posts.3 Classification of the articles on the basis of the presence of
risky terms is certainly less refined than the complex mathematical modelling and highly refined
model for sociolinguistic analysis, but, as we will see, its simplicity appears to offer good warranties for success. Just as Mazur focused only on the quantity of coverage regarding an issue, our
approach is not that much different: we assess the degree of the presence of specific keywords
dealing with risk and danger in the corpus of articles and posts we collected. To reach our goals,
we conducted a text mining analysis using the RapidMiner software,4 and we looked at a pattern
of ‘risky terms’ that we found particularly present in the press, as a measure of its orientation
towards risk.
The list of risky terms was retrieved by conducting a manual preliminary content analysis of
articles on the issues of risk and danger associated with NP. A total of 292 articles in which appeared
the words ‘energy’, ‘nuclear’, ‘risk’ and ‘danger’ were retrieved and read from the database of the
web archives of the three newspapers analysed, Corriere della Sera (79), La Repubblica (134) and
La Stampa (79). These formed the basis for the construction of the thesaurus of risky terms to be
used in order to classify the articles. The risky terms used for the analysis are: accident, alarm,
cancer, catastrophe, contamination, damage, danger, disaster, disease, dread, emission, fear, negative, pollution, radiation, radioactive, risk and tragedy.5
The relevance of risky terms in the articles was evaluated following two indicators: IND2.1 and
IND2.2. IND2.1 has been built by considering the frequency of risky terms in each group of documents (articles or posts) considered, that is, the articles or posts published in the period before the
poll. The indicator assumes a value of 0 (zero) when there are no documents with risky terms,
while it has its maximum value (100) when all the documents published in the considered period
of time contain all the risky terms at least one time. So, this first indicator can be considered an
indicator of the presence of risky terms, that is, a measure of how many risky terms are contained
on average in a given corpus of documents (presence measure).
IND2.2 has been instead built by considering the average values of the term frequency–inverse
document frequency (TF–IDF) scores of the risky terms in each group of documents (corpus) considered. TF–IDF is a measure that reflects how important a word is to a document in a collection
or corpus. TF–IDF assigns to the term t a weight in document d given by
TF − IDF ( t , d ) = TF ( t , d ) × IDF ( t )
In other words, TF–IDF assigns to each term in a single document a weight that is highest when
t occurs many times within a small number of documents, lower when the t occurs in many documents and lowest when the term occurs in all documents. To obtain a measure of the significance
of risky terms for each considered period of time, we calculated the average of TF–IDF values for
each risky term in relation to the group of documents – or corpus – within those periods (risk density measure).
So, while the first indicator (IND2.1) refers to the presence of the risky terms in a given corpus
without considering the features of the single articles but simply referring to the presence or not of
Downloaded from pus.sagepub.com by guest on October 9, 2014
7
Neresini and Lorenzet
76
80
70
60 56
50
43
40
30
22
22
38
39
25
19
20
10
0
2002
Unfavourable
don't know
3
2005
2007
2009
2011
Figure 1. Unfavourable to NP in Italy (%) and uncertainty towards NP (‘don’t know’ answers).
NP: nuclear power.
the terms in the considered articles, the second (IND2.2) gives a more detailed insight, describing
how much the semantic domain of risk is a significant attribute of the corpus.
We applied the same procedure to all of the three sources of data (newspaper articles, blogs and
forums6) and compared the results by considering the correlation coefficient of indicators coming
from the analysis of risky terms and the public opinion survey results.
5. Using risky terms in media coverage as a proxy for public
opinion
Survey data on Italians’ attitudes towards NP show relevant shifts in public opinion between the
years 2002 to 2011. As shown in Figure 1, while in 2002 a small majority of Italians were against
NP (56%), the percentage of opponents was below half in the following years (43% in 2005, 38%
in 2007 and 39% in 2009), and then, probably mainly as a consequence of the tragic events of
Fukushima in 2011, the majority of Italians were again contrary to NP (76% in 2011).7
It is interesting to consider the data on public opinion and the specificity of public opinion on
technoscientific subjects before comparing them with results from media analysis. As shown by
previous research on technoscientific topics, one of the main biases of surveys is the problem of
‘don’t know’ (DK) answers, that is, those given by people who feel that they have so little knowledge about technoscientific issues that they cannot answer properly on these subjects (Irwin, 1995;
Irwin and Michael, 2003; Michael, 1996). Looking at our survey data, we found that variables like
the degree of education and scientific alphabetization did not discriminate that much in relation to
the opposition to or support of NP, but tended instead to differentiate between people giving an
opinion or deciding to not give an opinion; a higher degree of DK answers is then to be found in
those that have lower education and lower scientific alphabetization. Moreover, we see that DK
answers are influenced by catastrophes: after the Fukushima disaster happened, only 3% of the
public felt uncertain whether NP should be financed or not. On the basis of this, we decided to
Downloaded from pus.sagepub.com by guest on October 9, 2014
8
Public Understanding of Science 
1200
1000
800
Corriere
La Stampa
La Repubblica
3 Newspapers
600
400
200
0
19
92
19
94
19
96
19
98
20
00
20
02
20
04
20
06
20
08
20
10
20
12
Figure 2. NP coverage in Italian newspapers, 1992–2012.
NP: nuclear power.
consider only the opposition of the public as a whole as an indicator for comparing this position
with data from media analysis.8
If we look at the trend of newspaper coverage on NP, two main periods can be identified: a first
period of scant attention by the media, going from 1992 to 2002, and a period of higher attention
starting in 2003. This pattern follows the ‘attention cycle’ dynamic, alternating phases of greater
attention with phases of less attention (Nisbet and Huge, 2006). The focusing event that brought
the debate on NP into discussion again was a huge blackout that happened on 28 September 2003,
when almost the whole of Italy was left without electricity due to an accident in the lines bringing
electricity from Switzerland. This blackout was considered to be the worst in the Italian electrical
system, and after this event, attention was devoted again to the necessity of independence of power
supply in Italy, and for this reason, NP became again an option (Figure 2).
Other peaks of coverage on NP were in 2006 and 2008. In 2006, this was the result of interaction
with the issue of nuclear weapons and the global debate on the Iran issue, while in 2008, it was the
consequence of the announcement by the Italian Government of a new strategy for bringing NP to
Italy and therefore independence of power supply, reducing its costs.
Coverage on NP finally peaked in 2011 as a result of the accident in the Fukushima plant and of
the planned Italian referendum on NP to be held in June. In 2011, the year of the coverage peak,
NP was very high on the media agenda.
We tested Mazur’s hypothesis comparing the results of public opinion surveys on NP as a first
instance with data from newspapers’ coverage relating to the previous 12 months before each public opinion survey. Results showed that there was no clear correlation between newspapers’ coverage and the public opinion data, apart from the fact that both were peaking in 2011 (see Figure 3).
Better results were obtained by considering the coverage during the previous 4 months (Figure 4).9
Here, we can consider, at least starting from 2005, that the quantity of coverage acts to a certain
degree as a proxy of public opinion so that we can test Mazur’s hypothesis comparing trends of opposition towards NP and media coverage, simply measured as the number of articles and posts (IND1).
We also tried to leave out from our sample the articles regarding the issue of Iran and ‘nuclear terrorism’, but results did not change substantially, as shown in Figure 4.
Downloaded from pus.sagepub.com by guest on October 9, 2014
9
Neresini and Lorenzet
120
100
80
% against NP
Corriere
La Stampa
La Republica
3 newspapers
60
40
20
0
2002
2005
2007
2009
2011
Figure 3. Comparing surveys’ results with media coverage (12 months before) – normalized data.
NP: nuclear power.
120
100
80
% against
NP
60
Articles
total
40
Articles
w/out Iran
issue
20
0
2002
2005
2007
2009
2011
Figure 4. Comparing surveys’ results with media coverage (4 months before) – normalized data.
NP: nuclear power.
The capability of coverage to be a proxy for public opinion seems here to be influenced by two
main factors. In 2002, coverage of the NP issue was still very low, and we can take this as a first
instance for comparing since this year is part of the first issue attention cycle and before the focusing event – the 2003 blackout in Italy – that unleashed greater media attention on the NP issue.
But considering the role of risky terms in the coverage of the previous 4 months, we have a different overview, and a strong correlation can be found between the quantity associated with the
tone of the coverage and the public opinion data, even if we look at the year 2002.
Data from the analysis of risky terms offer measures that seem to be even more comforting than
the ones obtained following Mazur’s hypothesis (Table 1): there is a very high correlation between
the presence of risky terms in the articles (measured with both indicators) and the degree of public
opposition to NP.
Downloaded from pus.sagepub.com by guest on October 9, 2014
10
Public Understanding of Science 
Table 1. Correlation between public opinion indicator (% opposition to NP) and three indicators for
media coverage (referring to the last 4 months before public opinion survey).
Public opinion
Media
Newspapers
Blogs
Forums
2002
2005
2007 2009
2011 Correlation
% opposition to NP
56
43
38
39
76
IND1 (no. of articles)
IND2.1 (presence of risky terms)
IND2.2 (average TF–IDF)
IND1 (no. of posts)
IND2.1 (presence of risky terms)
IND2.2 (average TF–IDF)
IND1 (no. of posts)
IND2.1 (presence of risky terms)
IND2.2 (average TF–IDF)
43
12.4
.007
–
–
–
–
–
–
162
7.7
.004
15
11.5
.007
73
18.4
.004
177
7.9
.005
100
13.5
.005
162
11.1
.004
158
7.6
.004
195
13.4
.006
314
14
.004
420
16.5
.007
184
21.3
.009
343
23.4
.008
.65
.99
.84
.40
.95
.93
.56
.89
.99
NP: nuclear power; TF–IDF: term frequency–inverse document frequency.
Mazur’s hypothesis found partial confirmation for the coverage of newspapers, since the correlation between the number of articles in the 4 months and the percentage of people against NP is
around .6 for this source of data, while it remains lower for the forums (.5) and especially for the
blogs (.4).
Analysis of the articles published 4 months before the surveys using IND2.1 and IND2.2 radically improves Mazur’s achievements; through the study of risky terms, the correlation between
public opinion and media coverage sensibly raises for all of the three sources considered, always
remaining above .8.
We obviously need to stress the point that we cannot reduce the measure of public opinion as a
whole to the simple measure of people who are against a technology, but this seems to be nevertheless a very appreciable result.
6. So, can really media monitoring be a proxy for public opinion?
Results presented here show clearly the existence of a strong correlation between the tone of media
coverage and the opinions – in favour or in opposition – about NP in certain conditions, that is, if
we measure the presence of risky terms in newspapers, blogs, and forums taking an adequate time
period before the survey measurement.
Now, coming back to our main question, does this really mean that we can generalize and state
that media monitoring can be a proxy for public opinion? There is at least one important reason for
offering a negative answer to this question, and it regards commensurability among different measurement tools. If we think about correlating media content analysis and opinion poll results, a
series of relevant considerations may be made. Availability of standardized longitudinal survey
indicators regarding public opinion is actually rather poor in quantity and limited in time (since
polls are made at specific time intervals due mainly to their costs), compared to availability of
information on media coverage which is instead more readily available for fast, rather cheap, heterogeneous analyses (especially in the ‘digital era’ with the use of advanced text mining tools) and
rather continuous in time (each second a huge amount of information is published in different
sources both in print and in the Web across the world). Obviously this difference could be
Downloaded from pus.sagepub.com by guest on October 9, 2014
11
Neresini and Lorenzet
considered a valid reason for making the two types of data not comparable at all, especially for a
specific topic such as technoscientific controversy. For a series of debates lacking comparable
longitudinal data on public opinion, this is actually a preferred choice. For us, the availability of
longitudinal comparable data on the NP issue over a ten year time span was enough to demonstrate
a correlation between risk discourse in the media and public opinion orientations towards a controversial technology; this fact, together with knowledge about Mazur’s previous experience on quantitatively comparing media coverage with public opinion survey data, provided us good arguments
for going on with our analysis to try to demonstrate that media monitoring can be a proxy for public
opinion. It is to be noted that even by choosing the topic of NP, for which there are plenty of data
available, comparison with other countries and cultures (e.g. with US public opinion data) was not
possible mainly because of the different formulation of questions between the US and the Italian
questionnaires considered during our research.10 A second kind of answer to our question (i.e. can
mass media be a proxy for public opinion?) has then been evaluated. Given the strong correlation
we found between our ‘risky terms’ indicators and public opinion, we could consider media coverage as a cause of changes in public opinion, following a well-defined tradition of studies that can
be identified with the label of ‘mass media effects’. Instead of denying the possibility of a correlation between the two variables, we could have decided to state a causal nexus on the basis of relevant correlation indexes. After reading the results of our textual analyses, this answer seemed to
us immediately implausible because the design of our study did not provide clear evidence of this;
a simple correlation between two variables must not be understood as a causal relation, something
that would have required far more complex analyses. We decided not to proceed with this kind of
analysis, on the one hand because mass media effects studies often adopt an individualistic epistemology, more based on information and mass media consumption patterns of the public than on
mass media published content, and on the other because our data are only aggregate on the side of
the media; moreover, we considered that even the most sophisticated, advanced and recent literature on mass media effects agrees to the fact that only weak effects can be effectively demonstrated
through comparisons based on complex causal models (Valkenburg and Peter, 2013).
To better illustrate the issue, we can reflect on the well-known case of the provisional model built
on the correlation derived by Google researches between queries about flu and the epidemiology of
flu (Ginsberg et al., 2009). It is clear that the first cannot be the cause of the second, while the second
cannot be obviously the cause of the first. Correlation can instead be interpreted as the double
expression of the fact that a flu epidemics are happening and that people are aware of this because
they are part of a social context that is sensitized by flu spreading. People then search for information when they or their relatives manifest possible symptoms. While we cannot say information
useful to know whether the people making queries are the same are those who will go to the doctor,
or have already gone to a doctor or will never go, we are only clear that a correlation exists.
The same seems to be valid for the relationship between media coverage and public opinion in
the case of NP. Taking correlation seriously, instead of causality, is, on the other hand, another
feature of approaching social problems by using big data (Mayer-Schoenberger and Cukier, 2013:
50–70) even if it does not mean the ‘end of theory’ as envisaged by Anderson (2008), but on rather
the opposite: an enforcement of it.
For example, what our results show is that the media seem to anticipate the orientation of public
opinion and that this should be interpreted as an expression of the fact that both are proxy for more
general social changes, not because they cause or are caused by them. In our case, evidence therefore gives an empirical confirmation of the theoretical stance according to which media and public
opinion co-evolve, as part of a more general transformation process regarding technoscientific
innovation, in particular when they become part of a public and collective discussion. Theories
referring to co-evolution between science, technology and society in order to
Downloaded from pus.sagepub.com by guest on October 9, 2014
12
Public Understanding of Science 
understand technoscientific processes of innovation (Jasanoff, 2007; Latour, 2005) in fact tend to
underestimate the role of mass media in the process of shaping technological innovation, apart
from some already mentioned attempts to interpret public controversies in the light of cultivation
theory (Bauer, 2002). Our findings go in the direction of establishing a ‘mitigated’ cultivation process, in which the relationship between media and public opinion is considered to be balanced and
bidirectional and having important consequences for innovation policies and practices. We therefore suggest the importance of recognizing these contributions and the importance of grounding
research on previous social science theory in order to realize and test indicators to be used in analyses similar to the ones that we presented here, using mass media data to assess more general
changes in the public sphere.
So we answer that yes, media monitoring can be a proxy for public opinion, and we moreover
suggest the possibility that content analysis based on terms indicators can be used to assess features of the wider public sphere on relevant social issues in other cases, including some orientation of public opinion considered as a whole, and not as an aggregate of individuals with specific
socio-demographic attributes. Obviously, in the case of our tests on the presence of risky terms in
newspapers’ coverage, correlation has to be considered a spurious measure and needs to be evaluated as a relationship between variables that have some semantic similarity (the orientation
towards a technology on the one hand and the public discourse on the risks connected to that
technology on the other hand); at the same time, as stated by Krippendorff (2004), correlations
between data collected from different sources – in our case opinion surveys and news articles –
are less likely to be so affected (p. 196). As a consequence, one of the strongest implications of
our study lies in the fact that if we have a correlation between two types of data, such as in our
case between mass media content and public opinion surveys, and the first type is readily available, while the second is difficult to retrieve or lacking, once correlation between the two is established, information obtained with the readily available measurement tool can be used to get the
relevant information (in our case, the orientation of the public towards a controversial technology) without any need to use the other tool (i.e. public opinion surveys). Of course, the information about the orientation of public opinion that is inferable is not as detailed as that of the second
measurement tool. In our case, for example, we could get information about opposition towards
NP only in general terms, and without any reference to socio-demographic variables, such as
gender, age, professional status or education, something that would require more detailed analyses, for example, regarding the relationship between newspapers and other media content readership on the one hand and opinion formation on the other.
Our data suggest that we can obtain encouraging results by conducting a very simple and fast
analysis. At the same time, it is important to stress two points of our study that helped gain a
stronger correlation between media coverage and public opinion.
The first element regards the sociological dimensions that build up our media indicator – ‘risky
terms’; while standard sentiment indicators and approaches rely on words and adjectives that use
positive and negative words and adjectives to determine the orientation of a corpus of texts, we
used a concept – that of ‘risk society’ (Beck, 1990). By selecting word tokens related to the semantic domain of risk, we actually translated in operative terms a very dense concept without referring
to a more probably broad but rather general idea of ‘positive’ or ‘negative’; we consider this to be
a very good way of solving difficulties in assessing the general sentiment of the public through
media monitoring techniques: if indicators are inspired by sound and relevant theory, it is more
likely that they will be able to reveal relevant quantitative patterns when coming to correlation
indexes and become themselves a source for other relevant theory.
The second element deals with the study of the features of newspapers’ coverage curve; in analysing NP, we found out that a stronger correlation is gained if we consider 4 months before the
Downloaded from pus.sagepub.com by guest on October 9, 2014
13
Neresini and Lorenzet
measurement compared to 1 year before.9 In media monitoring techniques for the analysis of social
issues, a crucial dimension to be considered is that of coverage patterns over time.
Funding
This research received no specific grant from any funding agency in the public, commercial or not-for-profit
sectors.
Notes
1. Recent essays in Italian language addressing the issue of nuclear power (NP) and its history are listed in
bibliography (Bettini and Nebbia, 2009; Chiesa et al., 2010; Clô, 2010; Labbate, 2010; Testa, 2008).
2. The five surveys have been realized by Observa Science in Society; for more information, see the website http://www.observa.it/category/osservatorio/?lang=en
3. For a review of sentiment analysis techniques, see Mejova (2009).
4. RapidMiner is an open-source software for data and text mining available at http://www.rapidminer.com
5. Although here presented in English, the Italian terms were used for the analysis. A first list of words
associated with risk has been extracted from the sample of articles and then those with a term frequency–
inverse document frequency (TF–IDF) value higher than zero have been used for the analysis.
6. Blogs and forum data were collected by doing a search with ‘Google Blog’ and ‘Google Discussions’
services, considering the 4 months before each survey.
7. Data were collected through computer-assisted telephone interviewing (CATI) surveys on representative
samples of the Italian population. The question asked to respondent has been the same over the years, and
precisely: ‘do you think that Italy should invest on NP?’ Possible answers were limited to three modalities, ‘yes’, ‘no’ and ‘don’t know’ (DK).
8. Furthermore, our data show almost a perfect correlation between (.98) the evolution of opposition values
and of those calculated as the difference between opposition and favour.
9. We chose to consider the previous 4 months of coverage before the survey on the basis of consideration
of the index of autocorrelation of the curve describing the media coverage of NP in Italian newspapers.
This index ensures that the curve is not indicating the existence of a ‘white noise’, that is, a curve without
any sort of regularity in its dynamics. The analysis of the autocorrelation index showed that in the case
of the Italian debate on NP, there has been a very high autocorrelation index for 3 months. In order to add
even more solidity to our analyses, we therefore decided to keep a 4-month interval in order to analyse
the relationship with opinion polls in the quantitative perspective of Mazur, that is, only considering the
number of articles on NP published in newspapers.
10. While in the US survey respondents were asked to evaluate NP in itself, in the Italian surveys we asked
opinion towards financing NP development. US data are available at: http://www.nei.org/CorporateSite/
media/filefolder/NEI-Perspective-On-Public-Opinion_April-2013_FINAL.pdf, last accessed 1 August
2014.
References
Anderson C (2008) The end of theory: The data deluge makes the scientific method obsolete. Wired Magazine,
16 July. Available at: http://www.wired.com/science/discoveries/magazine/16-07/pb_theory
Bauer MW (2002) Controversial medical and agri-food biotechnology: A cultivation analysis. Public
Understanding of Science 11(2): 93–111.
Bauer MW, Allum N and Miller S (2007) What can we learn from 25 years of PUS survey research? Liberating
and expanding the agenda. Public Understanding of Science 16(1): 79–95.
Bauer MW, Kohring M, Allansdottir A and Gutteling GM (2001) The dramatization of biotechnology in
the elite mass media. In: Gaskell G and Bauer MW (eds) Biotechnology 1996–2000: The Years of
Controversy. London: Science Museum, pp. 35–52.
Beck U (1990) Risk Society: Towards a New Modernity, vol. 17. London; Thousand Oaks, CA; New Delhi,
India: SAGE.
Downloaded from pus.sagepub.com by guest on October 9, 2014
14
Public Understanding of Science 
Bettini V and Nebbia G (eds) (2009) Il nucleare impossibile: perché non conviene tornare al nucleare.
Torino: UTET Libreria. Impossible nuclear: why it is not convenient to go back to the nuclear option.
Bijker WE (2007) Dikes and dams, thick with politics. Isis 98(1): 109–123.
Brossard D and Dudo A (2012) Cultivation of attitudes toward science. In: Morgan M, Shanahan J and
Signorelli N (eds) Living with Television Now: Advances in Cultivation Theory & Research. New York,
NY: Peter Lang, pp. 120–143.
Bucchi M and Neresini F (2002) Biotech remains unloved by the more informed. Nature 416: 261.
Bucchi M and Neresini F (2012) Monitoring science in the public sphere: The case of Italy. In: Bauer MW,
Shukla R and Allum N (eds) The Culture of Science: How the Public Relates to Science across the
Globe. London: Routledge, pp. 449–462.
Chiesa G, Sertorio L and Cosenza G (2010) La menzogna nucleare: perché tornare all’energia atomica
sarebbe gravemente rischioso e completamente inutile. Milano: Ponte alle Grazie. The nuclear lie: why
to go back to atomic energy would be very risky and completely useless.
Clô A (2010) Si fa presto a dire nucleare. Bologna: Il Mulino. Easier said than done when you’re talking
nuclear.
Entman RM (1993) Framing: Toward a clarification of a fractured paradigm. Journal of Communication
43(4): 51–58.
Gamson WA and Modigliani A (1987) The changing culture of affirmative action. In: Braungart RG and
Braungart MM (eds) Research in Political Sociology, vol. 3. Greenwich, CT: JAI Press, pp. 137–177.
Gamson WA and Modigliani A (1989) Media discourse and public opinion on nuclear power: A constructionist approach. American Journal of Sociology 95: 1–37.
Gaskell G and Bauer MW (2001) Biotechnology, 1996–2000: The Years of Controversy. London: NMSI
Trading Ltd.
Gerbner G (1998) Cultivation analysis: An overview. Mass Communication and Society 1(3–4): 175–194.
Ginsberg J, Mohebbi MH, Patel RS, Brammer L, Smolinski MS and Larry Brilliant L (2009) Detecting influenza epidemics using search engine query data. Nature 457(19): 1012–1014.
Gutteling Jan M (2005) Mazur’s hypothesis on technology controversy and media. International Journal of
Public Opinion Research 17(1): 23–41.
Hughes TP (1987) The evolution of large technological systems. In: Douglas DG, Bijker WE, Hughes TP and
Pinch T (eds) The Social Construction of Technological Systems: New Directions in the Sociology and
History of Technology. Cambridge, MA: The MIT Press, pp. 51–82.
Irwin A (1995) Citizen Science: A Study of People, Expertise and Sustainable Development. London; New
York, NY: Routledge.
Irwin A and Michael M (eds) (2003) Science, Social Theory and Public Knowledge. Maidenhead: Open
University Press.
Jasanoff S (2007) Designs on Nature: Science and Democracy in Europe and the United States. Princeton,
NJ: Princeton University Press.
Krippendorff K (2004) Content Analysis: An Introduction to Its Methodology. Thousand Oaks, CA: SAGE.
Labbate S (2010) Il governo dell’energia: l’Italia dal petrolio al nucleare 1945–1975. Firenze: Le Monnier.
Governing energy: Italy from oil to nuclear 1945-1975.
Latour B (2005) Reassembling the Social: An Introduction to Actor-Network Theory. Oxford: Oxford
University Press.
Mayer-Schoenberger V and Cukier K (2013) Big Data: A Revolution That Will Transform How We Live,
Work, and Think. New York, NY: Houghton Mifflin Harcourt.
Mazur A (1981) The Dynamics of Technical Controversy. Washington, DC: Communications Press.
Mazur A (1987) Putting radon on the public’s risk agenda. Science, Technology, & Human Values 12(3/4):
86–93.
Mazur A (1990) Nuclear power, chemical hazards, and the quantity of reporting. Minerva 28(3): 294–323.
Mazur A (1998) Global environmental change in the news: 1987–90 vs 1992–6. International Sociology
13(4): 457–472.
Mejova Y (2009) Sentiment analysis: An overview. Comprehensive exam paper. Available at: http://www.
academia.edu/291678/Sentiment_Analysis_An_Overview (accessed 16 September 2014).
Downloaded from pus.sagepub.com by guest on October 9, 2014
15
Neresini and Lorenzet
Michael M (1996) Ignoring science: Discourses of ignorance in the public understanding of science. In: Irwin
A and Wynne B (eds) Misunderstanding Science? The Public Reconstruction of Science and Technology.
Cambridge: Cambridge University Press, pp. 105–125.
Morgan M, Shanahan J and Signorelli N (2012) Living with Television Now: Advances in Cultivation Theory
and Research. New York, NY: Peter Lang.
Nisbet MC and Huge M (2006) Attention cycles and frames in the plant biotechnology debate managing
power and participation through the press/policy connection. The Harvard International Journal of
Press/Politics 11(2): 3–40.
Peters HP (2000) The committed are hard to persuade. Recipients’ thoughts during exposure to newspaper and
TV stories on genetic engineering and their effect on attitude. New Genetics and Society 19: 365–381.
Rogers R (2013) Digital Methods. Cambridge, MA: The MIT Press.
Scheufele DA (1999) Framing as a theory of media effects. Journal of Communication 49(1): 103–122.
Testa E (2008) Tornare al nucleare? L’Italia, l’energia, l’ambiente. Torino: Einaudi. Back to nuclear? Italy,
energy, and the environment.
Valkenburg PM and Peter J (2013) The differential susceptibility to media effects model. Journal of
Communication 63: 221–243.
Venturini T (2010) Diving in magma: How to explore controversies with actor-network theory. Public
Understanding of Science 19(3): 258–273.
Venturini T (2012) Building on faults: How to represent controversies with digital methods. Public
Understanding of Science 21(7): 796–812.
Wiegman O, Gutteling JM, Boer H and Houwen RJ (1989) Newspaper coverage of hazards and the reactions
of readers. Journalism Quarterly 66: 844–852.
Author biographies
Federico Neresini teaches Science, Technology and Society and Sociology of Innovation at the University of
Padova (Italy). His main research interests are in the area of the sociology of science, in particular on construction processes of scientific knowledge, public communication of science and social representations of
science; recently, he also addressed the relationship between big data and scientific research activities, as well
as the implications of big data for the social sciences.
Andrea Lorenzet is post-doc fellow at the University of Padova (Italy). His research regards communication
processes in technoscientific controversies, with a particular focus on the development of novel technoscientific fields such as synthetic biology, Recently, he developed an interest for media monitoring, big data, and
the connections between public controversies and public engagement and their relevance to quantification
processes in the social sciences.
Downloaded from pus.sagepub.com by guest on October 9, 2014