Project report
Linking Science to Technology Bibliographic References in Patents
Volume 3
Literature Review
IMPROVING HUMAN RESEARCH POTENTIAL
AND THE SOCIO-ECONOMIC KNOWLEDGE BASE
EUR 20492
Linking Science to Technology Bibliographic References in Patents
Volume 3
Literature Review
Authors :
A. Verbeek, E. Zimmerman, P. Andries (Researchers)
Prof. Dr. Ir. K. Debackere, Dr. M. Luwel, Prof. Dr. R. Veugelers (Promotors)
In co-operation with Synes NV
Katholieke Universiteit Leuven – Incentim, Belgium
This publication results from work funded by the European Commission,
Research DG, Directorate K – Kwoledge-based economy and society under the Common Basis of
Science, Technology and Innovation Indicators (CBSTII) action of the Support for the Development
of Science and Technology Policies in Europe sub programme of the Improving Human Research
Potential and the Socio-Economic Knowledge Base specific programme of the Fifth Framework
Progamme for Research and Technological Development (contract number ERBHPV2-CT-199303).
The projects were managed by Brian SLOAN of the Competitiveness, economic analysis,
indicators Unit of Directorate K of Research DG
.
2
3
EUROPEAN COMMISSION
RESEARCH
Commissioner : Philippe Busquin
Directorate-General for Research
Director General: Achilleas Mitsos
The Directorate-General for Research initiates, develops and follows the Commission’s political initiatives for the
realisation of the European Research Area. It conceives and implements the necessary Community actions, in
particular the Framework Programmes in terms of research and technological development. It also contributes to
the implementation of the “Lisbon Strategy” regarding employment, competitiveness at international level, the
economic reform and the social cohesion within the European Union.
The Directorate " Knowledge-based economy and society" (Directorate K) contributes to the realisation of the
European Research Area in the fields of the social sciences, economic, science and technology foresight, and the
respective analyses. To this end, it monitors and encourages science and technology foresight activities, conducts
the economic analyses necessary for the work of the Directorate-General, and co-ordinates policy as regards the
relevant political, economic, human and social sciences. It prepares the European reports on science and
technology indicators, and it contributes to the development and implementation of the Framework Programmes
in these fields. It monitors the progress made in the implementation of the Lisbon strategy.
Director : Jean-François Marchipont.
Head of Unit “Competitiveness, economic analysis and indicators”: Ugur Muldur
Scientific Officer: Brian Sloan
[email protected]
http://www.cordis.lu/rtd2002/indicators.html
4
PREFACE
This report is one of a series of 9 volumes presenting the results of a study whose aim is to trace and
quantify the linkages between scientific disciplines and fields of technology1.
The study focuses on a specific form of S&T interaction: the presence of scientific research in the “prior
art” description of a patented invention. It is a direct form of S&T interaction, which may be grasped
through rich, plentiful, and directly available and accessible data: patents and publications.
Establishing this bridge enables one not only to trace the linkages between specific fields of technology
and related science disciplines, but also to measure the science intensity of such linkages (in general, a
higher number of science citations is observed when a particular technology field is more sciencebased). Moreover, as most scientific articles are published by universities and public research centres,
and most patents are granted to industry, these linkages may provide an insight into the effectiveness of
the interface between publicly funded research and the industrial exploitation of science.
The study forms a starting point for the qualitative understanding of the science-technology interaction
by revealing networks and crossroads of scientific and technological activity. It may also provide
insights into the rate and the speed of science diffusion into technology, as well as into commonalities
within the science base that are relevant across different technologies, and that can help us to
understand which technologies interact with one another to breed new hybrids.
The study is presented in 9 volumes :
Volume 1:
Volume 2:
Volume 3:
Volume 4:
Volume 5:
Volume 6:
Science and Technology Interplay : Policy relevant findings and interpretations
Methodological Framework
Leterature Review
Detailed analysis of the Science-Technology Interaction in the field of Aeronautics & Space
Detailed analysis of the Science-Technology Interaction in the field of Biotechnology
Detailed analysis of the Science-Technology Interaction in the field of Environmental
Technology
Volume 7: Detailed analysis of the Science-Technology Interaction in the field of Information Technology
Volume 8: Detailed analysis of the Science-Technology Interaction in the field of Telecommunication
Volume 9: Detailed analysis of the Science-Technology Interaction in the field of Nanotechnology
1
This study was financed by the European Commission contract number ERBHPV2-CT 1993-03 under
the activity “Common Basis of Science, Technology and Innovation Indicators” (CBSTII) of the 5th
RTD Framework Programme.
5
6
TABLE OF CONTENT
PREFACE ...........................................................................................................................................1
ABBREVIATIONS .............................................................................................................................9
GENERAL INTRODUCTION ..........................................................................................................10
PART I: SCIENCE, TECHNOLOGY AND ECONOMIC GROWTH – A SYNOPSIS OF
RELATIONS ..............................................................................................................13
CHAPTER 1 - A BRIEF INTRODUCTION TO THE LITERATURE ON TECHNOLOGICAL CHANGE AND
ECONOMIC PERFORMANCE .................................................................................................13
CHAPTER 2 - THE DIFFERENT CONCEPTS AND THEIR RELATIONS .....................................................16
I. Science and Technology ................................................................................................................................... 16
II. Research and Development............................................................................................................................. 16
III. Innovation and technical change ................................................................................................................... 17
CHAPTER 3 - THE ROLE OF KNOWLEDGE IN TECHNOLOGICAL AND ECONOMICAL PROGRESS ...........19
I. Introduction ....................................................................................................................................................... 19
II. Tacit and codified knowledge ......................................................................................................................... 19
III. Knowledge as a productivity factor............................................................................................................... 20
IV. Diffusion of knowledge ................................................................................................................................. 21
V. From the ‘linear model’ to the ‘network model’ of knowledge production, transfer and use ..................... 22
CHAPTER 4 - ECONOMICS OF SCIENCE: CONTRIBUTION AND ASSESSMENT ......................................23
I. Academic versus industrial research ................................................................................................................ 23
II. The contributions of academic research to industrial innovation.................................................................. 24
III. Assessment and rationalisation of government-supported scientific activity.............................................. 25
CHAPTER 5 - THE NEED FOR PRIORITISATION IN SCIENCE AND TECHNOLOGY ................................29
PART II: SCIENCE AND TECHNOLOGY - EXAMINATION AS TWO SEPARATE SPHERES30
CHAPTER 6 - EXAMINING SCIENCE AND TECHNOLOGY ...................................................................30
I. Introduction ....................................................................................................................................................... 30
II. Patents and publications as output indicators ................................................................................................. 32
III. Some considerations around the use of S&T indicators............................................................................... 33
CHAPTER 7 - PATENTS AND MEASUREMENT WITH PATENTS............................................................34
I. Patents, the patenting system and its economic rationale ............................................................................... 34
I.1. A patent and its economic rationale .......................................................................................................... 34
I.2. Patenting systems....................................................................................................................................... 36
I.3. Patent data sources..................................................................................................................................... 37
II. Patent statistics and trivialities around patents............................................................................................... 38
II.1. Rationale of patenting: patenting behaviour............................................................................................ 39
II.2. The Classification problem ...................................................................................................................... 42
II.2.1. The International Patent Classification (IPC) .................................................................................. 42
II.2.2. Relating patents to economic categories .......................................................................................... 42
II.3. The problem of the ‘value’ of patents...................................................................................................... 44
III. The use of patent statistics for technology measurement ............................................................................. 47
III.1. Measuring and comparing with simple output indicators; ‘foreign’ patenting .................................... 47
III.2. Specialisation indices.............................................................................................................................. 48
III.3. Impact indicators ..................................................................................................................................... 51
III.4. Maps of technology................................................................................................................................. 51
7
CHAPTER 8 - BIBLIOMETRIC INDICATORS AND ANALYSIS OF RESEARCH SYSTEMS ...........................52
I. Introduction to ‘bibliometrics’ ......................................................................................................................... 52
II. Bibliometric information: publications and data sources............................................................................... 53
II.1. Publications as data .................................................................................................................................. 53
III. Limitations to the use of bibliometric data.................................................................................................... 56
III.1 Completeness of bibliometric data .......................................................................................................... 57
III.2. Coverage of scientific literature databases............................................................................................. 57
III.3. The problem of ‘statistics’ ...................................................................................................................... 58
IV. Citation analysis ............................................................................................................................................. 59
IV.1. Introduction ............................................................................................................................................. 59
IV.2. The use of citations as ‘information’ ..................................................................................................... 60
IV.3. Basic assumption and problems around citation analysis ..................................................................... 60
IV.3.1 A general overview........................................................................................................................... 60
IV.3.2. Detailed discussion: basic assumptions underlying citation analysis............................................ 61
IV.3.3. Possible problems/pitfalls related to ‘Citation Analysis’............................................................... 62
V. Use of bibliometric data in a science policy context ..................................................................................... 65
VI. Bibliometric performance indicators............................................................................................................. 66
VI.1. Indicators of publication output and productivity ............................................................................. 66
VI.2. Impact indicators..................................................................................................................................... 67
VI.2.1. The relationship between ‘quality’ and ‘impact’........................................................................... 67
VI.2.2. Short-term vs. long-term impact .................................................................................................... 68
VI.2.3. Actual vs. expected impact............................................................................................................. 69
VI.2.4. Overview of impact indicators ....................................................................................................... 70
VI.2.5. Graphic representation of impact analyses ..................................................................................... 72
VI.3. Indicators of collaboration...................................................................................................................... 73
VI.4. Indicators of scientific activity ............................................................................................................... 74
VI.5. Intermezzo: one-dimensional vs. two-dimensional measurement and assessment.............................. 75
VI.6. Mapping of science: techniques and utility for science policy purposes.............................................. 76
VI.6.1. Mapping of science: an introduction............................................................................................... 76
IV.6.2. Several techniques ........................................................................................................................... 77
IV.6.3. Utility of mapping exercises for science policy purposes.............................................................. 80
PART III: SCIENCE AND TECHNOLOGY - EXAMINATION OF THE INTERACTION........82
CHAPTER 9 - LINKING SCIENCE TO TECHNOLOGY ...........................................................................82
I. General introduction ......................................................................................................................................... 82
II. Background on the science and technology interaction................................................................................. 82
II.1. Importance of the S&T interaction .......................................................................................................... 82
II.2. Science and technology interrelation: who leads and who follows?...................................................... 83
A. The changes in the knowledge production and diffusion system.................................................. 83
B. Discussion of the science – technology interaction in the light of cross-citation analysis........... 85
III. Implications from a policy perspective ......................................................................................................... 86
IV. Exploration of the science – technology interaction .................................................................................... 87
I. Direct/explicit S&T interrelations.................................................................................................... 89
IV.1. Citations to scientific publications in patent documents....................................................................... 89
IV.1.1. Non-patent references (NPRs)......................................................................................................... 90
IV.1.2. Differences in citation intensity ...................................................................................................... 93
IV.1.3. Interpretation of the science – technology linkage based on citations .......................................... 96
IV.1.4. Relevance of direct linkage approach from a policy perspective .................................................. 97
IV.1.5 Science – Technology linkage indicators ........................................................................................ 97
II. Indirect/implicit S&T interrelations ............................................................................................. 110
IV.2. Patents of scientific institutions and publications of industrial enterprises........................................ 110
IV.3. Tendency to integrate scientific and technological activities ............................................................. 112
IV.4. Co-activities (joint activities) between scientific institutions and industrial enterprises................... 112
IV.5. Parallel observation of patents and publications ................................................................................. 113
IV.6 Cartographical approach based on co-occurrences of publication and patent keywords.................... 114
APPENDIX I: KEY CHARACTERISTICS OF SEVERAL EMPIRICAL STUDIES ON S&T ANALYSIS..............115
REFERENCES ................................................................................................................................120
8
ABBREVIATIONS
CHI
ISI
EC
EPC
EPO
NPR
SCI
USPTO
S&T
DST
REFI
FhG
Computer Horizons Inc., Haddon Heights, NJ, USA
Institute for Scientific Information
European Community
European Patent Convention
European Patent Office
Non-patent reference
Science Citation Index
United States Patent Office
Science and Technology
Decision Support Tool
REference FIle (document citations of the European patents)
Fraunhofer Gesellschaft
9
GENERAL INTRODUCTION
The purpose of this first chapter is to set the stage for the remainder of the report, and, even more
important, to provide the reader with a number of points of departure that should be kept in mind when
reading through the rather detailed and extensive description of the existing literature in the areas of
sciento-, techno-, and econometrics presented in this report. In the early work of Adam Smith and Karl
Marx science and technical change have been considered as driving factors behind economical
progress. Adam Smith pointed out that ‘each individual becomes more expert in his own peculiar
branch, more work is done upon the whole, and the quantity of science is considerably increased by it’.
The science factor underlying growth and wealth is apparent.
Ever since the seminal works of Joseph Schumpeter (see Hanusch (1999) for a comprehensive
overview), the role of and the interaction between science and technology (S&T) in explaining patterns
of economic growth and development have received widespread scholarly attention (see for example
Nelson and Winter, 1982; Scherer, 1985; Dosi and Fabiani 1994; Freeman, 1994; Silverberg and Soete,
1994; Nelson, 1994 or more recently, Dosi, 2000). Technical progress and technical change therefore
have become fundamental issues in many studies on innovation and economic development (Freeman,
1994; Grupp, 1998; Dosi 2000).
Scientific studies have explored, developed and adopted complementary approaches to understand the
intertwined dynamics of technological progress and economic development, including the effects of the
S&T interaction on these dynamics. This S&T interaction is, however, far from linear and sequential.
Instead, it is dynamic, heterogeneous, and highly complex. Although the systems of science and
technology are since long assumed to be converging (Toynbee, 1963), this convergence is at best
complex and requires a detailed understanding of its modus operandi (Freeman, 1982). Toynbee (1963)
compared the S&T interaction to a ‘pair of dancers.’ De Solla Price (1965) uses the same simile of
dancing partners. Only in his metaphoric adaptation, they are dancing to the same music, having their
own steps. The emergence of biotechnology (Bud, 1994), a prototypical ‘sciento-technology’, has
further stimulated scientific inquiry into technology’s increasing dependence on scientific discovery.
Especially in policy oriented (research) circles, this dependence has been subject to much examination
and speculation.
Processes of ‘knowledge creation’ and the variety of possible modes of ‘knowledge diffusion’ are
elements, even central themes, in the ongoing debate on science, technology, and innovation and their
interaction (Gibbons et al., 1994). The nature of the knowledge generation process itself is evolving
towards more network-embedded structures, with more emphasis on partnerships, the interplay between
knowledge demand and supply, as well as a growing transdisciplinarity and heterogeneity of the actors
involved in the dynamics of technology convergence. The network-embeddedness of the processes of
knowledge creation and diffusion is becoming increasingly manifest in phenomena like universityindustry interactions and the advent of technoscientific breakthroughs like “nanotechnology” and
“biology in silico.”
10
Since a couple of decades, it has become obvious that the linear model of the knowledge creation,
transfer and diffusion “value chain” is no longer valid since it does not capture the current complexity
and multiplexity of the relationships involved along the chain. This is despite the fact that some
technological fields, with biotechnology as a prototypical example, are heavily based on scientific
discovery. This complex and multiplex nature of the S&T interaction triggered the theoretical
development and the empirical testing of knowledge production functions (Grilliches, 1990).
Before one will be able to model an entire knowledge production function, a more detailed
understanding of the interaction between science and technology is needed. A central issue hereby is the
quantification and the modelling of the complex web of linkages and interactions between S&T
development and progress. In this report, we attempt to describe the ‘state of the art’ on science,
technology, and their interaction.
During the past two decades, wide-ranging socio-economic and technological transformations have
caused European governments to reformulate their policies concerning government-supported scientific
activity. This reformulation has been accompanied by shifts and even complete turnarounds in research
funding across fields of inquiry as well as concerning the orientation of the research conducted (basic
versus applied). The present constraints on public expenditures in general, the enormous investments
involved in sustaining the econo-techno-scientific complex, and the actual debate on the effectiveness
of government supported scientific research, all increase the need for more accountability and
effectiveness in the area of publicly funded research (Hanusch, 1999; Ziman, 1994; Moed, 1989).
Therefore, at a policy level, disentangling the S&T interleaving is assumed to lead to considerable
insights as to how to handle those challenges. More precisely, it can lead to an ex-ante decision support
instrument and an ex-post evaluation tool.
The purpose of this report is to provide the theoretical background upon which the further discussion on
science – technology interaction can be based. The structure of this theoretical background is as
follows. The report consists of three broad parts. Part I presents and discusses some major theoretical
contributions on the interrelation between science – technology – economical development thereby
paying substantial attention to the role of knowledge as one of the production factors. Part II, one of the
key parts of this report, discusses different methodological approaches for measuring, evaluating, and as
such reviewing science and technology independently. The focus lies on patents and publications as
respective proxies of technology and science. In part III of this report, the central issue of the present
project, the science – technology interrelation, will be extensively discussed. In figure 1 we present the
internal structure of this report and the theoretical building blocks it consists of.
11
Figure 1 - Internal structure of this report
PART I
Science, Technology and Economic
Growth - A synopsis of relations
PART II
Science and Technology - Examination as two
separate spheres
Foresight
Technology
Science
measurement
PART III
Science and Technology - Examination of the
interaction
Science
Technology
12
Part I: Science, Technology and Economic Growth – A synopsis of relations
CHAPTER 1 - A BRIEF INTRODUCTION TO THE LITERATURE ON
TECHNOLOGICAL CHANGE AND ECONOMIC PERFORMANCE
Before a detailed elaboration on the actual subject of our study, the science and technology interface,
we will provide the reader with a short overview of a number of approaches, contributions, in regard
with the impact and the nature of the relation between technological change and economic performance.
It stands to reason that understanding the nature of the interaction offers a scala of possibilities for
policy makers to actively intervene in this interaction.
There is a wide consensus concerning the importance of the technical progress and its contribution to
economic development. Technical progress and change are fundamental issues in economics (Grupp,
1998). Mansfield (1969) states: ‘Technological change is an important, if not the most important, factor
responsible for economic growth’. A number of schools have developed complementary approaches in
order to understand the dynamics of technological progress and economic development.
There are several possibilities for differentiating among the existing literature that contributed to a
greater understanding of the technology – economy interaction. A distinction can be made between
Neo-classical approaches (Schumpeter, Arrow, Von Hayek and others) and the Institutional or
Evolutionary approaches (Freeman, Dosi, Pavitt, Nelson and others). An unambiguous distinction
between neo-classical and evolutionary approaches, or theories on innovation research, is not feasible.
This is partly due to the non-contradictory nature of both approaches (Grupp, 1998). Grupp used three
pairs of opposites in order to categorise the various approaches. The first pair of opposites relates to the
view on technological change as exogenous or endogenous. The second opposite concerns the
description of the economic system by equilibrium states or developing processes. The last pair of
opposites encompasses the interaction of economic actors based on rational decision theory or on selfregulating empirical processes.
The exogenous/endogenous criterion refers to the process of technology genesis and its determinants.
An endogenous approach, closely linked to the knowledge creating process, considers technological
change to originate within the economy and society. Amable (in: Silverberg & Soete, 1994) states that
endogenising the technical progress implies focussing on learning effects as a source of improvement in
technology. In a typical neo-classical approach, technical change is made endogenous because
economic agents choose to allocate certain amounts of resources to its development (for instance R&D
expenditures). However, technologies are conceived, developed and diffused under multiple economic
constraints (OECD, 1992). Nelson (1994) differentiates between two aspects of economic growth
modelling that make technological advance endogenous. First, R&D investments are profitable for
firms because they can make proprietary at least a part of the value of the increased productivity or
better product performance won through R&D. Secondly, to deal with the recognition that technology
is in a way proprietary and that support of R&D is possible only if price exceeds cost of production,
markets are assumed to be imperfectly competitive. The neo-classical growth theory acknowledges the
centrality of technical change in the growth process (Nelson & Winter, 1974)
13
The second criterion differentiates among various approaches by emphasising the time dimension of an
innovation event. The neo-classical approaches pay significant attention to the properties of the
equilibrium state but less attention to the aspects of how the system has arrived at that point. In the
evolutionary and institutional approaches, also referred to as new neo-classical approaches, the relevant
processes and pathways are defined carefully in contrast to the specification of the final state. The last
criterion deals with whether a system is regarded in terms of carefully defined, stable relationships
between its constituents, as in the rational actors approach, or that the whole cannot be so reduced.
Instead, the properties of complex systems develop from non-linear interactions between their selfregulating components.
Although neo-classicism and institutional economics share many key areas, the latter is primary
concerned with treatment of the following subjects (Grupp, 1998):
§ The formation of institutions
§ The variable relationship between the economics and legal systems
§ The effects of technical change on institutional structure
Discussing all the contributions from either the neo-classical scholars either the institutional or
evolutionary scholars, would lead us too far. However, special attention should be paid to the
contribution of Freeman and his colleagues from the SPRU University in Sussex, who are regarded as
the grounders of the Institutional innovation theory. By analysing the most intensive twentieth century
branches of the economy (chemicals, mineral oils, plastics) Freeman points out that industrial
innovative activity is always professionally managed and that technological development is increasingly
dependent on scientific knowledge. However, according to Freeman, the professionalization of the
R&D system is not the only agency that affects the increasingly scientific nature of technology. Other
factors are growing complexity, highly developed system and network techniques and the general trend
towards division of labour and specialisation. Pavitt (1984), in a study aiming at identifying and
describing sectoral patterns of technical change, introduces the ‘science based’ sector, which can mainly
be found in the chemical and electronic/electrical sectors. Innovations are directly linked to scientific
progress.
Bart Verspagen (In: Silverberg & Soete, 1994) categorises the empirical literature on the interaction
between technology and growth mainly into two streams. The first one has its roots in the notion that
R&D can be viewed as an additional production factor (see also the work of Arrow, 1994) entering the
production function much in the same way as various types of fixed capital or labour. Mohnen (1992)
discusses this approach in more detail. A second stream tries to establish an empirical relation between
knowledge accumulation and growth of output or productivity, focusing on cross-country samples.
Inter-country knowledge spill over is the most important factor leading to knowledge accumulation and
thus growth. For a detailed and excellent overview of growth literature in retrospect, we can refer to
Nelson (1994) or Dosi & Fabiani (1994) and Grupp (1998).
14
Silverberg and Lehnert (in: Silverberg & Soete, 1994) formulate four basic characteristics of innovation
and productivity growth, to which researchers of all persuasion would probably subscribe.
I.
The innovation process is inherently uncertain and irregular, and neither the exact nature nor
the timing of innovations is under the control of actors.
II.
Although the Schumpeterian description of entrepeneurship hinges on imperfect competition
and appropriability, technical progress has very rarely been monopolisable for any length of
time by a single party. Instead, it has been the joint product of the actions of many agents and
therefore only imperfectly appropriable at best.
III.
The productivity enhancing effects of innovations only become manifest during diffusion.
Investment and capital formation, learning and organisational change thus play an important
role in translating innovation into ongoing technical change. Furthermore, diffusion often
requires considerable time.
IV.
It follows from the simultaneity of diffusion processes that the economy at any time will consist
of a superposition of different technologies inherited from the past, as well as a multitude of
‘cutting edge’ technologies competing for dominance on the frontier. The nature of competition
determines replacement dynamics and choice of technique.
One of the conclusions based on these four general points is that innovation in a multi-actor and thus
dynamic environment is to a certain extent uncontrollable. Furthermore, the effects of innovation on
economical growth become manifest only after diffusion of knowledge incorporated in the form of an
innovation. Knowledge and the diffusion of knowledge are thus crucial in realising economic growth.
Stoneman (1987) states that technology will have impact on productivity, trading performance,
employment, investment, income distribution, quality of goods, growth, inflation, environment, security
and defence and the industrial structure of an economy. Some impacts may be beneficial while others
may be harmful. The effect on national growth in GDP (per capita) has been discussed in detail by
Fagerberg (1994). He states that national growth curves are determined by the level of education and
training, and by the level of R&D, consisting of scientific research and technological development.
Narin (1994), who considered publications and patents as proxy-measures of knowledge production,
already found this positive effect of R&D on Gross Domestic Product. Only by understanding the
whole mechanism can policy makers construct policies to discourage harmful effects and to encourage
beneficial effects.
According to the 1998 report of The National Science Board on science and engineering indicators, the
relationship between science and technology on the one hand, and economic development on the other
hand, appears to be self-propelling. Science and technology lead to competitiveness and commercial
success. This in turn generates additional resources that can be spent on more scientific research and
technological development, thus closing the circle.
15
CHAPTER 2 - THE DIFFERENT CONCEPTS AND THEIR RELATIONS
I. Science and Technology
The first issue to be addressed concerns the definition of both science and technology. The term science
is understood to cover the creation, discovery, examination, classification, reorganisation and
dissemination of knowledge on physical, biological or social subjects. In a recent study carried out by
Senker et al. (1999), in the context of a TSER-project on the European comparison of Public Research
Systems, research has been defined as ‘original investigation undertaken to acquire new knowledge in
the natural sciences, social sciences and humanities’.
Technology is the science application know-how. In other words, it emphasises the blueprint of the
application of knowledge. As such, it belongs to a larger group of activities that embrace the creation
and use of artefacts, crafts and items of knowledge as well as various forms of social organisation
(Grupp, 1998). Mansfield (1969) points out that it is important to distinguish between technological
change and scientific advance. Pure science is directed towards understanding, whereas technology is
directed towards use. Technological change or evolution as a result of inventions is not necessarily
based on new scientific principles. It can also be based on alternative evolutionary use of existing
knowledge.
One of the key terms in the above-presented definitions is ‘knowledge’. Science includes processes of
knowledge creation and diffusion. Technology on the other hand focuses on the application or usage of
the created knowledge. The research process is of major importance for the materialisation of
innovations. This is a widely accepted principle. In present research statistics, a distinction can be made
between fundamental or ‘basic’ research, applied research and experimental development. These three
approaches are often combined under the heading ‘research and development’ and will be discussed
further in the next paragraph.
II. Research and Development
‘Research and Development is systematic, creative work that advances the state of our knowledge,
whether in connection with man, culture or society and uses this knowledge to identify new potential
applications’ (Grupp, 1998). The Frascati manual issued by the Organisation for Economic CoOperation and Development (OECD, 1994b) provides definitions and conventions for the measurement
of research and experimental development (R&D) aiming at a form of European standardisation. In the
Frascati manual, R&D comprises the creative work undertaken on a systematic basis in order to
increase the stock of knowledge, including knowledge of man, culture and society, and the use of this
stock of knowledge to devise new applications. This definition is very similar to the definition used by
Grupp (1998). As mentioned above, R&D covers three activities: basic research, applied research and
experimental development. These three activities will be discussed below.
Basic research or fundamental research refers to experimental or theoretical work, geared ‘primarily’
to the acquisition of new knowledge about the basic origin of phenomena and observable events without
targeting a particular application of use. It focuses on the advancement of knowledge without any
economic or social targets and it is certainly not dedicated to solving practical problems. The term
16
application-oriented fundamental research is used in situations where basic research targets certain
areas of general interest or is focused in their direction.
Applied research is predisposed to specific and practical purposes or objectives. It also includes a new
knowledge generation process but always in regard of the practical application. The results of applied
research are intended to be valid for a limited range of products of processes.
Experimental development is systematic work structured on existing knowledge with either research
of empirical origin, and which is directed towards production of new materials, products, equipment or
the installation of new processes, systems or services.
Research and development is perceived as the driving force behind the innovation event. Science and
technology need to be distinguished from research and development. Scientific research, in the sense of
research carried out by academia, can be regarded as taking place outside the company, in the public
domain (cfr. section 5.I). The latter contrasts with research and development itself, which can be
attributed more to the private domain (Grupp, 1998). In view of the impact of knowledge production on
the economic growth, discussed extensively among others by Arrow, Amable and Verspagen (In:
Silverberg & Soete, 1994), R&D can be regarded as an identifiable economic activity. Due to its
characteristics, science is increasingly being reviewed as an economic activity as well, as it can lead to
the introduction of return requirements (financial and non-financial) on public and private funding of
science (see chapter 5). For a more detailed discussion of R&D in terms of activities covered, statistical
measurements and budgeting issues, we refer to the Frascati manual (OECD, 1994b).
III. Innovation and technical change
A third important concept related to science and technology in a broader sense, is innovation. Several
definitions on innovation can be found in literature. Grupp (1998) defines it as follows: ‘Innovation
relates to the attained quantity of ideas’. Verspagen (In: Silverberg & Soete, 1994) associates
innovation with an original contribution to the stock of knowledge in the economy. Innovations can
occur in the form of new consumer goods, new production of transport methods, new markets or new
organisations. In the Frascati manual (OECD, 1994b) innovation concerns new products and processes
and significant technological changes in product and processes. An innovation is implemented, if it has
been introduced on the market (product innovation) or used within a production process (process
innovation). Innovations therefore involve a series of scientific, technological, organisational, financial
and commercial activities.
It is beyond discussion that innovation can basically be characterised by new ideas, concepts,
applications and, generally speaking, new knowledge finally contributing to a new product of process,
including service-processes. The result of the innovation process, a new product or process, can be
characterised as ‘innovation’. As Grupp (1998) points out, the novelty concept can be regarded as a
synonym for the innovation concept. The R&D activity is just one of the activities of the complex
process of innovation. R&D can be used in any phase of the innovation process, not only as an original
source of inventive ideas, but also as a catalyst for solving problems that can occur at any stage in the
implementation process. Besides R&D as an activity in the whole innovation process, six fields of
innovative activities are often distinguished (OECD, 1994b):
17
§
§
§
§
§
§
Tooling-up and industrial engineering
Manufacturing start-up and pre-production development
Marketing for new products
Acquisition of disembodied technology
Acquisition of embodied technology
Design
Another concept, closely linked to innovation, is imitation. Imitation can be understood as the diffusion
of knowledge to agents other than the original innovator. As we shall also see in the discussion of the
different theories on technology and economic growth in the next chapter, innovation and imitation
represent different aspects of technological change and also lead to different economical results.
Research and development is associated with both innovation and imitation, as it can be directed
towards the creation of new knowledge and/or imitation. This also implies that behind R&D lies a set of
profound choices. Verspagen (In: Silverberg & Soete, 1994) offers another view on innovation in
relation to imitation. He connects imitation to knowledge spill over and innovation to patenting and
knowledge creation. The most important criterion for an innovation to be patented is undoubtedly
‘novelty’.
This brief discussion of the concepts allows us to conclude that a clear and unambiguous distinction
between the concepts is a rather difficult task. This is partially due to the fact that the explanation of the
concepts is largely contingency-based. Science and technology are two different sides of the same coin.
Science aims at the expansion of the knowledge base, whereas technology refers to the application of
knowledge in order to realise an innovation, a new product or process. R&D, which is only one of the
activities of the process of technological innovation and clearly has an economic impact, seems to be
the trigger for research and at the same time for the creation of new applications. Finally, all this may
result in a single innovation, the result of the whole process of technological innovation. In the
subsequent discussion, this will be the context in which the above-mentioned concepts will be used.
18
CHAPTER 3 - THE ROLE OF KNOWLEDGE IN TECHNOLOGICAL
AND ECONOMICAL PROGRESS
I. Introduction
As we have seen in the concepts of science, technology and innovation, knowledge plays an important
role in technological progress and subsequently in economic growth. Knowledge is undoubtedly
connected with the idea of technical advance. According to Gibbons et al. (1994) the very nature of
knowledge is evolving to a more network-oriented structure, with greater emphasis on strategic
alliances, knowledge demand and supply chains and a growing transdisciplinarity and heterogeneity. As
a consequence, the identification of scientific priorities in the actual socio-economic environment
becomes increasingly complex.
The term ‘knowledge society’, recently introduced by Drucker (1993), reflects the increasing
importance attached to knowledge as the key resource for progress, and not just as another resource
alongside traditional factors of production - labour, capital and land (cfr. section 4.II.). The increasing
importance of knowledge as a competitive resource has stimulated and enhanced the debate on the
mechanisms and processes by which knowledge is created, both at individual and organisational level
(Nonaka and Takeuchi, 1995). Understanding these mechanisms is a complex challenge, given the
blurred boundaries between tacit and codified knowledge and their complementary and dynamic nature.
II. Tacit and codified knowledge
Codified knowledge is knowledge that is ‘transmittable in formal, systematic language’ (Senker, 1993,
1995). It is mainly embodied in manuals, textbooks, scientific and technical journals, technical
specifications of materials or components, operating manuals for commercial process plant and research
equipment. Codified knowledge is an important component of knowledge creation, although it is not
easily accessible by individuals and organisations. Martin et al. (1996) point out that in order to
interpret en derive value from the growing pool of codified knowledge, organisations have to invest
substantially in research capacity.
Codified knowledge becomes increasingly available and accessible. Due to the advances in information
technology (David and Foray, 1995) and the new combinations of existing stocks of knowledge stored
in different electronic resources, a significant variety and novelty may be generated. The process of
codification is thought to be diminishing the conventional borders between science and technology and
to be increasing the extent of knowledge formalisation, with direct impact on the production of new
products and processes, facilitated by advanced simulation in the design process.
Codified knowledge is complemented by tacit or person-embodied knowledge, of which the essence
can de defined as ‘One can know how to do things, without necessary being able to describe how’
(Arrow, 1994). The increased use of computer simulations and information technologies can be seen as
a means of distributing tacit knowledge and person-embodied skills, rather than a new source of new
codified knowledge. Senker (1995) argues that there are limits to the codification of knowledge, the
process of conversion of tacit knowledge into codified knowledge. These limitations have a number of
reasons, including: the difficulty of acquiring specific scientific or technological expertise within
specific firms or sectors, the emergence of new technologies, and the need for interaction between
producers and users of new technologies in order to appropriate tacit knowledge, skills and techniques.
19
Scientific research, especially through basic research activities, was considered to primarily produce
codified knowledge, which could afterwards be used by other individuals or firms as a ‘public good’.
Several authors claimed the importance of tacit knowledge and skills, accumulated during university
education and further developed through industrial training, especially in emerging technologies and
fast-moving scientific fields. Large amounts of tacit and codified knowledge are acquired by universitytrained scientists, researchers and engineers during their university education, and are further enhanced
during industrial in-house training provided by the firms who recruit them. For an excellent discussion
of the knowledge creation process on company level, including group-level, we refer to the work of
Nonaka and Takeuchi (1995).
III. Knowledge as a productivity factor
From an economic modelling point of view, knowledge can also be seen as a productivity factor, in
addition to the important factors labour and capital. Knowledge as a productivity factor can be related
with input and output processes. Kenneth Arrow (1994) discusses knowledge as one of the major input
factors for the production of goods. This principle can also be applied to the production of services.
Know-how about how to produce a certain product or service is a critical success factor. On the other
hand, knowledge can also be regarded as output, a result of the transformation of goods. When
producing a product or service, knowledge increases about the production process and its results, being
the product or service.
From a systemic point of view, this also increases the effectiveness of the feedback actions, thus
enabling double loop learning2. Knowledge can therefore be seen as a by-product of the transformation
process. In order to produce new knowledge, additional resources have to be combined, such as capital
and labour and of course existing knowledge. Devoting more resources to the production of knowledge
would then increase the rate of growth of the knowledge stock. However, an exogenous stock of
knowledge exists as well, in the form of public knowledge. For example, the knowledge creating
function of a university is partly based on the further development of existing knowledge. A large part
of basic science is self-generated. But the resources devoted to science play an important role. And this
is precisely the issue that is subject to policy decisions.
In this context, it is essential to recognize the need for a sufficient and well-composed knowledge base
that is necessary for acquiring and employing external knowledge in an effective and efficient way. It is
important for a country or region to have a critical mass of research and production competencies.
Especially in the case of exogenous knowledge input – for example through foreign investment in a
country – a critical knowledge mass is indispensable in order to ‘absorb’ this external knowledge and
‘create’ additional knowledge on this basis (‘absorptive’ capacity). Without building a certain scientific
level, a country cannot adequately participate in science-based technology (Schmoch et al., 1993). This
finding was confirmed by Tidd & Brocklehurst (1999) in their study of the Malaysian government
policy on high-tech stimulation. The elaborated government support for high-tech industries remained
without effect, due to the fact that a critical mass of local knowledge was lacking. It takes serious
efforts and time to build this critical knowledge mass in a specific industry sector.
2
Double loop learning is a concept linked to the control cycles. A control cycle of a production process consists of the
activities: measuring input/output, evaluating, interfering and norming that is the generation of norms or parameters in order to
control the process. A fifth activity that has the biggest learning effect and improves the whole control process is ‘evaluating’.
Evaluation concerns the discussion and review of the existing norms based on experience and results. This last activity creates
a double loop learning cycle (Van Amelsvoort, 1992).
20
The previous discussion of knowledge makes us conclude that knowledge is a multidimensional
concept, which makes it inappropriate to only identify knowledge with the factor productivity. Various
studies have looked at the positive effects of knowledge creation for the firm through scientific and
technological activities. Deeds, DeCarolis & Coombs (1999) found that new product development
capabilities are enhanced by the quality of the scientific team, under the condition that the scientist’s
attention is not distracted from research by involvement in management issues. In an earlier study
(Deeds, DeCarolis & Coombs, 1997), they found that the money raised in an initial public offering
(IPO) was positively influenced by the quality of the scientific team, measured by the number of
citations to their work. This has also been confirmed by McMillan, Narin & Deeds (2000). They
attributed the importance of science for individual companies to the fact that social norms in science
offer better knowledge protection than legislation. They also indicated the importance of social
networks, with special attention to the role of joint research with the academic world.
Allen, Booz & Hamilton (1982) found that the share of innovations in companies’ profits surged from a
modest 20% in the 1970s to 33% in the 1980s. However, when emphasising the importance of internal
knowledge creation, we cannot forget that process or product innovation, as an outcome of internal
knowledge creation, is not the only way for firms to gain success. R&D is not the only factor through
which a firm can achieve growth. Intelligent use of knowledge spillovers, leading to imitation, is
another. This leads us to the more general methods of knowledge diffusion (cfr. infra).
IV. Diffusion of knowledge
Another important aspect in technological progress and evolution is the actual spread out of knowledge.
Firstly, diffusion of knowledge enables technological progress, due to the availability of knowledge and
its possible applications. Secondly, available knowledge will be used as a basis for creating new
knowledge, re-generation of knowledge. Tacit or personal knowledge is, in contrast to codified
knowledge, complex to transfer. There are several mechanisms of diffusion (Arrow, 1994):
§ Through some mechanical means
§ As a result of deliberate policy
§ By special activities of people in order to obtain knowledge
With regard to the second group of mechanisms, there is one specific mechanism that should be
emphasised: imitation. Verspagen (In: Silverberg & Soete, 1994) also discusses the importance of
imitation for the spillover of knowledge. The firm goes out and looks at other firms and finds out what
they are doing and what can be good for them. Firms watch their more sophisticated competitors in
order to absorb and use the techniques of the latter ones. Not all knowledge can be diffused in an easy
manner. Tacit knowledge, which is the more personalised knowledge, will be more difficult to transfer
than codified knowledge, the more formalised knowledge. Learning by doing provides a possibility for
transferring tacit knowledge. Innovation and imitation represent different aspects of technological
change and also lead to different economical results. Research and development is associated with both
innovation and imitation, as it can be directed towards the creation of new knowledge as well as
towards imitation – see for example the practise of 'reverse engineering'.
21
V. From the ‘linear model’ to the ‘network model’ of knowledge production,
transfer and use
The traditional understanding of the contribution of basic research to industrial innovation is related to
the ‘linear model’ of knowledge production and transfer. Steinmueller (2000) links this model to the
view of science as a Social Instrument: a social investment in the production and dissemination of
knowledge that is expected to generate economic returns as this knowledge is commercially developed
and exploited. According to this approach, basic research is a source of ideas, theories and discoveries,
that are further developed through applied research, then tested as a part of the ‘development’ process
and translated into industrial innovations. Finally, through various stages of production a further
materialisation in new products and processes takes place.
This approach has been the basis for many economic studies and policy analyses. Some industrial
sectors like biotechnology or the pharmaceutical industry, which rely heavily on advances in basic
research, still reflect to a large extent the stages of the ‘linear model’. The usefulness of this model lies
mostly in its functioning as a starting point, heuristic for further research and as a way of understanding
the function of knowledge as input for industrial innovations. However, the linear approach is an
oversimplified representation, ignoring the evidence that technological change is often built on
experience and ingenuity, divorced from scientific theory or method and ignoring the role of
technological developments in motivating scientific explanation and the technological sources of
instruments for scientific investigation (Rosenberg, 1982)
In the light of more recent studies, the relation between academic research and industrial innovation has
been considered to evolve according to much more complex patterns of knowledge production, transfer
and use, which the ‘linear model’ fails to capture. For instance, Steinmueller (1994) argues that the
‘linear model’ does not reflect the economic and social determinants of scientific research activity and
overlooks the influence of technology on the scientific agenda.
The ‘network model’ of knowledge production, transfer and use is likely to characterise more
adequately the complex interactions between knowledge producers and users. Analysing the relations
between basic research and industrial innovation, Steinmueller (1994) stresses the fundamental links
between basic research and the economic and social determinants. He considers the fact that the
‘network’ approach opens new and useful economic perspectives, like the increasing network value
with the number of participants, the decreasing rate of overlapping research projects through network
centralisation, and the complementary investments for information dissemination that may lead to
economic benefits. Information flows within the network appear as more easily accessible by
governments and firms, increasing their choices about specialisation, co-operation and competition. The
network model can be associated with a view of science as a Social Institution. Science is assumed to
be a social institution of which the norms and practices are distinct from, and only partially reconcilable
with, the institutions of market (Steinmueller, 2000).
22
CHAPTER 4 - ECONOMICS OF SCIENCE: CONTRIBUTION AND
ASSESSMENT
I. Academic versus industrial research
The views on research presented so far have focused on the academic and industrial research
communities without explicitly differentiating between those specific dimensions/activities. In the
discussion of the concepts, we highlighted that R&D activities are more related to the private domain,
in contrast to science, which is more related to the public domain. In this section, we will focus on
science that is carried out by the public sector and financed with public funds. Universities and other
higher education institutions can be considered as institutions in the public domain.
Among the two basic functions, education and research, which universities and other higher education
institutions perform and which are both essential in the process of innovation, it is academic research
that is addressed specifically within the scope of our analysis. A further distinction will be made
between two aspects of academic research, basic and applied research. This distinction comes mainly
from their relation to commercial purposes.
In the spirit of a traditional ‘division of labour’ between industry and academia, the largest part of
academic research is concentrated in basic research, although considerable applied research efforts are
undertaken in many academic departments, such as engineering, new materials or computer sciences.
Basic research is considered to encompass:
‘...both ‘curiosity-oriented’ research (experimental or theoretical research undertaken
primarily to acquire new scientific or technical knowledge for its own sake) and ‘strategic’
research (undertaken with some instrumental application in mind, although the precise process
or product is not yet known).’ (Martin et al., 1996: 2)
In ‘Industry’, R&D function of a firm, the same distinction between basic and applied research can be
used (cfr. section 2.II.). It has to be stressed that the importance attached to performing basic research
by companies lies in acquiring ‘a ticket of admission to an information network’ (Rosenberg, 1990) or
in a prerequisite to better assess and perform applied research:
‘For one thing, firms often need to do basic research in order to understand better how and
where to conduct research of a more applied nature... For another thing, a basic research
capability is essential for evaluating the outcome of much applied research and for perceiving
its possible implications... In an even more general sense, a basic research capability is often
indispensable in order to monitor and evaluate research being conducted elsewhere... A firm is
much less likely to benefit from university research unless it also performs some basic
research.’
(Rosenberg, 1990: 170-1)
A sharp distinction between basic and applied research is often very difficult to make, given the very
high degree of interaction between them, especially in some areas and disciplines, like health, medicine
and agriculture. Pavitt (1997) distinguishes between academic and industrial research and development,
by examining their purposes. In his view, academic research as a main source of codified knowledge is
aimed at explaining and predicting reality, while industrial research and development is mainly
concerned with the design and development of artefacts, focussing on the more practical application.
23
There is a strong interaction between academic and industrial research. Academic research increases the
research capacity of business research to solve complex problems and challenges.
II. The contributions of academic research to industrial innovation
In this section we will discuss how academic research, public research carried out by universities,
contributes to technological progress. Following the title of this chapter, we will focus on the economic
aspects, that is the aspects that influence economical progress through industrial innovation. Martin et
al. (1996) classified the various forms of economic benefit from academic research into six broad
categories, interconnected and mutually supportive. The six categories are:
1.
2.
3.
4.
5.
6.
Increasing the stock of useful information;
New instrumentation and methodologies;
Skilled graduates;
Professional networks;
Technological problem solving;
Creation of new firms.
The importance of academic research in increasing the stock of useful information has been addressed
in the previous section. Furthermore, we have to take into account the complex interaction process
between the tacit and the codified knowledge that is transferred across University-Industry institutional
borders. Traditional representations of this contribution tend to draw implicitly upon the ‘linear model’
as offering new technologies for use in industry, but the greater emphasis here on tacit knowledge
transfer implies a greater emphasis on the mobility of skilled graduates that enable diffusion of tacit
knowledge. As Meyer-Krahmer and Schmoch (1998) point out, there is a ‘two-way interaction’
between public and private knowledge generation and diffusion (cfr. section 4.IV.)
New instrumentation and methodologies play an important role both as a source of scientific progress
and as one form of economic benefit from academic research. Rosenberg (1992) considers scientific
instruments as ‘capital goods’ of the scientific research industry, of which the development cannot be
separated from advances in basic research. He identified two main diffusion flows of instruments:
firstly, from one scientific discipline to another, and secondly, from basic research laboratories to
industry. Scientific instruments have become almost indistinguishable from industrial capital goods
(ibid.).
The third benefit lies in skilled graduates, who not only bring knowledge of recent scientific research
into industry, but who also have an ability to solve problems, perform research and develop new ideas.
These skills, developed during their education, may be valuable. The process of graduates entering an
industrial environment needs large investments (for example training in industrial practice).
The provision of access to professional networks is a major benefit of academic research. Government
funding allows individuals and organisations to participate in the world-wide community of research
and technological development. Within the professional networks, publications are very important not
only as a source of codified knowledge, but also for the tacit knowledge, which they signal.
Publications help to consolidate the scientific credibility of researchers within firms and serve as a
vehicle in the relation with academics, thus acting as an ‘entry ticket’ to professional networks (Hicks,
1995).
24
Rappa & Debackere (1992) showed that, although academics are more willing to publish their findings,
industrial actors also seek an exchange of ideas within their technological communities. Network
relations between academic and industrial actors are profound.
Technological problem solving is the most widely recognised benefit from academic research. The
extent of interaction between academic research and technological problem solving, is highly variable
across research fields and industrial sectors, coming from their differing origins and development.
A positive influence of academic research on the creation of new firms cannot be generalised (Salter &
Martin, 1999), despite the substantial and varied links between different industrial fields and
universities that sometimes lead to the creation of new companies aiming at further (commercially)
exploiting an innovation (spin-offs, join ventures etc.). It cannot be concluded unambiguously that
significant investment in basic research generates spin-off companies even if the correlation between
university research and firm birth is positive.
Martin et al. (1996) have identified three general methods for measuring the economic benefits of
academic research: a) econometric studies; b) surveys and c) case studies. Econometric studies focus
on large-scale patterns and are effective only in providing an aggregate picture of statistical regularities
and in estimating the rate of return on research and development expenditures. A big disadvantage of
this type of studies is that the results might be misleading due to the simplistic and unrealistic
approaches. Surveys analyse the extent to which government-funded research constitutes a source of
innovative ideas for firms. Surveys have focussed on the interaction between academic research in both
its aspects (basic and applied) and industry. Case studies are the best tool for examining the innovation
process directly. They generally provide support for the main findings from econometric studies and
surveys.
Until recently, few attempts were made to measure the rates of return on funded research and
development. Mansfield (1991) made substantial contributions in measuring these benefits; he focussed
on research within 15 years of the innovation under consideration. Using a sample of 76 US firms in
seven industries, he found out that 11% of new products and 9% of new processes could not have been
developed without academic research. Furthermore, he estimated the rate of return form academic
research to be 28% (ibid., p.10). Narin et al. (1997) have developed a new approach to evaluating the
benefits of publicly funded research based on analysing scientific publications cited in US patents. For
42.000 papers with at least one US author, they determined the sources of US foreign research support
acknowledged in the papers. Their conclusion was that, based on the increasing number of scientific
references cited in patents, the knowledge flow from US science to US industry has multiplied. Of
course some critical notes can be placed.
III. Assessment and rationalisation of government-supported scientific
activity
During the past two decades, wide-ranging socio-economic and technological transformations have
caused European governments to reformulate their policies for public sector funding (PSF). Such
policies affect the organisation and location of research, the prioritisation in specific fields, the agencies
responsible for funding research, and the mechanisms for allocating research funds. Determining the
principles governing the allocation of resources to science as well as the management and consequences
of the use of these resources, are the central issues of the economics of science (Steinmueller, 2000).
25
In a study performed by Senker et al. (1999) on Public Science Research (PSR) among 12 European
countries, clearly illustrated that governments had either explicit or implicit expectations that their
investments in PSR would achieve some of the following functions (in relative importance):
i.
The advancement of knowledge
ii.
The support of policy transformation and implementation
iii.
The support of public welfare (e.g. health, environment etc.)
iv.
The support of economic development (technology transfer)
v.
Programmes to build and support prestige activities and capabilities in ‘frontier’ science
Furthermore, it appeared that all the countries traditionally investing in PSR aimed at advancing
knowledge and promoting economic development, being central pillars in their science and technology
policies. This illustrates the awareness of the impact of knowledge creation and distribution on the
economical progress. Another interesting result of the study was the apparent growth in resources for
‘application-oriented’ research in new and key technologies, including biotechnology and the
environment. It appeared that there have not only been shifts in allocation between fields of research,
but also changes in the type of research funded.
Let us take a look at the possible explanations for the changes in this funding behaviour. Beside the socalled ‘sophistication factor’ (Martin, 1996), i.e. the growing costs conducting scientific research −
instrumentation, facilities and infrastructure to conduct frontier research became more expensive −, two
main reasons can be identified for the fact that assessments of government-funded research have
become increasingly necessary in recent years. The first reason has to do with the perceived increasing
constraints on public expenditure, including spending on research and, as a result, the requirement for
greater public accountability in all areas of public expenditure. The second reason relates to the
increasing pressure on the peer review system that was normally used to address these types of
problems (Martin, 1996; Moed, 1989). Both reasons will be discussed in more detail below.
§ Increasing constraints on public expenditure and requirement for greater accountability
During the 1960s, funding of scientific research occurred in a relatively unrestricted way in most
western countries. How to allocate an ever-growing science budget in a sensible way appeared to be
the main problem. However, from the start of the 1970s on, greater selectivity concerning the
allocation of research funds had to be displayed by government and industry (Moed, 1989). Science
was no longer seen as cheap. Moreover, it does not offer any immediate return to an investor. As a
result, science, requiring a total layout of several percent of a country's Gross National Product, found
itself in serious competition with other desirable items of public or private expenditure. Science entered
in a ‘steady state’. As such, funding of scientific research has to be justified in this context (Ziman,
1994).
Since the resources needed to develop a ‘science base’ can no longer be regarded as a marginal item in
the national budget, science will have to compete with rival institutions, if it aspires to gain a bigger
piece of the pie (Ziman, 1994). In this context, demands for evaluation and for performance indicators
are put forward to assure both the government and the public that public money is being spent well
(Martin, 1996).
At the end of the 1980s and in the 1990s, although the economic situation in many western countries
improved significantly, the general attitude on the funding of scientific research activities did not
change very much. Nowadays, there is less willingness to fund research ‘carte blanche’. As Moed
(1989) indicates, ‘value for money’ is often the main objective. Greater emphasis is placed on
26
selectivity and competition of research programmes for funding. Consequently, systematic evaluation
and monitoring of scientific programmes became customary. Under these circumstances, the publicsector funding agencies search for a more efficient resource allocation to scientific and technological
activities, for better meeting the nation’s economic and social needs3. If science is instrumental in
technological progress and ultimately in economic growth and prosperity, as indeed we have seen, then
the economic theory of resource allocation should be applicable to science (Steinmueller, 2000).
§ Pressure on the peer review system
In order to be able to obtain a truthful evaluation and monitoring of a particular scientific programme,
information on, for example, the quality or benefits of such programmes is needed. Traditionally, this
information was provided by the scientists themselves, and this seemed to work well in periods such as
the 1950s and 1960s, when government spending was increasing by five to ten percent a year. However,
the pressure on the peer review system increased in periods that were characterised by an essentially
level budget.
Evaluation of scientific research was always done in a rather qualitative way, mostly by means of expert
panels. Peer judgements are assessments of the ‘quality’ or ‘significance’ of scientific research
performance, as perceived by experts in the particular field involved (Moed, 1989). Peer evaluation is
thus inherently based on scientists' perceptions of contributions by other researchers. As such,
statements of well-informed experts are clearly prone to subjectivity (Tijssen, 1992).
Judgements from the peer review system have been criticised on several occasions, both from a
reliability and a validity point of view. In short, Martin (1996) identifies three main problems. First,
due to the political and social pressures within the scientific community, such as the increasing
competitiveness and the concentration of research resources in an ever-smaller number of large research
centres, it becomes more difficult to find truly impartial peers. Second, peers in different cognitive and
social locations may evaluate scientific contributions quite differently. Third, peers will often base their
judgement on limited or imperfect information on the contribution being evaluated. As a result of these
and other problems, their perceptions may provide no more than a partial indicator of scientific
contributions.
Nowadays, as research funding is supposed to cover wider areas of scientific activity, the expertise of
scientific peers is considered to be a base too small and too subjective to found allocation decisions on
(Tijssen, 1992). As a consequence, the need arose to develop a method for evaluating relevant aspects
of the input, throughput, and output of scientific research in a more ‘objective’ − that is, a more
quantitative − way.
Hence, analytical tools based on readily available quantitative data such as, in particular, bibliometric
information, were developed to be used complementary to expert judgements (Tijssen, 1992; Moed,
1989). Since the beginning of the nineties, a considerable amount of experience has been gained in
applying bibliometric analysis4 in the assessment of research performance (Van Raan, 1997).
Following the trend towards an increasing quantification of science, the use of bibliometric analysis for
the evaluation and monitoring of scientific outputs has become widespread (Tijssen, 1992). Partly due
3
The study by Senker et al. (1999), pointed out that most countries seem to focus their science policy on four main themes:
evaluation, coordination, prioritisation and technology transfer. Especially prioritisation is developing slowly through
‘Prospective’, ‘Forecasting’ and ‘Foresight’ activities.
4
Bibliometrics and the different techniques involved for measurement and analysis of science will be the central issue of
chapter 4)
27
to the existence of large computerised bibliographic databases, bibliometric research performance
analysis can be characterised as a methodologically and technically sophisticated field, as shown by the
variety of indicators and its application to different levels of aggregation. Recently also, more and more
bibliometric applications become apparent in fruitful combination with assessments by means of peer
review (Van Raan, 1997).
Science as a social institute has a high impact in many parts of society. As such, the assessment of
scientific activity and the prioritisation in science policy become more and more a public subject.
Certain European countries, especially The Netherlands and Scandinavia, which have a tradition of
public participation, are involving the public more in debates and priority setting on scientific and
technological issues. This has been described as a ‘post modern’ research system (Rip, 1992).
Apparently, apart from the increasing globalisation and world economic competition, a number of
drivers has put increasing pressure on government spending. Among them are the political dimensions,
international and EC influences (looking at Europe), industrial needs and the emergence of new
technologies (IT, biotechnology and new materials) leading to a shift in funding patterns and thus
priorities. Beyond that, governments are also facing hard constraints to balance their budgets, to
decrease the public spending allocated to scientific research and to cut taxes, which generates a high
demand for greater public accountability and better ‘value for money’.
28
CHAPTER 5 - THE NEED FOR PRIORITISATION IN SCIENCE AND
TECHNOLOGY
The variety of approaches on the relationship between science, technology and economic growth
proclaimed by the different scholars in the field, all share the statement that technical progress, in one or
another way, is seen as very important for economic growth. The creation and the use of knowledge,
generally referred to as science, plays a dominant role in technical progress.
A number of studies have pointed out that the relationship between the knowledge production, transfer
and usage, broadly speaking the view on the relation between science and technology, has shifted from
linear to reciprocal or network oriented. The ex-ante identification of scientific priorities in the actual
socio-economic environment, and the ex-post justification and evaluation of government support,
becomes increasingly complex.
The growing impact of emerging technologies, such as ICT, electronics, advanced materials, etc. on the
socio-economic development has added new dimensions to globalisation and world industrial and
economic competition. The emergence of knowledge-based industries, firms and services, generates an
enormous competitive pressure on international markets. The emergence of a new ‘long wave’ or
Kondratieff cycle of economic development (Freeman and Perez, 1988) has raised and is still raising
complex challenges for a better understanding of ways in which progress in new technologies is
science-dependent.
On the other hand governments face hard constraints to balance their budgets, to decrease public
spending allocated to scientific research and to cut taxes, which in turn generates a high demand for
greater public accountability and better ‘value for money’. The simple interaction between demand and
supply, which works fairly well for products with a clear cost and benefit structure, fails, due to the
absence of supply in the case of science. Governments therefore have to intervene and correct the
market imperfections. Martin (1996) mentions two reasons why, in the case of government-funded
research, objective evaluation seems to be necessary.
The first reason relates to the growing costs of scientific instrumentation, facilities and infrastructure
required to conduct frontier research – the so-called ‘sophistication factor’. The second reason concerns
the emerging problems, as Martin (ibid) perceives them, with ‘peer review’. He pointed out that peer
review groups worked well when government spending was increasing by 5-10% a year. New and
promising areas of research could be identified successfully, but the results are less satisfactory when it
comes to identifying declining areas and groups. Furthermore, peer review groups cannot fulfil the task
of showing public accountability in government spending, as they cannot be seen as totally unbiased
themselves. Thus, other instruments on which evaluation can be based are more than welcome.
Policy interventions also focus on a broader social integration of science and technology issues, which
should not be accessible exclusively to scientists and engineers, and which should not be reduced to
those technologies with a well-defined economic impact on productivity growth or supply and demand
mechanisms. The social shaping of science and technology issues also raises the point of the policy
choices that need to be made in relation to the endogenous nature of technological change, stemming
from a given economic and social system, and not as a consequence of factors external to this system.
Again, this points towards the need for a larger comprehension of the factors shaping the science and
technology interaction.
29
Part II: Science and Technology - Examination as two separate spheres
CHAPTER 6 - EXAMINING SCIENCE AND TECHNOLOGY
I. Introduction
Understanding innovation presupposes an understanding of S&T activities. Measurement of S&T
activities enhances the understanding of the S&T interaction and provides useful insights for policy
makers. One major concern of analysts is to describe S&T activities in qualitative as well as in
quantitative terms, so that the indicators can be used in the context of models, explicit or implicit.
However, the general problem is that S&T can only be measured indirectly, using input, output of
impact indicators (OECD, 1994a).
Measuring S&T activities should not be performed solely in a quantitative manner but also in a
qualitative manner. However, the importance of quantitative assessment should not be underestimated.
Martin (1996) pointed out that quantitative assessment of government-funded research is important and
necessary, also in view of the raising concerns around peer review groups (qualitative). Both forms of
measurement are necessary in order to perform a balanced evaluation.
Besides the various concerns about the measurement tools themselves, one should also have in mind
‘what’ the different indicators and models actually measure. As already mentioned in part I, S&T, and
even more R&D, is only one element in the whole innovation process. R&D processes are not
necessarily sequential and the borderlines between the stages in the R&D process are not clear-cut.
There is no universal indicator appropriate for the description of each phase of the R&D process. Thus,
the results of these activities cannot be measured in the customary scientific sense of ‘measuring’ a
variable. One of the solutions is to use indicators that are proxies, rather than direct measures of the
output of the R&D process. Certainly, the interpretation of these measurements should be done with
great caution.
In innovation economics, indicators can be classified into three categories: input indicators (or resource
indicators), byput (or R&D result) indicators and output indicators (or progress indicators) (Grupp,
1998). ‘Input indicators’ or ‘resource indicators’ should be regarded as a generic term embracing every
possible means for measuring personnel, monetary, investment and other expenditure on research and
development and innovation. They can contain for example R&D outlays, R&D personnel, investment
statistics, royalties paid, etc.
‘Byput indicators’ or ‘R&D result indicators’ measure sub-phenomena of technical change. Grupp
(1998) states that byput or throughput indicators measure ‘attendant’ or ‘partial’ effects of technical
progress and can thus be regarded as the result of R&D activities, regardless of whether or not these
results are important for the success of the innovation. Patent, publication and citation statistics are the
most utilised R&D result indicators.
‘
30
Output indicators’ or ‘progress indicators’ relate to the economic effects of the innovative
performance, rather than to detailed R&D activities. They represent the qualitative, quantitative or
value-rated advances in production processes of products, which are not solely caused by R&D
activities, according to the author. Clearly, the scope of the analysis, the innovation system as a whole
or the R&D system, influences the operationalisation and thus the construction of input, byput and
output indicators. Figure 2, clarifies the relation between the R&D functions, the input and output
indicators and their application. The specific relation between patents and R&D will be discussed in the
next chapter.
Figure 2 - R&D functions and their linkage to in- and output indicators
Input
indicators
Throughput &
output indicators
Functions
Applications
Knowledge
R&D personel
citations
Research
(fundamental /
basic)
R&D expenditures
Scientific papers
citations
Research
(strategic /
applied)
paper citations
S&T
transfer
Patents
Industrial
development
patent citations
Technometric
specifications
Innovation
& imitation
Foreign trade with
R&D intensive
products
Diffusion
(technological/
product)
Source: Grupp & Schwitalla (1989)
In the middle of the figure, we see the different functions of the R&D or invention process. It is
needless to say that the borderlines between the different stages are unclear and that innovation lines are
neither linear nor simple. However, a crude structure between R&D stages and the various indicators
does exist (Grupp & Schwitalla, 1989). Attention should be paid to the role attributed to ‘citations’ – i.e.
publications being cited - and ‘paper citations’ – i.e. literature citations in patents – in the S&T transfer,
the diffusion of knowledge. As an introduction, these two output indicators will be discussed in the next
two sections.
31
II. Patents and publications as output indicators
The most frequently used indicators for measuring technological output are based on patents. Patents, as
a detailed source of information on inventive activity, offer an interesting monitor device to identify
main lines and trends, and even, under specific conditions, the possibility to analyse R&D processes in
more detail (Engelsman & Van Raan, 1994).
But what exactly do patents measure? A patent does at least represent a minimal amount of invention
that has passed a thorough examination by the patent office on both the novelty of the claimed item and
its potential utility (Grilliches, 1990). On the input side, we see various R&D indicators such as
research personnel and internal and external R&D expenditures. The total image of relationships has to
be taken into account.
Scientific publications are considered as output for theory and model development, basic and applied
research. The field of Scientometrics covers the measurement of scientific and technical research
activity (Gauthier, 1998; Van Raan, 1997). Bibliometrics is a branch of scientometrics that focuses on
the quantitative study of scientific publication (scientific output) for statistical purposes.
Bibliometric methods serve three main functions: description, evaluation and scientific and
technological monitoring. Patents and the use of bibliometrics as technology and science indicators
will be discussed extensively in chapters 7 and 8. The science base of technologies can be analysed by
‘linking’ areas of science to technological domains. Patents and publications, as closely related to the
S&T interaction, play a central role in the linkage. The relationship between science and technology can
be of a direct nature, through literature references in patents, or of a more ‘indirect’ nature, for example
inventor – author relationships and co-operations. Both approaches are feasible and have frequently
been empirically tested leading to different results (see chapter 9).
In our study we will focus on the ‘direct’ linkage approach. In this approach the non-patent references
(references to literature) play a central role. Patent inventors occasionally refer to (scientific) literature
as a source of inspiration and knowledge for the creation of their invention, also known as ‘prior art’.
Extraction of the non-patent references and the identification of the relevant science domains referred
to, enables the identification of the science base of the technology under review. Further analysis of the
characteristics of the linkage such as linkage intensity, distribution references, validity range etc., will
enrich our understanding of the linkage and increase its applicability as an S&T policy decision support
tool, ex-ante and ex-post.
32
III. Some considerations around the use of S&T indicators
Before starting a more in depth discussion of patents, publications and linkage, we would like to briefly
discuss a number of issues related to S&T indicators. It was already pointed out that patents and
publications are only partial indicators of respectively technology and science. Therefore, when
interpreting results and drawing conclusion based on indicators, one should be very cautious. In a
detailed study on knowledge formation and technical development in the pulp and paper sector,
Laestadius (1998) argues that aggregated published statistics are not reliable for a deeper understanding
of the R&D activities and specifically of the radical technological change that took place in this sector.
Furthermore, he concludes that “they are thus also of little use for ranking and comparing countries and
or industries” (ibid., p. 393). Thus, statistics (or indicators) should, because of their nature as proxy
measures, be interpreted with great care.
A second related issue, pointed out to us by Martin (1996), is that evaluation of basic research is best
carried out using a range of indicators, or what we could be referred to as a ‘scorecard’ of indicators, for
example including peer evaluations and other qualitative as well as quantitative elements. Indeed, the
more indicators, the more balanced and reliable the evaluation will be. This approach may very well be
projected on the evaluation of not only basic research but also on the evaluation of a range of other
activities such as patenting and technological progress.
The last general issue that we would like to touch upon concerns what Glänzel (1996) refers to as “The
need for standards in bibliometric research and technology”. In order to increase the reliability and
validity of bibliometric analyses, standards need to be developed. Standardisation is not only important
in regard of used techniques but also in regard of data sources on which the techniques are applied.
Differences between classifications, terminology, methodology and databases need to be taken into
account when working with S&T indicators. Despite of all this, the construction of indicators is and
remains a highly important tool in the analysis of S&T and, as we will see further on, in many cases the
only tool available.
33
CHAPTER 7 - PATENTS AND MEASUREMENT WITH PATENTS
I. Patents, the patenting system and its economic rationale
I.1. A patent and its economic rationale
The analysis of patent information is considered to be one of the most established, directly available and
historically reliable methods of quantifying the output of a science and technology system (Soete &
Wyatt, 1983). Patents, more than any other innovation indicator, occur widely in economic literature.
Or like Grupp (1998) says:
‘No other innovation indicator can be traced back over comparatively long periods of time,
may at the same time be disaggregated at a very low level allocable to individual economic
units, and is also precise and accurate insofar as identification of the timing of the
innovation event is concerned’ (Grupp, 1998, p.144-145).
Before looking into patents as technology indicators, it may prove useful to identify what exactly a
‘patent’ is. Grilliches (1990) gives a clear description of what is understood under a patent:
A patent is a document, issued by an authorised governmental agency, granting the right to
exclude anyone else from the production or use of a specific new device, apparatus, or
process for a stated number of years. The grant is issued to the inventor of this device or
process after an examination that focuses on both the novelty of the claimed item and its
potential utility. The right embedded in the patent can be assigned by the inventor to
somebody else, usually to his employer, a corporation and/or sold to or licensed for use by
somebody else. This right can be enforced only by the potential threat of or an actual suit
in the courts for infringement damages (Grilliches, 1990, pp. 1662-1663).
A patent is a ‘property right’ based on an ‘officially sealed’ claim (Grupp, 1998). For the claim to be
recognised by other competitor companies, all property right details have to be public. The purpose of
the patenting system is the protection of the inventor. Without property rights, technological knowledge
would be public property and competitors would be able to imitate without penalty and claim new
knowledge to be their own. The inventor is granted a temporary monopoly situation and, by doing so,
ensuring him of sufficient benefits for his innovative efforts. Patenting also keeps incentives to innovate
high enough for private inventors, so that a sufficient number of innovative ‘efforts’ are made, which in
turn favours technological advance and economic growth.
34
According to Grupp (1998), a patent has three qualitative properties that require attention in connection.
On one hand, a patent grants to the owner the exclusive right of exploitation of a precisely defined
technical knowledge for a specific period of time. Three conditions for this grant need to be fulfilled:
novelty, quality and the possibility of being commercially applicable. The stimulating function of the
patent facility is supplemented by the information function, which manifests itself through publication.
The information function is the second qualitative property of patents. Patents can be used by others
than the inventors, with the purpose of obtaining knowledge about the progress of technological
knowledge. This is mainly possible through the patent and non-patent citations that have a major
knowledge component.
Thus, from a social-economic point of view, patents have the advantage that the information contained
in the patent administration is publicly accessible (a slight ‘contradictio in termini’ when we look at the
nature of patents), which leads to a greater diffusion of technological innovations. The patenting system
fulfils an important role in the information distribution in the sense that it avoids needless duplication of
R&D efforts, which in turn can accelerate technological progress.
The third function relates to the output function of a patent document. Successful R&D activity is
usually followed by a patent giving detailed information (date, time, circumstances, locations etc.) on
the activity itself. The last mentioned property is mainly used in connection with innovation
measurement.
With regard to the second function of a patent, it is not clear whether the patenting system also
enhances social welfare. The system stimulates innovations and technological competition.
Simultaneously though, it grants monopoly rights in order to keep the incentives for private inventors as
high as possible, which disturbs regular competition, albeit temporarily. Economists like to point to the
fundamental discord between appropriation and diffusion of knowledge, a very important factor in
economical progress (cfr. section 4.III.). The economic literature suggests various instruments that can
be used by policy-makers to deal with this discord. In particular, both length − the duration in time −
and width − the range − of the patent protection can be used to influence social welfare in an optimal
way.
The relationship between patents, as a measure for innovative output, and R&D, as a measure of
innovative input, has repeatedly been investigated. The results in the literature indicate a strong
statistically significant relationship between R&D expenditures and patent numbers in the crosssectional dimension, i.e. across firms and industries. Pakes and Grilliches (1984) for example found that
firms that spend more on R&D, possess more patents. In the within-firm time-series dimension, a
statistically significant relationship between R&D and patent counts is found too, but here the
relationship appeared to be much weaker. In a study performed by Arundel and Kabla (1998) it was
shown that large R&D intensive firms do not patent a higher percentage of their innovations than firms
with low R&D intensities.
35
I.2. Patenting systems
An inventor (individual, agency or company) wanting to protect an invention in a particular country
files an application in that country’s patent office. In the application, one or more claims will be stated,
referring to the aspects in which the product or process has to be considered innovative. In the so-called
prior art examination, these claims will be subject to further assessment and comparison. It is also
possible to protect an invention in more than one country. Up to one year after the ‘priority year’ the
applicant can file for a patent in any desired country, at a regional patent office such as the European
Patent Office (EPO) for simultaneous protection in more than one country or at the international patent
office, such as the World Intellectual Property Institute (WIPO), for a broader protection.
Two of the major regional patenting systems are the European Patent Office (EPO) and the United
States Patent and Trademark Office (USPTO). As a general rule, a patent office acts only for its own
country. An exception on this rule is the EPO, which in some matters can act on behalf of a group of
European countries. The difference in patenting procedures is naturally reflected in the patent data made
available by each of the patent offices. The major points of difference between the EPO data and the
USPTO data are summarised below (Debackere et al., 1999).
§
"First-to-file" vs. "first-to-invent"
Whereas the European Patent Office assigns property rights to the one who is first to
file a patent application, the US Patent and Trademark Office employs the "first-toinvent" principle. That is, whoever can demonstrate that he was the first to invent
something (for example, by means of "laboratory notebooks"), will be assigned all
intellectual property rights, irrespective of the fact that someone else has filed a patent
application for the same invention.
§
Patent applications vs. patent grants
Within the system employed by the European Patent Office, patent applications without
exception are published 18 months after the priority date. In this system, a patent
application is also published, even if the patent is not granted in a later stage. In the
United States, however, until March 2001, only patent grants are published by the
USPTO. Grants do not follow a strict timetable, and can in many cases take up to five
years (OECD, 1994a). After March 2001 however, a separate application-database has
been made available by the USPTO. In terms of statistical comparisons, obviously, this
will result in a difference in terms of "presence" and "visibility" of companies or other
actors in the respective patent databases (see figure 3).
Figure 3 – The different lead between EPO and USPTO application procedures (prior to March
2001)
National
priority
(first appl.
EPO
publication
EPO grant
Years
since
priority
1,5
1
Priority year
2
3
4
US publication and grant
(patent date)
36
5
§
Patent and non-patent references
Unlike in the EPO database, the USPTO database also includes citations to documents
considered to be relevant. That is, citations to relevant patents as well as to scientific
publications are included in the database.
As a result of the aforementioned differences, information extracted from the EPO and USPTO
databases is not fully comparable. In addition, we also have to be aware that, when making international
comparisons with respect to patenting activity of various actors and/or countries, such comparisons are
hampered by international differences in the legal conditions surrounding the granting of a patent. In
practice, this results in US companies having a comparable advantage with respect to patent grants in
the USPTO system, relative to foreign companies. This is because US companies have better
knowledge of how the system works. In the EPO system, however, such distortions seem to influence
the granting of a patent to a lesser extent. This is mainly due to the fact that the European Patent Office
is a supranational agency. European applicants therefore will have little, if any "home advantage" in the
EPO system, when compared to the US situation.
I.3. Patent data sources
In short, patents, because of their electronic availability (e.g. CD-ROMs of the USPTO or "Espace
Bulletin" of EPO), make it possible to investigate a whole lot of research questions. Besides the already
mentioned "official" patent databases, such as EPO and USPTO, there are various ("commercial")
information providers that offer patenting information both in "raw" and in "edited" format, such as e.g.
Dialog and Datastar, the most well-known patent database being DERWENT. The two most widely
used patent databases are (1) the USPTO patent database, administered by the United States Patent and
Trademark Office (USPTO), and her European counterpart (2) the EPO patent database, administered
by the European Patent Office (EPO).
USPTO Patent Database
The USPTO patent database is administered by the United States Patent and Trademark Office
(USPTO). The database contains data on all granted patents over the period 1976 up to now. A weekly
update is possible through USPTO's FTP-site. The information available consists of:
§ Title and description of the patent;
§ Main IPC-code and supplementary IPC-codes;
§ All relevant time data (application filing date, issue date, priority date);
§ Assignee information (e.g. name and address);
§ Inventor information (e.g. name and address);
§ References to other patents - U.S. Patent Reference and Foreign Reference - and references to
non-patent documents - Other References.
q
EPO Patent Database
The establishment of the European Patent Office (EPO) was a consequence of the 1973 Munich
Convention that laid down a number of rules concerning filing procedures and the drafting of patent
applications. As a result of this convention, it became possible to file one single application for all
associated European countries. After granting the patent, it is being split up in a bundle of national
patents for each country for which legal protection is requested.
q
37
The EPO patent database contains information on all European patent applications as well as on all
patent grants since 1978. It contains all relevant information with respect to the technical side of the
invention, as well as the administrative and legal history of the patent application and possible grant. In
particular, the database contains information on:
§ Title and description of the patent;
§ Main IPC-code and supplementary IPC-codes;
§ All relevant time data (application filing date, issue date, priority date);
§ Assignee information (e.g. name and address);
§ Inventor information (e.g. name and address);
§ Opposition against the patent.
Contrarily to the USPTO patent database, the EPO database does not contain information on possible
references to other patents or scientific literature. This information is nevertheless included in the REFI
database. The REFI (Reference File) database contains all references that are included in European
patents from 1978 on. These references consist of citations to other patents, as well as citations to
scientific publications that served as an inspiration source for the invention described in the patent
document.
II. Patent statistics and trivialities around patents
As mentioned before, if we want to address questions about the sources of economic growth, the rate of
technological change or the competitive position of different firms and countries, it appears that we
have almost no good measures. As a consequence, we are reduced to pure speculation or to the use of
various, only distantly related, ‘residual’ measures and other proxies. Patents are one of these residual
measures. As Grilliches (1990) indicates: ‘In this desert of data, patent statistics loom up as a mirage of
wonderful plenitude and objectivity’. Patents offer a host of methodological-technical advantages.
However, like with any indicator or methodology, several biasing aspects have to be kept in mind. The
overview below illustrates the different pro’s and contra’s (the latter mainly biases that have to be kept
in mind during interpretation).
Advantages of patents indicators
§
The special proximity of patents to the output of industrial R&D and other inventive and
§
Patents cover virtually every field of technology useful for the analysis of the diffusion of key
innovative activities implies that there is no other equivalent for this purpose
technologies (excepted software, which is generally protected by copyright and can be patented
only when it is integrated in a technical process of product)
§
Patents offer a world-wide geographical coverage
§
The very detailed classification in patent documents, which allows for almost unlimited choice of
aggregation, levels from broad fields to single products
§
Patent documents include many details of interest, such as year of invention, technical
§
The statistical processing of data is largely free of errors, because patent documents are legal
classification, assignee, inventor and so on
documents in which the details are recorded carefully
§
Accessibility, electronic availability of patent data (EPO/USPTO data on CD-ROM’s,
DERWENT, Dialog, Datastar)
38
Biases towards the use of patent indicators
§
Firms differ in their propensities to patent (# patents per unit of expenditure on R&D or just # of
patent applications)
§
Technology fields differ in their propensity to patent
§
Countries differ in their propensity to patent: size and geographical position give rise to different
expectations of the returns from patent protection (combination with other input or output indicators
is necessary)
§
Differences among national patent systems, arising from legal, geographical, economic and cultural
factors (issue of ‘home advantage’)
Many of the abovementioned advantages are in practice not very visible, and in some cases not visible
at all, meaning that some advantages are disadvantages at the same time. However, to cite Grilliches
(1990) once more:
‘In spite of all the difficulties, patents statistics remain a unique resource for the analysis of the
process of technical change. Nothing else even comes close in the quantity of available data,
accessibility, and the potential industrial, organisational, and technological detail.‘
(Grilliches, 1990, p.1702)
When using patent statistics in order to answer questions about sources of economic growth, the rate of
technological change or the competitive position of different firms and countries, certain other
trivialities intrinsic to patents and the patenting system have to be kept in mind, besides the ones
mentioned above. These trivialities are:
§ Rationale of patenting
§ Classification system of patents
§ Value of a patent
We will discuss these trivialities in the following three paragraphs.
II.1. Rationale of patenting: patenting behaviour
With ‘rationale of patenting’ we refer to the underlying motives and considerations of the decision to
apply for a patent. As we have mentioned before, various reasons for not applying for a patent can exist,
apart form the reasons behind the procedure for patenting. Patent protection is not the only way to reap
market success from an innovation. Secrecy, rapid launching, low prices and so on can supplement or
even replace patent protection. In areas of fast-moving development, patent protection may be of little
value because inventions quickly become obsolete. This implies that:
§ Not all inventions are utilised and commercialised, and consequently do not lead necessarily to
innovations;
§ Not all innovations are patentable and those that are patentable, are not necessarily patented;
§ Sometimes a company patents inventions that are not used.
39
The economic literature also pays a lot of attention to the strategic patenting behaviour of companies.
Many empirical studies have pointed out that not all innovation-active companies engage in patenting
and that companies that do engage in patenting do not apply for patents for all of their innovations. This
has important implications for the utility of patents (the validity range) as a measure for innovativeness
of companies and nations.
Figure 4 – Validity range of patent indicators according to product groups
PATENTABLE
INVENTIONS
ALL
INVENTIONS
PATENT
APPLICATIONS
INNOVATIONS
Innovation relevant
patent applications
Patent
applications
without
economic
applications
Innovationrelevant
inventions
without patent
application
Non-patented
inventions without
economic application
Source: Grupp (1998)
The relation between inventions, innovations and patents is summarised in figure 4. The figure shows
that not all inventions are patentable and that not all patentable inventions are actually patented.
Furthermore, not all patents or inventions are economically applicable. These differences between
inventions and patents and between patents and final economic application are crucial in the
interpretation of patent indicators or of other quantitative measurements with patents.
Why do companies choose not to patent a certain invention? Various important reasons can be
identified. First, the benefits of patenting an innovation may not outweigh the costs involved,
implying the direct costs involved with the application process, and also the disadvantages experienced
due to the patent conveying information to possible imitators, which in turn are able to innovate more
quickly around the granted patent with less R&D investment. Scherrer (1996) mentions that in the
Cohen survey, it was found that applications are only submitted for 52 percent of product inventions
and for 33 percent of process inventions. Other reasons for not patenting are summarised in the table 1.
Table 1 – Overview of motives for not patenting an invention
Reasons for not patenting
In %
Too easy to invent around
65%
Too hard to show novelty
55%
Reluctance to disclose
47%
Cost of application
37%
Source: Scherrer (1996)
40
Scherrer (1996) concludes from these results that mostly the unimportant inventions are not patented. In
1993 MERIT in The Netherlands and SESSI in France conducted a survey on the innovative activities
of Europe’s largest industrial firms in collaboration with INSEE5. The propensity rate was assumed to
be equal to the percentage of innovations for which a patent application is made. The analysis revealed
that the average propensity rate for product innovations is 35.9%, varying between 8.1% in textiles and
79.2% in pharmaceuticals. The average for process innovations is 24.8%, varying from 8.1% in textiles
to 46.8% for precision instruments. Only four sectors have patent propensity rates that exceed 50%, for
both product and process propensities (Arundel & Kabla, 1998). Furthermore, the effect of secrecy as a
reason for not patenting was examined. It appeared that firms that find secrecy to be an important
protection method for product innovations are less likely to patent, but that secrecy has little effect on
the propensity to patent process innovations. The R&D intensity of the firms has no effect on patent
propensity rates for both product and process innovations (ibid., 1998).
Mansfield (1991) pointed out that protection by means of a patent has little effect on the speed and cost
associated with imitating a particular innovation. On average, copying patented product innovations
appeared to cost no more than 11% more than copying non-patented innovations. Moreover, companies
estimate the cost of imitation being a mere 6% higher, if they had patented the products that are
currently not patented.
Whereas some innovations are not patented, others lead to a plenitude of patents. This depends on the
nature of the innovation. In chemistry, for instance, an entire family of molecules can be used to make a
product with specific characteristics. The industrial application, however, will probably only use one of
those molecules, but the innovator is not effectively protected against imitation until he has a patent for
the entire family of these molecules. In addition, strategic patenting behaviour of firms can cause a
profusion of patents. A patent portfolio strengthens a company's bargaining position in technology
agreements, and it can also be used to keep new companies out of the market.
The environment in which companies operate, influences the cost-benefit analysis of protection by
means of patenting. Important determining environmental factors are the intensity of technological
competition and the ease with which technological innovations can be copied. This can explain why
patenting behaviour appears to be very sector specific and innovation type specific (process or product),
as also pointed out in the MERIT/SESSI survey (see Arundel & Kabla, 1998).
Furthermore, the stage of the technological lifecycle in which the product is situated has an influence on
the patenting behaviour. Pakes (1986) finds that patents are applied for at an early stage in the inventive
process, a stage in which there is still substantial uncertainty concerning both the returns that will be
earned from holding the patents, and the returns that will accrue to the patented ideas. Large companies
(>100 employees) are inclined to consider both strategic and legal protection mechanisms, whereas
small companies, in view of their financial possibilities, apply less quickly for patenting protection.
5
SESSI is the Statistical Service of the French Ministry of Industry. The survey was conducted in co-operation with the
National Institute of Statistics and Economic Studies.
41
II.2. The Classification problem
A second major issue around patents and the patenting system is the classification problem. To use
patents in economic research of any kind, we must be able to relate them to meaningful economic
categories of interest. However, the classification system is primarily based on technological and
functional principles and is only rarely related to economists’ notions of products of well-defined
industries (Grilliches, 1990).
According to the OECD patent manual (1994a), ‘Inventions are classified by one or more symbols, so
that patents belonging to a given technological field can be filed and retrieved’. Patent offices classify
patents in classes and subclasses based on the need to ease the search for prior art. ‘Some system is
necessary for accessing the technical information contained in published patent documents’ (ibid.,
1994a). Thus, the classification scheme has been constructed from a technical point of view. First of all,
we will briefly explain the International Patent Classification (IPC). Then we will discuss the issue of
relating the classification scheme to meaningful economic categories of interest.
II.2.1. The International Patent Classification (IPC)
In view of the international dissemination of patent information, a single international system has
proven itself necessary. This international system became the IPC, which is applied in a large number
of countries and four international organisations. Its symbols are printed on published patents. The IPC
is divided in sections, classes, subclasses, groups and subgroups.
IPC is designed so that each technical object to which a patent relates can be classified as a whole. A
patent may contain several technical objects and may therefore be allocated several classification
symbols. An invention is normally classified according to its function or intrinsic nature, except when
its application alone determines its technical characteristics. For example in IPC, subclass F02K is a
product-oriented subclass that contains all jet propulsion plants. But subclass H03K is a functionoriented subclass covering the whole range of pulse techniques and contains many different products.
According to the OECD patent manual (1994a), the IPC is a combined function/application
classification system in which the function takes precedence.
The IPC is revised and if necessary amended every five years, but not retroactively. This means that no
re-indexation whatsoever takes place and that the whole classification is no longer very accurate
(Grupp, 1994, Research Policy). This is one of the biggest disadvantages of the IPC. Due to the
changing classification, comparison of the evolution of technology fields, by using the classification,
becomes difficult. It is essential that the relevant versions of the classification should be used for a
study covering number of years.
II.2.2. Relating patents to economic categories
Classification schemes evolve in time due to changes in the categories (add, remove, merge categories).
Furthermore, if we want to make use of patenting information to examine economic performance of
innovations, we are confronted with the problem that patent classifications (IPC) and industrial
classifications (such as SIC for OESO data and NACE for Eurostat/NIS data) are not directly
comparable. As we discussed, the patent classification system is based primarily on technological and
42
functional principles and is only rarely related to economists’ notions of products or well-defined
industries.
Grilliches (1990) notices that even before any classification is attempted, one has to take into account
the inherent ambiguity of the task. When we want to assign a particular invention to an industry, we
can either assign it:
a.
to the industry in which it was made (‘industry of origin’);
b.
to the industry that is likely to produce it;
c.
to the industry that will use the resulting product or process and whose productivity may benefit
thereby (‘industry of destination’ or ‘industry of use’).
In the last 15 years several attempts to overcome this problem have been undertaken. Grilliches (1984)
and his colleagues at the National Bureau for Economic Research started from patent totals for
particular firms and then grouped them into industries according to the company’s primary activity.
This thus comes down to a classification by ‘origin’. However, firm data are often not the most
appropriate to look at.
This approach may be useful for the analysis of company level data, relating patents to R&D
investments and the subsequent wealth of the companies in, which they originated. But, according to
Grilliches (1990), it is much less useful for the analysis of industrial data, because particular patents
may have an impact far beyond the boundaries of their industry of ‘origin’. Grouping of companies by
industrial classification codes (SIC) also appears to be a rather unsatisfactory solution, because of the
conglomerateness of many of the large corporations. A company may change its name and moreover,
the patent office does not always use a consistent company code in its computer record.
At the micro-level, Trajtenberg (1987) provides a more pragmatic, rather than an all-out solution for the
classification problem. He proposes a more modest approach based upon the availability of powerful
techniques for computerised search in large databases. Search techniques can make use of keywords in
the title and/or in the abstract (possibly established with the help of experts) pertaining to the product in
question that may appear, identifying a small set of relevant patents, their classification codes and any
other relevant information. Those techniques should allow one to identify quite easily all the patents
issued in predetermined economic categories and to retrieve them for further analysis. Nevertheless,
Trajtenberg (1987) also acknowledges that there is no absolute certainty that this approach will deliver
all the patents in a given field, and only those. Rather, the search process consists, at least at first, of
trial and error, something that is also supported by Grupp (1998).
Therefore, a distinguishing feature of Trajtenberg’s (1987) approach is that the units of analysis are
narrowly defined product classes or technology fields, rather than firms or industries. Additionally,
Grupp (1998) mentions that the so-called combined search strategies for patent documents, by using
both a systematic search by means of classification symbols in combination with the relevant
keywords have been shown to be rather successful (see for example Noyons et al. 1998; Meyer,
2000b).
Over the years, various attempts were undertaken to develop ‘concordance’ tables for classifying patent
data by economic sectors. In the mid-1970s, the US Office of Technology Assessment and Forecast
(OTAF) tried to produce patent statistics at the three and two-and-a-half standard industrial
classification (SIC) digit level. This was done by developing a ‘concordance’ table between the patent
class and subclass classification system and the SIC industrial classification. If a subclass did not
43
obviously appear to belong to a single SIC industry, it was counted in all of the relevant ones, resulting
in considerable double counting. This concordance table was heavily criticised, both because of the
arbitrariness in the assignment of some of the subclasses and because of the misleading inferences that
could arise from the double counting. These fundamental problems are inherent to concordance
schemes.
In 1994, Verspagen et al. (1994) developed the MERIT concordance table between the International
Patent Classification (IPC) used by the European Patent Office (EPO) and the International Standard
Industrial Classification of All Economic Activities (ISIC – rev.2) of the United Nations. The table
makes it possible to assign the patent data by field of technology (IPC) to a classification by economic
sector (ISIC). In this concordance table, if an IPC-class does not obviously belong to a single ISICindustry, it is assigned to all the relevant ISIC-classes for a percentage proportional to the share of the
patents in that IPC-class. The percentages thus give the share of the patents in each IPC-class assigned
to the accessory ISIC-category. It should be mentioned that this concordance is also arbitrary.
According to Grupp (1998), the division of a functional relationship between markets and areas of
technology that is not derived from a specific innovation project, but that is to be universally valid,
would seem to be particularly unrealistic when a radical innovation is present. This is because the
application of such an innovation is expected to be wide ranging with a great influence on several
economic sectors. ‘The construction in each case represents an attempt of a pragmatic partial solution
with a shake theoretical basis’ (ibid., 1998). However, Grupp (1998) considers the concordance
problem soluble for individual innovation processes (microeconomic level), but not for universal
purposes (macro- and meso-economic).
II.3. The problem of the ‘value’ of patents
A second major problem with patents concerns the value and valuing of a patent. Patents appear to vary
tremendously in their technological and economic ‘importance’ or ‘value’. Only a very small number of
patents comprise inventions of high technological or economic value, while the bulk of patents are only
of marginal value. Grupp (1998) discusses this issue by questioning the existence of so-called ‘key
patents’. These are patents serving as the basis for a number of other patents and which are thus
extremely valuable. With regard to the latter, Grupp (1998) also mentions the ‘basic patent’ concept,
which is often applied to genetic engineering and robotics, areas generating fundamental breakthroughs.
The central question is: how can we differentiate between these very valuable and less valuable patents?
By simply counting the number of patents, we give equal weight to all patents, implicitly considering
them to be of equal importance. Because the economic significance of a patent is so variable, it is very
difficult to estimate the average value of patent rights or the average value of the invention represented
by a particular patent.
Scherrer (1996) identifies three prolific avenues of research dealing with the problem of ‘importance’ of
patented inventions:
1. The use of patent citation counts as an index of the importance of patents.
2. The use of data on the number of countries in which a particular patent is applied for.
3. The use of patent renewal data.
4. The stock market valuation of firms to investigate the ‘value’ of patents.
44
Grupp (1998) also mentions patent citation counts as a possibility for valuing a patent and making it
comparable. But, furthermore he also mentions the use of patent claims as another way of making the
economic value of patents comparable. In the next sections, we will briefly discuss each of the
identified ways.
The use of patent citation counts
Trajtenberg (1990) found that the economic value of patents could better be measured with a so-called
citation weight than with individual patent figures. He worked out how often a patent is cited in later
patents (‘prior art’ search information) and argued that this frequency is related to the economic value
of the patent. The higher the number of citations, the higher the economic value of the patent.
Interesting at this point is to mention the study of Narin & Noma (1987), who examined the links
between corporate patent and citation data, and several other indicators of corporate performance in 17
US pharmaceutical companies. One of their conclusions was that the number of patents correlates with
expert opinions, budgets and publication data but not with financial performance, whereas citations per
patent do correlate with financial performance. This confirms that patent citations relate to the
economic value of a patent.
q
This line of research can be traced directly to the widespread use of citations appearing in the scientific
literature (more about this subject in chapter 8). Unlike in scientific publications, when applying for a
patent, it is the (legal) responsibility of the applicant to describe fully the ‘prior art’ (United States). The
applicant, assuming that he or she is also the inventor, must set out the background in such a way as to
show how the claimed invention relates to, but is innovatively different from what was already public
knowledge. The applicant's citations therefore identify work that is related to the new invention,
whereas the examiner's citations serve to define the area in which the invention is truly original and
therefore merits the granting of a patent.
The use of data on the number of countries in which a particular patent is applied for
Here, the ‘value’ of a particular patent is assumed to be proportional to the number of countries in
which it has been applied for, also referred to as ‘geographical scope’ of the patent requested protection
(Debackere et al., 2000). The reasoning is that for the more ‘important’ inventions, patents are applied
for in relatively more countries than for inventions that are less ‘important’ in the judgement of the
applying company.
q
The original application and its duplicates (in other countries) are termed a ‘patent family’. When a
patented invention is applied for in three countries, the ‘patent family’ would thus consist of three
members. If it would have no external/foreign applications then it would only have one ‘family
member’. As already mentioned, the reasoning would be that inventions with a large patent family
should be considered as more important than a patent that is only applied for domestically.
Grupp (1998) however, objects that the patent's contribution to progress − i.e. what makes it a more or
less ‘important’ patent − cannot be correctly estimated from the size of the family, because taking out a
patent in other countries than the domestic one is often subject to marketing and other corporate
strategic considerations. Moreover, he mentions, one has to take into account specific economic
geographic conditions as well. It is generally known that, for example, European companies, due to
their geographical situation and high volume trade with other European countries, apply for patents in
many more external (European) markets than for example U.S. companies.
45
The use of patent renewal data
In this approach, the ‘value’ of a patent is determined by the patent renewal information. In most
countries, holders of patents must pay an annual renewal fee in order to keep their patents in force. If
the renewal fee is not paid in any single year, the patent is permanently cancelled. It is assumed that
renewal decisions are based on economic criteria. This means that agents will only renew their patents
if the value of holding them for an additional year exceeds the cost of such renewal. Moreover, patents
for which patent protection is more valuable will tend to be protected by payment of renewal fees for
longer periods of time.
q
By using a learning model for the early years of a patent's life, Pakes (1986) finds that patents are
applied for at an early stage in the inventive process. At that moment, often, there is still a substantial
uncertainty about the returns that will be earned from holding the patents. Gradually, the applicants
uncover more information about the actual value of their patents. Most of them turn out to be of little
value, but the rare ‘winner’ justifies the investments that were made in developing them.
By using these results, Schankerman and Pakes (1985) estimated the average value of a patent right. It
appeared that the average value of a patent right is quite small, about $7.000 in the population of patent
applications in France and the U.K. and $17.000 in Germany. However, the distribution of these patent
right values appeared to be very dispersed and skewed. One percent of patent applications in France
and the UK appeared to have values of $70.000 or more, while in Germany one percent of the patents
granted had values of $120.000 or more. They also estimated that annual returns to patent protection
decay rather quickly over time, with rates of obsolescence of around 10 to 20 percent per year.
Stock market value
This line of research uses data on the stock market values as indicators of the success of inventive
activity and as the driving force behind the investments in it. According to Grilliches, Pakes and Hall
(1987), the use of stock market values as an ‘output’ indicator of the research efforts has one major
advantage: under simplifying circumstances, a change in these research efforts would be reflected
immediately in the stock market value of the firm. A significant disadvantage of this type of
measurement is the large volatility in stock market measures. The results of Pakes' (1986) study on the
relationship between patents, R&D and the stock market rate of return indicate that about 5 percent of
the variance in the stock market rate of return is caused by the events that change both R&D and patent
applications.
q
Finally, we mention the suggested approach by Grupp (1998): the use of patent claims in valuing a
patent. The number of claims that are in the patent can represent its economic value. For instance, at the
American Patent Office, Japanese patents on average have fewer claims than German patents. A big
disadvantage of this method is that the patent claim indicator cannot be detected in databanks and has to
be constructed manually.
Before turning to the actual measurement of patents, let us summarise a number of findings that will
have to be kept in mind when measuring with patents, also in our study. In short, not all inventions are
patented (for different strategic considerations) and not all patent applications have an economic value.
The value of patents differs greatly depending on the used approach. Measuring with patents is not
without trivialities. However, understanding them makes their use easier.
46
III. The use of patent statistics for technology measurement
In this paragraph we will look at the quantitative aspects of measuring technological progress through
patents. Patent data can, as already mentioned, be aggregated and analysed in a number of ways,
including (OECD, 1994a):
1. Patenting by type of inventor, by firms or groups of firms
2. Filings in one or more fields of technology
3. The patenting activity of a country or a region
4. Patenting patterns over time
It is clear that patent indicators can be constructed on macro- (country or region), meso- (industry) and
micro level (the single firm or institute). The aggregation level differs, depending on the study under
review. Our study will focus mainly on the macro level (for more details on the indicators that will be
used in this study, we refer to part III).
Besides the simple counts of patents of an actor (country, region, firm etc.) in a certain field of
technology, it is also possible to relate patent counts with indicators such as R&D spending, indicators
for innovation etc. In the next sections, we will discuss a number of indicators and aspects of using
patents in measuring technological progress, that are relevant for our study.
III.1. Measuring and comparing with simple output indicators; ‘foreign’ patenting
The simplest type of patent indicator is derived by counting the number of patents that comply with
one or more criteria (technology field, application year, certain inventor etc.). Comparing the number of
patents between countries in a certain technological field can provide a basic insight in the differences
in technological performance.
In a clearly defined macroscopic domain, comparisons based on simple counts (such as between
countries) are feasible and reliable. If more than one inventor makes a joint application for a patent,
there are mainly two possibilities for assigning this patent to their countries of origin (OECD, 1994a).
The first one suggests ‘sharing’ the patent among the various countries concerned, each country
receiving an equal share. The second possibility suggests appointing a full count (1 patent) to each
country concerned (cfr. section 8.IV.3.3.) For large aggregations of patents, both methods provide
equally reliable findings.
However, national patenting activity depends on institutional factors, on the nature of the legal system
and on many domestic factors, including the size of the population and the economy, the size of its
R&D and research community and the technological infrastructure. In order to take these differences
into consideration, we can relate patent counts to demographic (# inhabitants), economic and research
variables (GDP, R&D expenses). This will provide patent indicators that are independent of the size and
characteristics of the countries and that enable a more ‘equal base’ comparison. This process is
frequently referred to as ‘normalisation process’.
An often-constructed measure is the ‘propensity to patent’. This contains the number of patents per
dollar (or other monetary unit) invested in R&D or per R&D staff member. The differences in the
propensity to patent, as pointed out by Caniëls (1997) and others, can be caused by inefficiencies in the
innovation process and/or by the fact that R&D investment is used merely for imitation. In other words,
47
it measures the extent to which R&D inputs (in terms of expenditures or personnel) are translated into
patents, and it can therefore be regarded as an (imperfect) measure of R&D output.
When making international comparisons of patenting activity, one has to take into account possible
biases in the degree to which countries (or regions) patent their inventions. According to Soete & Wyatt
(1983), international comparisons raise the question of whether there are any specific biases in the
degree to which countries patent their inventions. In the 1970s, it was widely known that the Japanese
patented significantly and systematically more across all industries, whereas socialist countries hardly
made use of patents.
International comparisons of patents are hampered, furthermore, by international differences in the legal
conditions surrounding the granting of a patent, above referred to as institutional factors, domestic
factors and the legal system. The ratio of patent grants to patent applications can vary significantly
between countries.
McCulloch (1980), Soete & Wyatt (1983), OECD (1994a) suggest that in order to avoid such
differences, and again compare countries on a ‘equal basis’, one should make use of the relative
penetration of third country markets. For example, when comparing U.S. and Japan, one could calculate
the fraction of total United Kingdom patents issued to each country. In this way, national (or regional)
performance is compared in the face of a common set of legal and economic conditions, such as a
similar screening procedure, etc. However, Soete & Wyatt (1983) explicitly mention that firm- and
industry-specific patent propensity biases will not be corrected. For a further discussion of a number of
studies about foreign patenting, we refer to Pavitt (1985).
Foreign patenting is an approach that will also be used in our study. When comparing the patenting
activity of countries, we will analyse their patenting activity in a third country or, in this case, a
relatively impartial patenting office (USPTO or EPO).
III.2. Specialisation indices
‘Specialisation indices’ can be used to answer questions such as ‘where does the country (or region)
under study stand in various technology domains compared to other countries (or regions)?’. The
‘Revealed Technological Advantage (RTA)’, developed by Soete and Wyatt (1983), is the most
frequently used specialisation index and will be discussed in the next section.
§ Revealed Technological Advantage (RTA)
To understand the structural factors underlying the relative technological positions of the major
economic zones, insight is needed into their respective technological strengths and weaknesses. The
RTA-index or ‘specialisation index’ or ‘activity index’ or ‘relative specialisation index’, frequently
used in patent analyses, is an index that can provide this kind of insight.
The RTA-index consists of the ratio of the number of patents of a country in a particular technological
sub-domain, divided by the total number of patents in this sub-domain, and number of patents of the
country under study in the whole field, divided by the total number of patents in the field. In other
words, the RTA-index compares the share of a particular country’s patents for a particular technological
sub-domain with the share of other countries in the same domain. As such, it is a relative indicator of
technological specialisation (strength). If country X has a share that appears to be bigger than that of
48
other countries, we can say that country X has a ‘revealed technological advantage’ for that specific
technological domain.
The definition of the RTA index is as follows:
åP
RTA =
åP åP
Pij
ij
i
ij
j
Pij
ij
ij
= Number of patents of country i in sub-domain j
åP
= Number of patents of all countries in sub-domain j
åP
= Number of patents of country i in the whole field
åP
= Number of patents of all countries in the whole field
ij
i
ij
j
ij
ij
The value of the RTA-index varies from 0 to +∞. A value lower than 1 reflects that country i has a
relative disadvantage in category j. A value of 1 corresponds to a neutral position, whereas a value
exceeding 1 signifies a relative advantage.
The index6 will be used to compare and to reflect possible changes in time in relative positions for the
relevant technological sub-domains. Evolutions in specialisation levels for a particular country or group
of countries will also be reflected.
According to Soete & Wyatt (1983), the evaluation of the technological performance of a country over
time has to be done by splitting the considered time-span of evaluation into two (or more) time frames
and calculating RTA-indexes for both of them. The indexes for both time frames can be plotted in a
diagram that makes further comparison of the performance possible (see figure 5).
RTA 1st period
Figure 5 – RTA-index diagram for comparing countries
3
I
II
IV
III
2
1
0
1
2
3
RTA 2nd period
Source: Soete & Wyatt (1983)
The vertical axis represents the RTA-indices for the first time frame, whereas the horizontal axis does
the same for the second time period. As a result, in the upper left quadrant (I), technological sub6
A major disadvantage of the RTA-index is its skewed distribution (it is asymmetrical and unconstrained on one
side), which makes it less suited for "distance" measurements between countries for various technology classes.
For distance measurements a symmetrical version is available, the RPA-index.
49
domains can be found where the country under study had a relative advantage in the first time frame,
but had lost it in the second. The upper right quadrant (II) then represents sub-domains in which the
country had a technological advantage in both periods. The lower right quadrant (III) contains subdomains in which the country developed a technological advantage during the second period, from a
position of disadvantage in the first period. Finally, technological sub-domains for which the country
concerned had a relative disadvantage in both periods can be found in the lower left quadrant (IV).
§ Revealed Patent Advantage (RPA)
In order to make the RTA-index suitable for distance measurements, the RPA-index is advisable. The
RPA is an adjusted version of the RTA-indicator. The method of calculation is given below:
RPA = 100 ln RTA = 100 x (RTA2p1) / (RTA2+1)
The value of this index varies between −100 and +100, with 0 as a neutral value. It can appropriately
be used for distance measurements.
Interpreting the RTA-index should be done with great caution, especially in the case of a low number of
patents in the fields under review. The RTA is actually a combination of a specialisation index and an
activity index. Accordingly, the only correct interpretation of the RTA-index is that it reveals
specialisation of a certain country in a certain domain and corrects this specialisation by the countries
patenting activity. Indeed, results for countries with low numbers of patents should be interpreted with
care.
Furthermore, we would like to mention a less known patent indicator introduced by Grupp (1998), the
‘International Technology Production (ITP). This indicator yields an adjustment to control for the
domestic advantage. This ‘domestic advantage’ H (Home country) for domestic applications from
country X can be calculated with the aid of external patent applications in another country Y. The ITP
results in the actual number of patent documents applied for externally (or which were applied for on
domestic territory if no domestic advantages existed). A normalisation is achieved for the size of
market, the attractiveness of the patent office and the legal peculiarities.
50
III.3. Impact indicators
§ Attractivity Index (AII), Citation Performance Index (CPI)
In order to analyse the quality or value of patents (cfr. section 7.II.3.) of a certain country, and/or blocks
of countries, the impact of patents (per domain) can be measured through their citation intensity. For
example: what is the impact of German patents in Europe? Or what is the impact of German patents in a
certain technological field? The added value of this type of analysis is to add a qualitative aspect to the
pure quantitative analysis. In order to address impact issues of patents, the Attractivity Index (AII) can
be calculated. This indicator is a variant on the RTA index, except now citations are used instead of
patents (for calculation, see RTA-index).
Another indicator for measuring the impact of patents, developed by CHI Research which is a United
States company specialising in science and technology indicators, is the ‘Citation Performance Index’.
Its procedure is to compare the most highly cited 10 percent of patents for a country to those of the
world. A ration of 1.0 means the country’s citation performance exactly matches that of the world. This
indicator also measures the impact or ‘quality’ of the patents of a certain country. The formula for the
indicator is:
Pi
Pt
Pi = the percentage of country i’s patents appearing among the most cited 10 per cent
Pt = the same percentage for the worlds patents
III.4. Maps of technology
Finally, we should mention the use of information contained in patents to construct ‘maps of
technologies’. For this purpose, besides information on the innovating company and the specific
characteristics of the invention itself, information is gathered on the references contained in each patent
application both to previous relevant patents and to research papers reporting results on which the
invention is based. ‘Maps’ of various technological (sub-) domains can then be constructed by
examining the interrelation between frequently cited patents. Co-citation, co-classification or co-word
analyses are possible. Moreover, within each (sub-) domain, an assessment can be made of the relative
position of different actors ranging from companies, research institutes to entire countries or regions.
We shall discuss the technique of mapping extensively in the next section. As Pavitt (1985) mentions:
“Carpenter, Narin and their colleagues, have carried over into the analysis of patent statistics many of
the bibliometric techniques developed for the analyses of scientific papers”. That is why we prefer to
discuss these items at their origin: bibliometrics.
51
CHAPTER 8 - BIBLIOMETRIC INDICATORS AND ANALYSIS OF
RESEARCH SYSTEMS
I. Introduction to ‘bibliometrics’
Just like patents constitute an (imperfect) output-indicator of inventive activity, scientific publications
constitute an (imperfect) output-indicator of scientific activity. Whereas patents indicate a transfer of
‘knowledge’ to industrial innovation and a transformation into something of commercial and social
value, scientific research is the production of ‘knowledge’. Publication has three objectives: to spread
scientific findings, protect intellectual property and to gain fame. These three objectives reveal that
scientific publications can be seen as a means of examining the knowledge production function and
specifically analysing the R&D results in regard of scientific performance. Scientific journals −
sometimes referred to as the ‘serial literature’ − obviously play a leading role in the communication of
research findings in many scientific fields (Moed, 1989).
The area of research utilising the information contained in research publications to get a clearer view on
an actor's scientific output is labelled ‘bibliometrics’. In 1969, Pritchard coined a new term –
‘bibliometrics – for a type of study that had been in existence for half a century and was at the time
known s ‘statistical bibliography’. In literature, in relation to bibliometrics we can find the terms
‘scientometrics’ and ‘informetrics’, which in some cases are being used as synonyms but actually have
a different scope. Several definitions of bibliometrics can be found in literature. Pritchard defines
bibliometrics as:
‘… the application of mathematics and statistical methods to books and other media of communication.’
(Pritchard, 1969, pp. 348 – 349)
‘Informetrics’ deals with the quantitative aspects of information in broader sense whereas
‘scientometrics’ refers to the application of bibliometric techniques to science and or technology
measurement, according to Van Raan (1997). Bibliometrics thus can be seen as a branch of
scientometrics that focuses principally on the quantitative study of scientific publications for statistical
purposes. In view of the aims of our study we shall continue with referring to the measurement and
evaluation of science as ‘bibliometrics’. It should also be noticed that in our view bibliometrics are also
suitable, and frequently used, for monitoring science and technology.
Derek de Solla Price, one of the pioneers in scientometrics and author of the milestone work Little
science, Big science (1963), indicated that the quantification and mathematical treatment of science can
be extremely valuable because of their power to bring order in the world of observation. However, he
notifies, quantification will surely not solve all problems of science. In addition, he recognised that
measuring ‘quality’ of scientific research in a quantitative way is by no means an easy topic to deal
with.
52
‘Bibliometrics’ then indicates the collection, handling and analysis of quantitative bibliographic data,
derived from scientific publications. Bibliometric methods serve three main functions: description,
evaluation and scientifical and technological monitoring. As a descriptive tool bibliometrics provide
an account of publishing activities at micro-, meso- and macro level. Secondly, bibliometrics can be
used for assessing the performance of research institutes or even countries. Finally, and that is the third
function, bibliometrics can be used for monitoring developments on S&T since longitudinal studies are
possible. The use of bibliometric data made it possible to study science unobtrusively, that is,
independently of information obtained from the scientists − the object of study − themselves (Tijssen,
1992).
Just like patents appeared to be an important, yet imperfect indicator of technological activity,
publications in scientific journals play the same role with respect to the measurement of scientific
activity. Still, although a large part of the communication of ideas takes place in the form of spoken
language (e.g. personal communications or lectures), and although not all written communication (e.g.
reports, memo's, working papers) appear as articles in scientific journals, one can assume that
eventually all relevant research findings are reported in the serial literature (Moed, 1989). However,
several studies (Moed et al, 1985; Moed & Van Raan, 1988) have indicated that the role of the serial
literature is not equally important among all sub-fields within science.
II. Bibliometric information: publications and data sources
II.1. Publications as data
The principle objects of measurement in bibliometrics are scientific publications. A scientific
publication can be defined as any kind of written material, either in a physical format, or as its
electronic equivalent in a computerised database, containing information with respect to scientific
research activities. Research reports, books, conference proceedings and articles in scientific journals
are considered to be the most important types, whereas research notes and letters, for example are
considered to be less important (Tijssen, 1992). As Martin (1996) points out that scientific publications
are a reasonable measure of scientific production. Furthermore, he points out that most publications
make only a very modest incremental addition to knowledge, while only a very few make a major
contribution. Publication counts, just like patent counts, are only a partial indicator of contributions to
knowledge. It is a variable reflecting (a) the level of scientific progress made by an individual or group
and (b) a number of other factors such as social and political pressures (ibid., 1996).
A scientific publication usually contains a title, the names of the authors and their institutional
affiliation. Within the text, frequently, citations are given to other publications of which the content, in
the mind of the authors, is related in some way or another to their own article. All citations given in the
text are usually reproduced at the end of the article, in the reference list. In order to make them easily
retrievable, every publication is uniquely represented by the bibliographic information it contains:
author(s) name(s), journal title, publication year, volume number and starting page number. Finally,
specific key-words, terms or classification codes, added by the author or the journal's reviewers, can be
included in order to define the content of the article adequately (Moed, 1989). However bibliometric
information contains a lot of errors. As Glänzel (1996) argues there is still a long way to go on the
subject of standardisation in bibliometric research and certainly on the issue of publication information
like author names, addresses etc. This topic will also prove to be very important in our study.
53
According to Tijssen (1992), scientific publications contain a multitude of quantifiable elements related
to three main aspects of scientific activity:
(1) The ‘size’ of scientific activities, as reflected in the output of research publications (the
‘products’)
(2) The transfer of knowledge (the ‘process’). Scientific publications are considered to be the
major communication channel for presenting original scientific findings and dissemination of
scientific knowledge, and the citation process reflects communication of knowledge in the
scientific community.
(3) The social and cognitive networks of science (the ‘structure’). Relational aspects of science
can be studied by means of information on the authors of co-authored publications, their
addresses, reference lists, and keywords taken from abstracts or the text.
Very similar to these elements of scientific publications are the categories that Martin (1996)
distinguishes: scientific activity, scientific production and scientific progress. Quantitative measures in
general give the impression of being more objective, more precise, and more reliable than information
from a qualitative nature. However, validity, reliability and relevance of measurements should be
checked when trying to quantify science (Tijssen, 1992). Besides, just like with patents, science
measurement and assessment should not be solely based on qualitative or on quantitative measures.
Reliability refers to the problem of stability or instability in measurement respectively leading to
similar or not similar outcomes. Any bibliometric measurement incorporates a certain amount of
unsystematic (‘random’) error, such as e.g. misspellings of author names. However, also systematic
errors, which represent a serious problem, may occur, e.g. when not all-important researchers in a field
are covered. Validity pertains to the issue whether measurements really measure what they purport to
measure. In the field of bibliometrics especially the validity of bibliometric variables as indicators of
‘quality’ of the scientific research performance is questioned.
Finally, relevance is concerned with limitations in the range of application. This definitely also applies
to bibliometrics, since it is founded upon the assumption that scientific publication represent in a
consistent way the total outcome of scientific research activity. The issue of relevance can pose a
problem in those (sub)-fields of scientific activity that are characterised by local orientation of research
activities, non-journal publishing practices, and insufficient reliance on references to acknowledge
intellectual debts (Nederhof, 1991).
Since insufficient validity, reliability, and coverage are for some part inherent to bibliometric data,
Tijssen (1992) stresses that one always has to take into account that any bibliometric measure will only
remain a proxy for a qualitative parameter in the complex system of knowledge production.
II.2. Scientific literature databases
The source for bibliometrics is always a database (OECD, 1997). Relevant bibliographic pieces of
information are extracted from scientific publications and gathered in large scientific literature
databases. The content of these databases is usually available in a hard copy format, but nowadays
most databases are also available in computer readable form and even through the Internet. The
producers of these databases extract bibliographic information from scientific publications, and usually
also apply some kind of indexing system by adding keywords or classification codes to the articles. In
54
parallel with the growth of science, the number of scientific publications has been doubling about every
15 years in the past 300 years (Tijssen, 1992). To keep up with this pace, new ways of cataloguing and
indexing the literature and making it retrievable have been developed.
Probably the most important database for bibliometric analyses is the Science Citation Index (SCI),
produced by the Institute for Scientific Information (ISI) in Philadelphia, USA. Several studies have
argued that at the moment the SCI is the most suitable source for detailed international bibliometric
analyses on macro level. The ‘inventor’ of the SCI is Eugene Garfield. The Science Citation Index is a
multi-disciplinary database covering more than 5700 major journals across 164 scientific disciplines in
the natural and life sciences. The SCI limits the scope of coverage to world-class scientific journals,
representing the ‘core’ scientific output in specific fields and eliminating research that not presented in
the ‘mainstream’.
Unlike most other databases that cover at most one discipline − e.g. Chemical Abstracts, that covers the
field of (bio) chemistry, or INSPEC, a world-wide database on Physics, Electronics and Computing −
the SCI is a multi-disciplinary database that claims to cover the most important journals in the natural
and life sciences. The subject coverage has been expanded from the initial SCI to include the Social
Sciences Citation Index (SSCI) and the Arts and Humanities Citation Index (A&HCI). The SCI
Expanded provides access to the SCI from 1973 forward and allows for searching full-length, Englishlanguage author abstracts for 70% of the articles in the database. Some of the disciplines covered by the
SCI Expanded include agriculture, astronomy, biochemistry, biology, biotechnology, chemistry,
computer science, materials science, mathematics, medicine, neuroscience, oncology, paediatrics,
pharmacology, physics, plant sciences, psychiatry, surgery, veterinary science and zoology.
The Social Sciences Citation Index is also a multi-disciplinary database covering the journal literature
of the social sciences. It covers more than 1725 journals across 50 disciplines, as well as individually
selected, relevant items from over 3300 of the world's leading scientific and technical journals. Some
of the disciplines included in the Social Sciences Citation Index are anthropology, history, industrial
relations, information science and library science, law, linguistics, philosophy, psychology, psychiatry,
political science, public health, social issues, social work, sociology, substance abuse, urban studies and
women's studies.
The Arts & Humanities Citation Index is the third multi-disciplinary database indexing 1144 of the
world's leading journals in the fields of arts and humanities, as well as a selection of relevant items from
over 6800 major science and social science journals. Some of the disciplines covered by the Arts &
Humanities Citation Index include archaeology, architecture, art, Asian studies, classics, dance,
folklore, history, language, linguistics, literary reviews, literature, music, philosophy, poetry, radio,
television and film, religion, and theatre.
There are two important sections to this Science Citation Index: the Source Index and the Citation
Index. The Source Index contains bibliographic information, by author, of all articles published in the
journals or books processed for the SCI. Bibliographic information enclosed includes all co-authors, the
full title of the publication, the journal title, volume, issue, page, year, type of item and number of
references in the bibliography (Egghe & Rousseau, 1990).
An important aspect is the classification system of the SCI covered journals. Journals and not the
individual publications are classified in so-called ‘subject categories’ (over 160). This classification is
based on the content and citation relations between the journals. If necessary the classification is being
55
altered yearly. The subject categories are partly comparable with scientific sub-disciplines. Between the
journals and the subject categories there is no one-to-one relationship. This means that a journal can be
assigned to more that one subject category. Luwel (1999a) grouped the categories in scientific
disciplines based on a classification scheme used in ‘Science et Technologie 1994, Rapport de
l’Observatoire des Sciences et de Techniques’.
Of all scientific literature databases, the Science Citation Index probably is used most frequently in
bibliometric analyses. However, other scientific literature databases exist. We already mentioned some
field-specific databases as Chemical Abstracts and INSPEC. Other databases are Compendex, a
specialist engineering and technology database, Embase, a specialist physical science database and
Pascal, also a general database covering several fields. Furthermore, a number of bibliographic
databases, varying in scope and size, are available via host-computers of vendor agencies such as
Dialog, DIMDI, ESA and STN.
These databases provide easy access to an immense body of readily available catalogued bibliographic
data on scientific research. As a consequence, a quantitative assessment of scientific performance can
be performed in a relatively easy and timesaving way. However, a number of limitations have to be kept
in mind. First of all there are the limitations of each database for compiling bibliometric indicators.
There is a suitable database for each study. Each database has it own characteristics like structure,
coverage, reliability, classification etc. Secondly, when working with citations to publications, we have
to be aware of the trivialities around citations. Examples are ‘self-citations’, ‘negative citations’,
‘language factors’ etc.
A final remark around bibliometric databases is that beside the advantages that commercial databases
provide in retrieval and use of information, there are also several disadvantages that occur with the
commercial exploitation of this kind of data. According to Moed (1989), and also suggested by Glänzel
& Schoepflin (1992), the dominance of commercial firms in the computerised bibliometric analysis has
been one of the impediments for progress in the field of bibliometrics. First of all, expertise was often
kept ‘in house’, and often few details were given on the methods that were used. Second, analyses
performed by these firms were often very expensive. Moreover, due to the specific licence agreements
of Computer Horizons Incorporated (CHI) and CRP with ISI-Thomson Scientific, these firms are only
allowed to present ‘final results’ to their customers. The data underlying these ‘final results’ could
not/and cannot be disclosed.
III. Limitations to the use of bibliometric data
As already mentioned before, the concept of reliability may be defined as the extent to which the results
are independent of its technical effectuation (Moed, 1989). This section deals with the specific
problems that exist with respect to collecting, handling and interpreting bibliometric data. Roughly
three main issues related to reliability of bibliometric indicators can be distinguished (issues around
citation analysis will be discussed in the next paragraph):
(1)
Completeness of bibliometric data;
(2)
Coverage of scientific literature databases;
(3)
The problem of statistics;
Each of these issues will be discussed in more detail in the following sections.
56
III.1 Completeness of bibliometric data
The first problem relates to the completeness of the data. Several researchers (among which Moed et
al., 1985) mention that it is very difficult to obtain complete publication and citation data. Almost
always, several omissions are detected in the ‘raw’ data, and have to be completed by manual checking.
Moed et al. (1985) mention that the number of missing data can easily amount to about 10 percent of
the total number of data involved.
Especially if the units of analysis are quite small by nature, e.g. when the research group is taken as a
unit of analysis, this can pose a problem. Let us take research groups as an example. Research groups,
containing on average two to ten researchers, normally produce a relative low number of publications
and citations. Consequently, bibliometric indicators are constructed, based on small numbers, and
obviously even small errors or a small number of omissions can lead to dramatic differences in results
and interpretations. Moreover, the distribution of citations among articles published by a research group
is found to be highly skewed. Consequently, omission of even one publication can have great
implications if the omitted publication happens to be a highly cited one.
Of course, if like in our study a higher level of aggregation is used, thus operating on larger numbers,
completeness becomes less a problem. This is, for example, the case when large departments, clusters
of departments, (sub) faculties, or entire countries or regions are taken as units of analysis.
III.2. Coverage of scientific literature databases
The second reliability issue is related to the coverage of scientific literature databases. Let us take a
closer look at the Science Citation Index in particular. The question is whether this database covers the
scientific literature in a sufficient way, both as to the different (sub-) fields (of natural and life sciences)
and to all countries. As can be seen, this question relates to two equally important aspects: (1) coverage
of all countries, and (2) coverage of all (sub-) fields of science.
As for the coverage of the scientific literature of the various countries, it is found that the ISI-databases
(the SCI, SSCI and A&HCI) show a bias towards publications from the USA (Egghe & Rousseau,
1990). Moreover, significant differences appear to exist between the SCI and other sources of national
coverage of fields with a more dispersed literature. This seems to be especially so when it concerns
journals from countries with non-Roman alphabets, such as Russia and Japan (Moed, 1989).
Problems of inadequate coverage appear to arise particularly in scientific fields in which researchers do
not publish primarily in a limited number of international oriented, and consequently mainly Englishlanguage, top journals, but rather in a wide range of journals with a national scope, which may be
nonetheless of high quality. The latter will often be non English-language journals. Moed et al. (1995)
found that such journals from non English-speaking countries indeed are less likely to be included in
the ISI-databases. This obviously points to a problem of a possible Anglo-Saxon (mainly US) bias.
Nevertheless, in a recent study by Luwel (1999a) about the possible US-bias of the SCI, it appeared that
for the disciplines covered by the SCI the average fraction per journal of publications from the EU and
the US is roughly equal. Furthermore, for the basic and applied sciences, the SCI seems to cover
equally well the US and the EU research works and as such ‘no US-bias’ was observed.
57
As for the coverage of the various (sub-) fields within the Natural and Life sciences, the question is
what the coverage is of the SCI relative to what researchers list in their research reports, annual surveys
and so on (Moed, 1989). In order to obtain an insight into the appropriateness of SCI source journals
and SCI source books as an adequate tool for bibliometric output and impact analysis, Moed et al.
(1985) studied the publications for all research groups from the faculties of Medicine, Mathematics and
Natural Sciences at the University of Leiden. They determined the percentage of total publications that
was covered in the Science Citation Index.
Summarising, the coverage for chemistry, physics and astronomy, and pharmacy/pharmacology
appeared to be sufficiently high (more than 70%). However, for biology and in particular for
mathematics, this percentage was found to be lower (54% and 26% respectively). The lack of coverage
in these cases is caused by two factors: (1) the output in Dutch is not covered by the SCI source journals
or books, and (2) articles and research reports (in English) that are published in non-SCI journals and
media such as books, reports, proceedings, and so on, are of course not covered at all. Moreover, Moed
(1989) mentions that several interviewed scientists from the fields of biology and mathematics judged
the SCI coverage as inadequate, and that it does not constitute a representative sample of the research
output of groups from these fields. Moed et al. (1985) conclude that, possibly, the role of the scientific
journal literature for communicating research findings may differ from discipline to discipline; a point
we will take up in the next section as well.
Finally, another disturbing factor relates to trend analysis over the years based on SCI data. The
problem relates to the large changes in coverage of the SCI database during the time period considered
(Moed, 1989). Indeed, when using the SCI or another ISI database, one has to take into account that
considerable disturbances can occur resulting from the fact that the ISI has included more source
journals during the 1980s and that, as from 1977, non-journal material, such as books, are also included
(Moed et al., 1985). Consequently, one should be very careful in interpreting trends in the numerical
values of bibliometric indicators, when trying to relate them to actual performance of e.g. a research
group.
III.3. The problem of ‘statistics’
This problem, identified by Moed et al. (1985), pertains to the question when a given difference
between two bibliometric scores should be considered as ‘significant’ with a certain probability, and
when it should be ascribed to mere chance. This topic has been discussed frequently. It has been
proposed to consider citing as a stochastic process, in which publications are assumed to attract
citations due to their impact, but also due to many accidental factors. However, small effects in trends
or small differences between bibliometric scores should be interpreted with great care (Moed et al.,
1985).
58
IV. Citation analysis
IV.1. Introduction
Citation analysis constitutes an important device in the quantitative study of science and technology.
The primary objective of the scientific community is the generation, processing, diffusion and
utilisation of scientific knowledge. Science as a social and cognitive system of knowledge is
characterised by a set of widely accepted and utilised norms, one of which is the practice of
acknowledging the various sources of information which have been consulted and/or used in a scientific
research activity (Tijssen, 1992). The latter becomes manifest through the list of references.
In this list of references, authors of scientific publications refer to earlier publications of which the
author has made use in doing his research activity or that support, provide precedent for, illustrate, or
elaborate on what the author has to say (Garfield, 1979). The author should indicate a used publication
uniquely so that retrieval of the original source is possible (Moed & Vriens, 1989).
The terms ‘reference’ and ‘citation’ are used interchangeably, in the strict sense the first is the
acknowledgement that a document gives to another, whereas the latter is the acknowledgement that a
document receives from another (respectively ‘citing’ and ‘cited’). Citation analysis is the area of
bibliometrics that deals with the study of the relationships between citing and cited. Citations may be
considered a measure of the impact of the articles cited, as well of their timeliness and utility. It is
presumed that a paper must have a certain quality in order to have an impact on the scientific
community, a presumption that can be disputed. Most criticism pertains to the problem of the unknown
motivations of the citing author (Egghe & Rousseau, 1990). Citations do represent a relation between
the cited and the citing publication. However, the precise nature of this relationship is often somewhat
unclear. Various motivations for citing are identified by Garfield (Smith, 1981). The most important
will presented below and discussed in the next sections.
1. Paying homage to pioneers;
2. Giving credit for related work (homage to peers);
3. Providing background reading;
4. Criticising and/or correcting previous work;
5. Identifying original publications in which an idea or concept was discussed;
6. Disclaiming work, ideas or claims of others (negative claims and homage).
It is very likely that norms for citing vary from discipline to discipline. However, one also has to take
into account that, just as there are a number of reasons why citations are enclosed, there may on the
other hand also be a number of reasons why a particular author has not established a link to another
document.
In other words, many documents that should have been cited are omitted, while many other documents
that the author does cite are only slightly relevant (Smith, 1981). Over half (55%) of the articles
published in the scientific journals covered by the SCI are not cited a single time in the five years
following their publication (OECD, 1997). Nevertheless, citations are objective measures that can be
gathered without the co-operation of any respondent (Smith, 1981).
The practice of citing the work of other authors has led to the construction of so-called ‘citation
indexes’. Citation indexing is based on the idea that an author's references to previous publications
identify much of the earlier work relevant to the subject of his own article (Egghe & Rousseau, 1990).
Citation indexing, of which the Science Citation Index (SCI) is by far the most important example,
59
offered an additional way for retrieving relevant publications by examining the citation linkages drawn
by authors in their publications (Garfield, 1979).
IV.2. The use of citations as ‘information’
The main subject of this section is how to use citations as items of information. The easiest way to use
citations is by citation counts. In this approach the number of citations is counted that have been
received by a given document or set of documents over a certain period of time from a particular set of
citing documents (Smith, 1981). If the unit of analysis is on the level of single articles appearing in a
particular journal, calculating e.g. the ‘impact’ factor of a publication (cf. infra) can refine this measure.
However, over the last three decades, new techniques and measures using citations were introduced and
new tools exploited. Two heavily used citation-based techniques devised to identify documents likely to
be closely related are bibliographic coupling and co-citation analysis. Two documents are said to be
bibliographically coupled if their reference lists contain one or more of the same cited documents
(Smith, 1981). Their strength of bibliographic coupling depends on the number of references they have
in common (Egghe & Rousseau, 1990). However, bibliographic coupling may not be a valid measure of
relatedness between two publications. Just because two papers have a certain reference in common does
not guarantee that they are both referring to the same piece of information in the cited article.
In recent times, co-citation analysis has become more popular. Two documents are said to be co-cited
when they are jointly cited in one or more subsequently published documents (Smith, 1981). Again the
strength of the relationship is defined as the frequency with which the two documents are cited together
(Egghe & Rousseau, 1990). Thus, while bibliographic coupling focuses on groups of papers that cite a
common source document (later documents become linked because they cite the same earlier
documents), co-citation analysis groups papers that frequently are cited in pairs (documents are later
cited together). Since co-citation analysis is of particular interest in the domain of the mapping of
scientific specialities, this technique will be dealt with in more detail in the section on mapping of
science.
IV.3. Basic assumption and problems around citation analysis
IV.3.1 A general overview
In this section we shall discuss a number of basic assumptions underlying citation analysis, and a
number of problems or pitfalls around citation data. Before going into more detail on these aspects in
the next sections we first present a general overview in the table below.
60
Table 2 – General overview of assumptions and problems/pitfalls related to citation analysis
Assumptions underlying ‘’Citation Analysis (Smith, 1981)
(a) Citation of a document implies use of that document by the citing author
(b) Citation of a document reflects the merit, in terms of quality, significance or impact, of that
document
(c) Citations are made to the best possible works
(d) A cited document is related in content to the citing document
(e) All citations are equal
Possible problems/pitfalls related to ‘Citation Analysis’ (Smith, 1981)
(a) The problem of multiple authorship (co-authorship)
(b) The problem of self-citations
(c) The problem of unique identification of authors
(d) Implicit citations
(e) Fluctuations in time
(f)
Variations among (sub-)fields
(g) Errors
IV.3.2. Detailed discussion: basic assumptions underlying citation analysis
(a) Citation of a document implies use of that document by the citing author
Smith (1981) indicates that this first assumption actually consists of two parts: (a) the assumption that
the author refers to all, or at least to the most important, documents used in the preparation of his work;
and (b) the assumption that all documents that are listed in the reference list reflect actual usage, i.e. a
reference to a particular document is only made if that document actually contributed to his work. If
these assumptions do not appear to hold, certain documents will consequently be underrated since not
all sources used were cited, while other documents will be overrated since not all sources cited were
actually used.
(b) Citation of a document reflects the merit, in terms of quality, significance or impact, of
that document
There is a high positive correlation between the number of citations a particular document receives and
the quality of that document. However, we can already mention here that the use of citation counts as
quality indicators have generated a great deal of discussion. For now, we can already mention that this
issue was investigated in several studies and that results produced support for this assumption most of
the times, although results of citation counts should always be compared with alternative quality
indicators, such as for example expert opinions. This is also a remark we made previously when
discussing that assessment should be based both on qualitative and quantitative indicators.
(c) Citations are made to the best possible works
Only if one assumes that authors, in describing their research efforts, sift through all possible
documents that could be cited in order to select only those references judged best, this assumption can
be expected to hold. However, studies of science information use have suggested that accessibility
probably may be as important as quality as a factor in the selection of an information source. In short,
an article might well have been cited because it happened to be on the writer’s desk rather than because
it was the ideal paper to cite.
61
Accessibility of a document, Smith (1981) argues, is a function of its form, place of origin, age and
language. However, just like a paper may be more or less accessible, a researcher may be more or less
visible or even famous. It may well be that anything that enhances a researcher’s visibility is likely to
increase his citation rate, irrespective of the intrinsic quality of his work.
(d) A cited document is related in content to the citing document
The use of citation indexes to retrieve relevant documents supports this assumption. The results of an
experiment by Barlup, mentioned in Smith (1981), provide additional support for this statement.
Authors, asked to assess the degree of relatedness of citations to their own work, judged 72% of the
citations to be definitely related, and only 5% to be definitely not related.
However, when it is extended to other uses of citation measures, this assumption becomes more
problematic. In the case of bibliographic coupling, the assumption does not hold, since it is not sure
that two documents citing a third are citing the same unit of information in it. Consequently, Smith
(1981) argues that bibliographic coupling merely indicates the existence of the probability (possibly
zero) of a relationship in the content of two documents. Analogously, the fact that two papers are cocited is no guarantee that a relationship exists in the content of both papers.
(e) All citations are equal
Studies using citation counts generally assume that all citations (with the possible exception of selfcitations) can be weighted equally. However, many researchers have looked for ways to refine citation
analysis in the sense that all citations to a particular article would not be treated necessarily as
equivalent. Smith (1981) distinguishes between two types of refinements: mechanical (e.g. looking for
multiple occurrence in an article) and intellectual (content analysis). The differences between the two
categories being that mechanical refinement requires no judgement or inference, whereas intellectual
refinements do need human analysis and are, consequently, rather time consuming.
IV.3.3. Possible problems/pitfalls related to ‘Citation Analysis’
Besides the rather subjective assumption underlying citation analysis, one must also be aware of
problems associated with the sources of citation data or with the use of secondary sources, such as
citation indexes. A range of problems is associated with the use of citation data. These problems are
often very down-to-earth and rather technical in nature; still they can have a big impact on the results, if
one fails to take them into account. A brief overview of the various pitfalls and problem issues shall be
presented. Once again we base ourselves on the excellent overview given by Smith (1981).
(a) The problem of multiple authorship (co-authorship)
Several authors publish articles of which they are only co-author rather than the first author. This poses
a problem because the ISI citation indexes list the cited references only by first author. As a
consequence, by only counting citations found by looking in the citation index under the name of a
particular author, one misses all citations to publications where he was a co-author rather than a first
author. Studies on how reliable ‘first-author’ citations are in predicting ‘all-author’ citations show that
the two measures are correlated highly, but that nevertheless large discrepancies arise between the
authors in the top of the two rankings (Moed, 1989).
62
In the case of multi-authored works, there is also the problem of allocating credit to the various authors.
In other words, which weighting procedure should be applied in allocating credits to authors of multiauthored works? A similar problem is encountered when dealing with patent: how do we count and
attribute a patent to the different inventor or applicant countries. In general, there are three ways to deal
with this problem: (1) straight count; (2) normal count; and (3) adjusted count (Lindsey, 1980; Egghe
& Rousseau, 1990).
In the case of straight counts, multi-authored works are treated the same as single-authored works, in
the sense that the first author receives all the credit. This is by far the simplest approach and, in
addition, it greatly reduces the work required to collect the data. Note that the use of straight counts can
be considered as a sampling strategy that, as such, should be examined in terms of its
representativeness. In this rationale, when using the straight count procedure, it is assumed that the set
of publications in which an author's name occurs first constitutes a representative sample of all of this
author's publications. It is concluded that no strong empirical evidence, nor theoretical rationale is
found to support the assumption underlying the use of straight counts (Egghe & Rousseau, 1990).
By using normal counts the problem of distributing credit for multi-authored publications is solved by
giving full credit to all contributors. The problem with this approach, however, is that it tends to inflate
the publication or citation scores of those researchers who produce many multi-authored papers.
Moreover, problems can occur due to the fact that, in the normal count approach, the sum of the number
of publications of all authors becomes larger than the actual number of papers under study. For
scientists and politicians who use science indicators, normal counting is far more comprehensible and
easy to interpret? ‘A share of 10% of country X means in this sense that 10 out of every 100 papers in
the world have at least one contributor from country X’. Furthermore, it seems that the ‘meaning’ of a
10% share in the adjusted count (or fraction scale) is far more difficult to explain (Braun et al., 1991).
It is safe to say that the best way to handle the problem of multi-authorship is by assigning credit
proportionally. In the case of adjusted counts, every co-author is assigned a fraction of the authorship.
Thus, if two authors write an article, each would be assigned half a credit; in the case of three authors,
each would get a third; and so on.
(b) The problem of self-citations
As indicated by Egghe & Rousseau (1990), there are very few articles that do not include any selfcitation. In addition, authors are inclined to refer to their own work to a greater extent than to the work
of any other single author. The problem with the inclusion of self-citations into citation analysis is that
it, obviously, tends to inflate citation scores for those researchers who cite their own work abundantly.
As a result, in almost all studies, self-citations are excluded. This can easily be achieved for singleauthor papers. When citations are used for science policy purposes or if a particular research group or
research institute is taken as the unit of analysis, self-citations are often interpreted as ‘in-house
citations’. The latter comprise citations by members of a research group or a research institution to the
work of their own group or institution (e.g. Nederhof & Van Raan, 1993). In this case information has
to be collected to identify all members of the research group or institution under study.
(c) The problem of unique identification of authors
This problem pertains both to the issue of ‘homographs’, resulting in failure to differentiate between
researchers with the same name, and ‘synonyms’, resulting in failure to group references to an author's
work due to the lack of a standard form for the author's name. Because it is possible that more than one
researcher with the same name and initials (a case of homographs’) is publishing in the same field,
63
incorrect attributions of citations to authors can be the result. To differentiate between them, additional
information on the authors is needed, such as for example their institutional affiliation.
Another problem indicated by Smith (1981) pertains to the so-called ‘synonyms’, which are not
recognised as such. She mentions that, unless some standard form for an author's name is established,
citations to the work of many authors will be scattered. Examples of ‘synonyms’ in the context of
citation indexes include an author's name with a variable number of initials, a woman's maiden and
married names, different transliterations of foreign names (especially in the case of Russian, Chinese or
Japanese names), and misspellings.
Also relevant for our study are the synonyms or misspellings for one and the same journal title. Luwel
(1999a) signals and studies these problems extensively. For example in the SCI, in the so-called target
articles, journal titles are written fully, whereas in the cited references, abbreviated journal titles are
used. This problem influences the possible matching approach, a significant step in the ‘linkage’
methodology (for more details see part III). In addition to variations in titles, or their abbreviated forms,
journals also merge, split into new ones, change titles and are translated (Egghe & Rousseau, 1990).
(d) Implicit citations
Because most citation indexes include only explicit citations, these are the only ones considered in
citation analyses. The one exception is the Arts and Humanities Citation Index (A&HCI), which also
includes implicit citations when a paper refers to, though not by means of an explicit citation, and
substantially discusses a scientific work. Smith (1981) mentions that implicit citations often also occur
in the form of eponyms (an author's name plus an idea or concept associated with this person; e.g.
Murphy's law). Furthermore, once an idea becomes widely accepted, a citation to the original source is
often omitted.
(e) Fluctuations in time
Smith (1981) indicates that large variations in citation counts may occur from one year to another.
Therefore, she recommends that citation data should not be too restricted in time. This also makes clear
that seemingly large variations, often in relatively small numbers, should be interpreted with care. In
addition, Moed et al. (1985) mention that, within fields, citation practices can change in time. The
results of their analysis of the SCI database between 1970 and 1980, for example, show that, on
average, a journal article contains an increasing number of citations to 0 to 2 year-old articles during the
decade.
(f) Variations among (sub-) fields
This problem relates to differences both in publication practices and in citation practices among (sub-)
fields or science disciplines. We already mentioned before that the scientific journal literature might
play a different role in communicating results of research activities among different (sub-) fields. As a
consequence, also citation counts strongly depend on the internal characteristics of the research fields.
For example, it is found that the average chemistry or physics article contains about twenty references,
while a mathematical article contains, on average, less than ten. This of course leads to difficulties in
cross-discipline comparisons (Egghe & Rousseau, 1990). Furthermore, the coverage of a publication in
terms of how many people study the publications in that area, is also relevant for the number of
citations these publication will get. In short, due to differences in publication and citation practices
between different scientific (sub-) fields, direct comparison of, for example, research performance of
groups from different disciplines is inappropriate.
64
(g) Errors
It goes without saying that citation analyses, whether they are based on citation indexes or not, can be
no more accurate than the raw material used. A general problem with the use of citations is that authors,
in their list of references, give bibliographic descriptions that do not exactly correspond to the
bibliographic data of the article they intend to cite. Errors can include discrepancies in cited author
names, journal title, page, volume, and year. As for the use of citation indexes, one can assume that, by
processing citations for inclusion in citation indexes, additional errors are introduced, while others are
eliminated.
Incorrect citing of sources appears to be far from uncommon. In their data set taken from SCISEARCH,
Moed & Vriens (1989) find that the number of citations showing a discrepancy in at least one data field
(full author name, journal title, publication year, starting page number and volume number), relative to
the number of citations not showing any discrepancy, amounts to 9.4%. This means that for every 100
citations showing no discrepancy, 10 citations show a variation or error in at least one data field. They
also find that the number of citations not showing any discrepancy constitutes 91% of the total number
of citations, while citations with discrepancies in only one data field account for almost 5%. Moreover,
their results also suggest that copying references from other articles may be a cause of the observed
multiplication of errors in cited references.
V. Use of bibliometric data in a science policy context
Bibliometric data is by no means exclusively used in science policy. Various disciplines make use of
publication and citation data for different purposes. Indeed, ‘bibliometric information may serve
different masters simultaneously’ (Tijssen, 1992, p. 14). Bibliometrics provides an account for
publishing activities at the level of countries, provinces, cities or institutions, and is used for
comparative analysis of productivity. Bibliometric data are also used as a benchmark for monitoring of
science and technology, since longitudinal studies of scientific output help identify areas of research
that are developing or regressing. This last aspect is of particular interest for out study.
Although different users will have different reasons to make use of such quantitative information on
science, they all rely upon numerical data on research publications to obtain a more objective
assessment of different aspects of science and its participants. As an illustration of different uses by
different groups of people and for different purposes, we see that, for example, information scientists
focus on publications in the context of information retrieval systems, while sociologists of science use
them to study the professional behaviour of scientists. Scientists, on the other hand, rely on
bibliometric data for measuring individual (or group) scientific performance. In science policy,
bibliometric data is mainly used in the light of accountability and justification of research funding
allocations, on the one hand, and comparison of scientific input and output, on the other (Tijssen, 1992).
Bibliometric data are not always the most suitable way for evaluating scientific performances. Bain and
Rinhaldini (1989) pointed out that, due to the nature of bibliometric evaluation, a rather competitive
model of scientific activity (number of publications, number of received citations), bibliometric
indicators are wholly misleading when applied to the evaluation of co-operative research.
65
Nevertheless, by means of bibliometric data, such as publications and citations, a company or a research
institute can address questions, such as:
§
What countries and/or organisations are most active in a specific (sub-) field?
§
How did patterns of activity change over time?
§
What is the impact of a particular research group or institute's work?
§
Which organisations were involved in the crucial breakthrough(s) in a particular (sub-)
field? (Hicks, Martin & Irvine, 1986).
Within the field of bibliometrics, we can differentiate between ‘bibliometric research performance
analysis’ (also referred to as evaluative bibliometrics) and ‘mapping of science’, that is the analysis of
the more structural features of science (Engelsman & Van Raan, 1991). Another frequently used
distinction is the one between ‘descriptive indicators’ and ‘relational indicators’. The primer is of a
more quantitative nature whereas the latter implies a more qualitative way of analysis by looking into
the interactions and relations between science and technology. Furthermore, methodologically we can
distinguish between one-dimensional and two-dimensional techniques. The performance indicators are
mainly constructed with one-dimensional techniques usually based on ‘direct’ counts of elements such
as publications, patents and citations. The relational indicators, and specifically ‘mapping’, are based on
two-dimensional techniques constructed from co-occurrences of specific information elements
(Engelsman & Van Raan, 1991).
In ‘bibliometric research performance analysis’ publication output and received citations (volume and
impact) can be used to assess the research performance of various actors at various levels. This may be
an author, an institution (Noyons et al., 1999; Martin & Irvine, 1983), a sector of activity covering
several institutions, or even a geographic area like a country or a region (European Commission, 1997;
Tijssen & Van Wijk, 1999; Katz & Hicks, 1996). The use of bibliometric data for the assessment of
research performance will be dealt with extensively in the next section.
The second stream of research relevant to science policy-makers is concerned with the ‘mapping’ of
scientific (sub-) fields. Mapping is a way to monitor the production of researcher in a particular S&T
field. Maps of science can be created with different techniques mainly based on co-occurrences of
information elements (cf. infra). The aim of mapping science is mainly concerned with understanding
both the structure and the evolution of scientific (sub-) fields displaying both the structural and dynamic
aspects of science and technology (Korevaar & Van Raan, 1992). Mapping of science will be discussed
in section 8.6.6.
VI. Bibliometric performance indicators
VI.1. Indicators of publication output and productivity
Output indicators are essentially based on the numbers of articles published by the unit of analysis
under study − e.g. a research group, research institute, or a country − in the international scientific
literature (Moed et al. 1985). Numbers of scientific publications, indeed, can be considered to be a
reasonable measure of scientific production (output) − i.e. the extent to which a consumption of the
inputs to research creates a body of scientific results (Martin, 1996). According to the OECD (1997) the
benefits of science – or outputs – encompass the carriers of new knowledge (patents, publications,
application etc.). However, counts of publications appear to be a much less adequate indicator of
66
contributions to scientific progress − i.e. the extent to which scientific activity results in substantive
contributions to scientific knowledge (ibid., 1996).
As we discussed above, most publications only contribute to the addition of knowledge in a rather
modest way, while only very few make a major contribution. The mere counting of publications,
however, does not capture this. Therefore, publication counts can only be considered, at best, as a
partial indicator of contributions to knowledge since, aside from reflecting the level of scientific
progress made by an individual or a group, they also reflect a number of other factors not related to
scientific progress, such as publication practices of the institution, use of publications to obtain grants
etc. As a solution, citations are used to account for the diversity in contributions made by various
publications. This practice will be discussed in more detail in the section on impact indicators.
To summarise, output can be measured in a satisfactory way by means of publication counts. However,
productivity of a research group, a field, or an entire country is often obtained by normalising the
output with various input indicators, in order to allow for differing size of research activity (cfr. chapter
7). Productivity measures thus can be seen as size-adjusted output indicators. In the evaluation of
research groups, productivity is often defined as the number of publications per full-time equivalent
spent on research. Other productivity measures are publications per person or per unit of funding. Just
comparing the technological performance of countries by using patents normalisation is necessary for
creating an ‘equal base’ of comparison, comparing the scientific output should also be normalised in
order to correct for size differences (see for example Luwel, 1999b; European Commission, 1997).
VI.2. Impact indicators
VI.2.1. The relationship between ‘quality’ and ‘impact’
As already mentioned in the previous section, citations are used to take into account the different
contributions to knowledge of different publications. Correspondingly, we can introduce the notion of
‘quality’ of scientific research, a concept virtually impossible to operationalise, since it may refer to a
variety of values. With regard to scientific research, Moed et al. (1985) distinguish between three
aspects of quality. Cognitive quality relates to the importance of the specific content of scientific ideas.
Methodological quality, on the other hand, is concerned with the accuracy with which methods and
techniques are applied. Finally, aesthetic quality is related to the attractiveness of mathematical
formulations, models, and so on. Needless to say that the assessment of especially this last type of
quality can be seen as a highly subjective enterprise.
Moed et al. (1985) consider these three aspects of quality to be related to, what they call, ‘basic quality’.
Since an assessment of such ‘basic quality’ is based on criteria intrinsic to scientific research, only peers
can make a truthful judgement as for the basic quality of research projects. Next to this they also
recognise a ‘scientific quality’. Scientific quality necessarily comprises basic quality but, in addition, it
also relates to the extent in which researchers are active in presenting their research findings to
colleague researchers. As such, only if the ‘scientific quality’ of a particular research programme is
sufficiently high - that is, if the research is of a certain basic quality, and if colleague researchers are
made sufficiently aware, it becomes possible that this research does have a significant impact. In this
respect, Martin (1996) makes a distinction between quality, importance and impact of publications. He
defines ‘quality’ as follows:
67
‘ It describes how well the research has been done, whether it is free from obvious 'error', how
aesthetically appealing the mathematical formulations are, how original the conclusions are,
and so on.’ (Martin, 1996, p. 348)
The ‘importance’ of a publication, then, is seen as:
‘ … its potential influence on surrounding research activities − that is, the influence on the
advance of scientific knowledge it would have if there were perfect communication in science.’
(Martin, 1996, p. 349)
Finally, the ‘impact’ of a publication is described as:
‘ … its actual influence on surrounding research activities at a given time. While this will
depend partly on its importance, it may also be affected by such factors as the location of the
author, and the prestige, language and availability of the publishing journal. ‘ (Martin, 1996,
p.349)
If we come back to the notion of contribution to scientific knowledge, it is clear that, of these three
concepts, it is the third, scientific ‘impact’ that is the relevant one. As such, citation counts should be
regarded more as an indicator of impact, rather than of quality or importance (Martin, 1996).
VI.2.2. Short-term vs. long-term impact
If it is assumed that scientific publications in a particular field are reflections of what happens at the
research front in that field, then insight into a particular group's impact at the research front can be
gained by looking at the number of times a research group's publications are cited.
Moed et al. (1985) distinguish between ‘long-term impact’ and ‘short-term impact’. By looking at the
long-term impact of a research group's work, one can make inferences on the degree to which a group
has made a more permanent contribution to scientific advance. Short-term impact, on the other hand,
relates to the impact of researchers' work only a few years after the publication of research results. As
such, short-term impact, operationalised by means of citation counts, is mainly concerned with factors
such as the extent to which a research group forms part of the research community, and the extent to
which research results are widely known by colleague researchers.
Consequently, short-term impact should be related more to the visibility of research groups (how
famous are they), rather than with the extent to which a group has made a permanent contribution to the
field, which is more related to the long-term impact. The latter is the more permanent contribution to
scientific knowledge.
Of course, a high short-term impact will not guarantee a high long-term impact, since many theories
will be rejected and, as a result, will not acquire any long-term impact. However, if a group's work
succeeds to acquire a high short-term impact during a relatively long period of time, older publications
of that group may have a high long-term impact as well (Moed et al., 1985).
As for the relevance of short-term and long-term impact for scientific research evaluation purposes, it
seems plausible that research groups can hardly be required to produce work with a high long-term
68
impact. However, they can be required to take part in scientific discussions and engage in activities at
the research front of their field. As such, indicators of short-term impact, rather than of long-term
impact, are of primary importance, when it comes to the evaluation of research performance of research
groups, but also of research institutes or entire nations (Moed et al., 1985).
VI.2.3. Actual vs. expected impact
When assessing research performance of different research groups, institutes or countries, some kind of
‘benchmark’ is needed in order to give those absolute citation counts a meaning. Direct one-to-one
comparison between different research groups or countries is not possible. For example, as pointed out
by Narin & Hamilton (1996), a high-quality paper in a lightly citing field may receive fewer citations
than an average-quality paper in a heavily citing field. As such, it should be clear that citation counts
obtained in one of the lightly citing fields might not be directly compared to ones obtained in highly
citing fields. Therefore, data should be properly normalised and adjusted for field and sub-field
differences (Narin & Hamilton, 1996). For this purpose, a comparison is often made between the actual
versus expected impact of a research group's work (e.g. Nederhof & Van Raan, 1993; Moed et al.,
1985; Moed et al., 1995; Noyons et al., 1998; European Commission, 1997).
One type of normalisation often employed is to adjust citation counts to the impact of the journal in
which the paper was published. In this approach, citation counts to a research group's publications are
compared with the ‘expected’ number of citations; the latter being the average number of citation
counts to all publications in the journals in which the group has published (Moed et al., 1985). This
procedure, thus, entails counting the average number of citations a particular set of publications
receives in, for example, the third year after publication (often the year in which a publication is cited
most frequently), and comparing this to the average number of citations of an article in the same journal
(Nederhof & Van Raan, 1993).
In order to do this, first of all, for every journal a Journal Citation Score (JCS) is calculated (Moed et
al., 1985). This JCS represents the number of times, on average, a typical article, published in a
particular journal, is cited by other journal articles in the third year after its publication, often the year in
which a publication is cited most frequently.
The JCS can be calculated in the following way (an example of the calculation is presented in the
textbox below).
Total number of times articles, published during year t
in journal A, are cited in year t+3 by other journal articles
JCSA =
The number of articles published during year t in journal A
By comparing the actual number of citations per publication with the expected number, operationalised
by the JCSm value, an indication can be obtained on whether a group's articles are cited frequently or
not, relative to the citation rate of all papers published in the same journal set. This approach, however,
has one major drawback: the level or impact of the journals in which the research group publishes is not
taken into account. To give an example, let us assume that research group X publishes in prestigious,
high-impact journals, while group Y publishes in rather low-impact journals. Then, the citation rate of
papers published by both groups may obtain equal results relative to the average citation rate of their
respective journal sets. However, in this example, group X surely can be considered to have gained a
69
higher impact than group Y (Moed et al., 1995). As a result, it may be clear that this type of analysis can
only be used as a partial indicator for the actual impact of a research group's work.
Assume that 212 articles were published in Journal A during 1997, and they received 482 citations in 1999,
then the JCS of Journal A would amount to 482/212 = 2.27. Now, a research group's articles are usually
published in different journals with different JCS values. To account for this, a weighted-average JCS value
(JCSm) of the journal set in which the group's articles are published is calculated, with weighting factors
equal to the number of publications in the different journals. Now, assume that a particular research group X
published 32 articles in 1997, of which 18 were published in journal A, with a JCS of 2.27, and 14 in journal
B, with a JCS of 1.85. The weighted-average Journal Citation Score then equals:
JCSm =
(18 × 2.27) + (14 × 1.85)
18 + 14
= 2.09
This value represents the average number of citations received by the 32 articles published in Journal A and
Journal B in the third year after they were published. It is this JCS indicator that is used by Moed et al.
(1985) as an indicator of ‘expected impact’ based on the journal set in which a research group publishes.
A much more appropriate way to normalise citation counts is on a sub-field basis (Moed et al., 1995;
Narin & Hamilton, 1996). Instead of comparing the citation rate of papers, published by a group, to the
average citation rate of articles published in the same journal set, the former is now compared with a
world citation average for all papers published in the sub-fields in which the group is active. In other
words, instead of using journal citation rates, now sub-field citation rates are used as a reference level
(Moed et al., 1995).
The calculation of this mean citation rate of the sub-fields in which a group is active (FCSm, the mean
Field Citation Score) is done in exactly the same way as with the JCSm, only with journals replaced by
sub-fields. The definition of fields (or sub-fields) is usually based on a classification of journals into
categories developed by the ISI, although this classification is far from perfect (Moed et al., 1995).
Fields and sub-fields, thus, are operationalised by means of journal categories. A consideration that
applies to the use of the FCSm is whether the performance of multidisciplinary oriented groups can be
expressed in an adequate way in one single index.
VI.2.4. Overview of impact indicators
By way of summary, we will give a brief overview of the various impact indicators that are usually
calculated in order to make an assessment of the research performance of a particular research group
(Moed et al., 1995), research institute or an entire country or region (Noyons et al., 1998). The various
indicators are illustrated in the table below.
70
Table 3 – Overview of various impact indicators
Description
Indicator
Number of publications in SCI
P
Total number of citations received
Average number of citations per publication
C
CPP
CPP excluding self-citations
Percentage of self-citations
CPPex
%Self-Cits
Average journal impact factor
JCSm
World field citation average
FCSm
Citations per publication, compared to journal impact factor
CPP/JCSm
Citations per publication, compared to world field citation average
CPP/FCSm
Journal impact factor, compared to world field citation average
JCSm/FCSm
Source: based on Noyons et al. (1998) and Moed et al. (1995)
The first indicator is the total number of papers published by the research group, institute or country
under study (P). The next two rows consist of respectively the total number of citations received (C),
and the average number of citations per publication (CPP). The fourth indicator gives the average
number of citations per publication if so-called self-citations are excluded (CPPex). Depending on the
unit of analysis − research group, institute, or country − a different definition for self-citations are
employed. Usually, also the percentage of self-citations relative to the total number of citations received
is given (%Self-Cit).
Next, the mean citation rate of the journal set in which is published is given (JCSm, the mean Journal
Citation Score). In the seventh row, the mean citation rate of the (sub-) fields in which the research
group, institute or country is active is presented (FCSm, the mean Field Citation Score). As already
mentioned before, the definition of (sub-) fields is usually based on a classification of journals into
journal categories, as developed by the ISI. The FCSm can then be considered to reflect a world average
in a particular (sub-) field or combination of (sub-) fields.
In the eighth and ninth row the average number of citations is normalised both by the journal mean
citation scores (CPP/JCSm), and by the world average in the (sub-) fields (CPP/FCSm). If the ratio
CPP/FCSm is above 1.0, the work of the unit under study received more citations than an ‘average’
publication in the (sub-) field(s) in which the group is active. In this way it becomes possible to obtain
an indication of the international position of the unit under study, in terms of its impact compared to a
world average. This ‘world’ average usually is calculated for the entire population of articles published
in ISI journals assigned to a specific sub-field or journal category (Moed et al., 1995).
Usually, also the ratio JCSm/FCSm is calculated. If the mean citation score of the set of journals in
which the unit under study publishes exceeds the mean citation score of all papers published in the subfield(s) to which the journals belong, that is, if the ratio JCSm/FCSm is above 1.0, the conclusion is that
the unit publishes in journals with a high impact, i.e. journals with an impact above the world average in
the field.
71
VI.2.5. Graphic representation of impact analyses
Although all relevant information is incorporated in the values of the various bibliometric indicators, a
graphical presentation of the results often facilitates interpretation, especially with respect to
comparative purposes. The graphical depiction most frequently used to summarise research results is
one in which the average citation rate of the unit of analysis is compared to the world citation average in
the sub-fields in which it is active (e.g. Van den Berghe et al., 1998; Luwel et al., 1999). For illustrative
purposes a typical outcome of a study on the productivity and impact of a number of research
departments is depicted in figure 6.
Figure 6 − Graphical presentation of a productivity and impact analysis
2
1
I
m
p
0
a
c
t
-1
-2
0
100
200
300
400
Number of publications
Source: adapted from Van den Berghe et al. (1998) and Luwel et al. (1999)
Each circle represents a research department. The total number of articles published in SCI journals is
depicted on the horizontal axis. The vertical axis represents the log CPP/FCSm, which gives the impact
of a department during a given period of time, compared to the world citation average in the (sub-)
fields in which it is active. Circles above the horizontal reference line represent departments for which
the impact (CPP) is higher than the world citation average in the (sub-) fields in which they are active
(FCSm).
The impact of a faculty's research activities is represented in such a way that the performance of its
constituent departments is still visible. As such, this figure facilitates interpretation, since it constitutes a
useful synthesis between an analysis at the micro-level, the departments, and at the meso-level the
faculty (Van den Berghe et al., 1998; Luwel et al., 1999).
A slightly different, and equally useful graphic representation of an impact analysis, illustrated in figure
7, is presented by Noyons et al. (1998) in their paper on the assessment of Flemish R&D in the field of
information technology. The horizontal axis represents the world citation average (FCSm), while the
vertical axis gives the average citations per publication (impact). As a result, if a sub-domain (depicted
as a data point) is situated above the diagonal, its impact is above world average.
72
Figure 7 − Alternative graphical presentation of an impact analysis
8
Citations per Publication (Impact)
7
6
5
4
3
2
1
0
0
1
2
3
4
5
6
7
8
World Citation Average
Source: Noyons et al. (1998)
In addition, the size of the circles, which represent the data points, indicate the proportional numbers of
papers published by the unit under study, in this case Flemish researchers, in each sub-field, relative to
the total number of an actor's papers (here, again Flemish papers) in the whole field. Finally, different
colours are used to indicate sub-domains with high versus low JCSm/FCSm ratios. Noyons et al. (1998),
for example, used dark grey to indicate sub-domains with a JECS/FCSm > 1.2; light grey for subdomains with a JCSm/FCSm between 0.8 and 1.2; and white for sub-domains with JCSm/FCSm < 0.8.
If for a particular sub-domain the ratio JECS/FCSm > 1.2 (indicated by dark grey), this means the
impact of the journals used by the researchers is above the impact of the field.
VI.3. Indicators of collaboration
Another indicator that could shed some light on the research performance of, in particular, research
groups is an indicator measuring the extent to which research groups engage in collaboration. On the
basis of addresses of the authors, Moed et al. (1995) adapted an approach in which a group's papers
were assigned into three different categories reflecting three different types of collaboration. Papers
were assigned to the ‘no collaboration’ category, in the case they were single-authored or published by
more than one author from the same research group. Remaining papers were assigned to the category
‘collaboration type within the Netherlands’, in case the co-author(s) participated from other groups
within the Netherlands, or ‘collaboration type International’, when scientists from groups outside the
Netherlands were involved.
In an excellent study by Katz & Martin (1997) the whole concept of collaboration is being discussed
extensively. They point out that collaboration can take various forms ranging from offering technical
advice to active participation in the research. Based on a definition of collaboration they argue that
collaboration should be distinguished from co-authorship that at a certain point may be one of the
73
results of collaboration. This leads to the major conclusion that co-authorship is a rather approximate
partial indicator of collaboration and that it should be used with the necessary precaution.
The purpose of the indicator, staying in the context of Moed et al. (1995), is to show how often a
particular research group has co-published papers with other groups. Moreover it can give an indication
on the impact of co-published articles resulting from national or international collaboration, compared
to the impact of papers published by researchers from a single research group. As such, impact of
publications is analysed as a function of the type of collaboration.
Moed et al. (1995) find that all three ‘relative’ impact indicators − i.e. both the average impact
compared to the impact of the journal set (CPP/JCSm) or to the world citation average in the sub-fields
the group is active (CPP/FCSm), as well as the impact of the journal set relative to the world citation
average (JCSm/FCSm) − point in the same direction. All three indicators show the highest values for
publications resulting from international collaboration, and the lowest for publications in which no
collaboration with other groups was involved.
VI.4. Indicators of scientific activity
To compare research performance between different countries the calculation of the Revealed
Literature Advantage (RLA) is particularly popular. The construction of this RLA-index is exactly the
same as for the Revealed Technological Advantage (RTA) discussed in the chapter on patents. The RLA
is often being referred to as a specialisation index. For more details on this we refer to the discussion of
the RTA in the previous chapter. The formula for calculating the RLA is shown below.
åL
RLA =
åL åL
Lij
ij
i
ij
j
Lij
ij
ij
= number of publications of country i in sub-domain j
åL
= number of publications of all countries in sub-domain j
åL
= number of publications of country i in the whole field
åL
= number of publications of all countries in the whole field
ij
i
ij
j
ij
ij
This RLA-index indicates the country's share in the world's publication output in a particular sub-field j,
compared to the country's share in the world's publication output in the whole field. A value higher than
unity reflects a relative specialisation (in terms of publications) of country i in sub-field j. A value of
one signifies a neutral position, while a value smaller than one indicates a relative disadvantage of
country i in sub-field j.
As such, the indicator reflects whether a particular country engages in a particular sub-field to an extent
above or below average, relative to its other publication activities. Note that, this RLA-index does not
measure the absolute ‘strength’ of a country in a particular sub-field, but that it merely expresses the
74
relative amount of activity in a particular sub-field of a country that is engaged in several sub-fields. As
such, it should be interpreted as an ‘activity index’ (Hinze, 1997; Egghe & Rousseau, 1990).
Egghe & Rousseau (1990) also mention the existence of, what they call an Attractivity Index (AAI). Its
calculation is identical to that for the RLA-index, except that now citations are used instead of
publications. The Attractivity Index should characterise the relative impact of a country's publications
in a given sub-field as reflected in the citations they attract. As such it reflects a particular country's
share in citations attracted by publications in a particular sub-field j, compared to that country's share in
citations attracted by publications in the whole field. Again, a value higher (lower) than unity would
reflect a higher (lower) than average impact in the given sub-field, relative to the world's average.
VI.5. Intermezzo: one-dimensional vs. two-dimensional measurement and assessment
Before continuing our discussion with the second stream within bibliometrics focussing on the more
structural aspects of science and science fields, we will first briefly mention several considerations
about measurement and assessment. In Martin and Irvine's (1983) methodology of converging partial
indicators it is debated that all quantitative measures of research performance are, at best, only partial
indicators. It is stated that these indicators will only partly reflect what they are purported the measure,
according to Martin and Irvine (1983), contribution to scientific progress. However, a network of other
factors will also influence these indicators.
Their methodology of converging partial indicators then assumes that, although each indicator in itself
can only be considered, at best, to be a partial indicator, through the combined use of multiple
indicators, the influence of other, disturbing factors can be minimised (Moed et al., 1985; Tijssen,
1992). Then, in occasions where all, or a majority of the indicators point in the same direction − i.e. if
convergent results are obtained −, confidence may be attached to the results. Or, as Martin (1996)
himself, puts it:
‘ … a result based on the convergence of several indicators (preferably including extensive
peer evaluation) is likely to be more reliable than one based on a single bibliometric indicator
or on peer review alone (Martin, 1996, p. 352).
Although their approach has been criticised on several occasions (e.g. Moed et al. 1985; Moed, 1989), it
is widely accepted that an assessment based on multiple indicators − bibliometric and non-bibliometric,
should be more reliable than one based on a single indicator. Since any single indicator can at best only
capture one aspect of performance, it is argued that using a range of performance indicators, preferably
both quantitative and qualitative, is always better than relying on just one or two. So, rather than
attempting to measure research performance on a one-dimensional scale, it is much more adequate to
make an assessment based on a multidimensional set of ratings or some sort of profile (Martin, 1996).
75
VI.6. Mapping of science: techniques and utility for science policy purposes
VI.6.1. Mapping of science: an introduction
The enormous and still increasing amount of information on technology and science necessitates a
systematic and careful approach to achieve a sensible data reduction. Large numbers of complex tables
are mostly not very useful in this respect. New and additional ways of representing the data may reveal
these underlying and until then ‘hidden’ features (Engelsman & Van Raan, 1991).
Based on the analysis of information from the scientific literature, quantitative techniques are used to
display both structural and dynamic aspects of scientific research. Its main purpose often is to display
the foci of interest and attention that prevail in a particular scientific area (field or sub-field) in a certain
period of time (Braam, 1991). Cartography of technology or science reformats the data into a specific
graphical representation and also reduces the data while retaining only the essential information.
Maps of science are constructed to display relational aspects in science. Contrary to indicators in the
form of frequency lists, ranks or tables, maps can easily provide information on links between scientific
entities (Tijssen, 1992). Maps, in this sense, are particularly helpful for visualising the pattern of such a
large and/or complex structure inherent to the data.
In his work ‘Mapping of Science: Foci of Intellectual Interest in Scientific Literature’, Braam (1991)
identifies four steps in the bibliometric mapping of science:
(a) the selection of (a set of) relevant scientific documents covering an area of scientific research
and the subsequent collection of bibliographic data, derived from these scientific texts;
(b) the construction of a separate data set in which the data are structured in a way appropriate for
analysis;
(c) statistical analysis of the bibliometric data; and
(d) some guiding theoretical framework for the interpretation of the results.
The mapping of science is performed by means of relational, two-dimensional indicators which are
based on the analysis of the number of times different information items, such as author names, keywords, classification counts or citations, occur together (co-occurrence) (Hinze, 1997).
By
investigating connections through the ‘co-occurrence’ of references, words and/or classification codes,
it becomes possible to unravel the immense network of interrelated pieces of knowledge, and to
uncover major ‘hidden patterns’ in the vast amount of information carried by the scientific literature
(Van Raan, 1997). The result of such an exercise is often represented in a two-dimensional way, a
‘map’, in which information items (and the publications containing these items) are structured
according to their links, as uncovered in the ‘co-occurrence’ analysis (Luwel et al., 1999). As such,
mapping techniques make a systematic use of the information carried by scientific articles possible
(Hinze, 1997).
Engelsman & Van Raan (1991) distinguish three main types of bibliometric maps. The first one is based
on co-citations, the number of times two particular articles are cited together. A second type of
bibliometric mapping is based on co-word analysis, or more broadly speaking, based on co-occurrence.
Word co-occurrences reflect the network of conceptual relations from the viewpoint of the scientist in
the field concerned. A third type of bibliometric maps, somehow artificial in our view, is the coclassification map, for example based on the co-occurrence of different classification codes.
76
Different techniques, making use of different information items included in scientific articles, are used
in order to map scientific fields, as well as their development over time. It should be pointed out that it
is not only possible to map scientific fields but also, by applying the same techniques, creating maps of
technology (Engelsman & Van Raan, 1991; Korevaar & Van Raan, 1992). The specific techniques will
be discussed briefly in the next section.
IV.6.2. Several techniques
Within the domain of the mapping of science, four principal types of bibliometric maps can be
distinguished: (1) journal-to-journal citation maps; (2) co-classification maps; (3) co-citation maps;
and (4) co-word maps. Mapping analyses using co-citation data or co-word data are by far the most
popular (Tijssen, 1992). Especially the co-classification maps may prove of importance for analysing
the internal dynamics of the science and technology link.
(a) Mapping by means of data on journals
In this approach maps of science are derived from existing inter-journal networks. By examining the
links between journals as created by citations given to and received from other journals, it is believed
that the macro-level structure of scientific activities can be pictured. The assumption underlying this
approach is that inter-journal citation-frequencies reflect the magnitude of subject relation between
journals (Tijssen, 1992).
(b) Co-classification maps
The structure of scientific fields may be studied by representing the relations between sub-fields as
described by classification codes. As a rule, different classification codes are assigned to an individual
publication. As such, the co-occurrence of classification codes (or subject-classification terms) can be
studied. The number of times different classification codes occur together is taken as a measure for
similarity. By means of multivariate statistical methods this similarity can be represented in a ‘map of
science’, in which the structure of the analysed scientific areas becomes visible. Examples of studies
based on co-occurrence analysis of classification codes are: Engelsman & Van Raan (1991), Van Raan
and Peters (1989).
The main advantage of this approach consists in the relative ‘straightforwardness’ of the method. Since
all classification codes have well-defined meanings, the interpretation of the resulting map should not
pose much problems. However, this relative ‘straightforwardness’ in the same time also illustrates the
main drawback of this method. Since this kind of analysis is based on an existing classification system,
it can never reflect the development of new scientific (sub-) fields in an adequate way, simply because
classification systems are not updated on a continuing basis (Hinze, 1997).
(c) Co-citation maps
The mapping of the structure of scientific research can also be done on the basis of citations given by
authors in their publications. This is done in co-citation analysis, initiated by Small at the Institute for
Scientific Information (ISI) in Philadelphia (Small, 1973), and nowadays one of the major quantitative
techniques used to map both the structure and the dynamics of scientific (sub-) fields. In co-citation
analysis information on how often pairs of articles are cited together in other papers is used to construct
maps of scientific research (Hinze, 1997). The resulting frequency of co-citation is used to measure the
degree of association between two documents. The clustering of documents in co-citation analysis thus
is based on existing co-citation relations and, as such, this approach is considered to be an alternative to
existing classifications of scientific research activities (Braam, 1987).
77
(d) The technique of co-citation analysis
The first step in co-citation analysis usually concerns the selection of documents that are cited more
than a specified number of times, from the reference lists of set of (citing) publications published in a
particular year. Only publications exceeding this ‘citation threshold’ are included in the analysis. The
use of such a ‘citation threshold’ is used both to separate signal from noise, and to save computer time
(Braam, 1987).
Next pairs of publications are selected that co-occur relatively frequently in the citing publications
reference lists. Two papers being cited together are considered to be linked. The number of times two
publications are co-cited is a measure of the strength of this linkage. By specifying an integer ‘cocitation threshold’ or by using a specific index and defining some threshold value for this index, a
distinction is made between significant and insignificant pairs of co-cited items, that is pairs with a
relative strong versus a weak link. Pairs of publications exceeding this specified ‘co-citation threshold’
are included in the mapping exercise, while pairs not reaching this threshold are discarded.
Several indices can be used to measure this co-citation strength. By far the most popular ones are
Salton's formula and the Jaccard Index (Sneath and Sokal, 1973). Finally, the pairs of co-cited
publications that reach the above mentioned ‘co-citation threshold’ participate in a clustering routine.
An often used clustering routine is ‘single linkage clustering’ (Braam, 1987), in which clusters of
documents are aggregated by sequentially linking together all selected pairs of documents having at
least one document in common. Clustering thus brings together document pairs with high co-citation
strength relative to the number of citations received by the individual articles. For more details on cocitation analysis and the construction of maps based on co-citation analysis we refer to Braam (1987);
Mombers et al. (1985) and Tijssen (1992).
At this point we would like to mention a number of limitations of maps based on co-citation analysis.
Co-citation analysis has been criticised on several occasions. Hicks (1987) identifies five major
limitations of co-citation analysis as a tool for science policy:
1. Delayed inception of clusters
In general, a time lag can be perceived between the emergence of new specialities and their appearance
on a co-citation map in the form of a new cluster. Hicks (1987) indicates that in some fields the number
of researchers may grow rather quickly, resulting in a relatively short time lag before their work appears
on a co-citation map. However, in other fields where it may take some years before a critical mass is
reached, this time lag may be more substantial.
2. Inconsistent definition of the (sub-)field
Although the identification and demarcation of ‘specialities’ is extremely difficult, strict boundaries
have to be constructed for the purpose of analysis. First of all, definitions of fields, sub-fields and
specialities of scientific researchers, on the one hand, and the bibliometric analyst, on the other, should
agree in broad outline. Moreover, in order that observed trends should reflect actual changes in national
research effort, definitions should be consistent over time.
3. Effects of errors in citations
As already mentioned before, obviously co-citation clusters are also affected by errors made by citing
authors.
78
4. Under-representation of experimental papers
On several occasions, it is found that papers with an experimental nature are relatively underrepresented in co-citation clusters and theoretical papers are relatively over-represented.
5. Subjectivity in co-citation analysis
Finally, the subjectivity inherent in the setting of both the ‘citation threshold’ as the ‘co-citation
threshold’ has been a point of critique. This is especially so because the choice of these threshold levels
can strongly affect both the size and the content of resulting clusters.
(e) Co-word maps
The last technique that shall discuss is the technique of co-word mapping. In the field of ‘mapping of
science’, co-word analysis has been developed as a rival technique to co-citation analysis by Michel
Callon and his colleagues at the Centre de Sociology de l'Innovation of the École des Mines de Paris, in
co-operation with British and Dutch scientists (Callon et al., 1983; Callon et al., 1986). Whereas cocitation analysis builds on co-occurrences of pairs of cited documents in publications, co-word analysis
focuses on the co-occurrences of pairs of content words related to these publications. These words can
consist of manually or automatically established terms (free terms or controlled terms), words appearing
in the title, in abstracts, or from the full text (Braam et al., 1989).
Assuming that words designate specific loci of interest, the amount of publications associated with a
given word then provides an indication of the number of people involved and time invested in research
activities focusing on that particular ‘locus of interest’. The degree of overlap between distinct loci of
interest can they be measured by the amount of times words occur together in a set of publications. If a
set of words appears to co-occur relatively frequently, this can be interpreted as constituting a broader
‘problem area’ or ‘research theme’. Interrelations between distinct problem areas or research themes
are then indicated by co-occurrences of words from these different areas (Braam et al., 1989).
In order to map the structure of science, co-word analysis can be based on indexing terms, ‘controlled
terms’, which are externally supplied by journal editors or professional indexers at documentation
services, or on key-words, ‘uncontrolled terms’, terms supplied by scientists themselves (Tijssen, 1992).
The use of ‘controlled terms’ in a mapping exercise resembles the before-mentioned co-classification
analysis, since either some pre-specified terms or some pre-specified classification code is assigned to
the publication. As a result, like in co-classification analysis, when performing a mapping exercise by
using ‘controlled terms’ most recent developments will most likely not be reflected (Hinze, 1997).
Moreover, when using ‘controlled terms’ or classification codes, the result of the mapping exercise may
be flawed by what is called an ‘indexer effect’ (Healey et al., 1986). Since both ‘controlled terms’ and
classification codes are assigned to publications by professional indexers, their view about a particular
scientific paper may influence the results of the analysis in cases where the assigned terms or
classification may not correspond with the authors view. Therefore, co-word analysis based on
‘uncontrolled terms’, key-words provided by the authors themselves, rather than terms given by the
database producer should be used by preference (Hinze, 1997).
The resulting network of co-occurrences between different words, collected from a set of publications,
can then provide a detailed insight in the structure of the publication contents. By comparing
publications with respect to the occurrence of similar word-pairs, co-word maps provide a direct
quantitative way of linking the conceptual contents of publications. As such, through the relationships
between various research themes, as reflected in the occurrence of word-pairs, research activities within
79
a particular scientific domain can be depicted (Tijssen, 1992). A computer program, LEXIMAPPE, was
developed for obtaining graphical representations of co-word maps (Callon et al., 1983).
Compared to co-citation maps, co-word maps are found to be more inclusive and more up to date, in
that the emerging specialities are included relatively more quickly in co-word maps. The main
drawback of co-word analysis is that words, or at least their meanings, are not always unambiguous and
are not seldom context dependent (Hinze, 1997; Van Raan, 1993).
However, Braam (1991) mentions that co-citation and co-word analysis should not so much be seen as
competing techniques, but rather as complementary, since they both draw on fundamentally different
sources of data. Indeed, whereas co-word analysis draws on data directly related to the content of
current research activities, co-citation analysis draws on the content ascribed to the intellectual base, as
reflected in earlier literature, on which this current research is based. As such, the extent to which coword and co-citation maps converge can give an indication on the relation between a shared intellectual
focus on particular topics of research − as reflected in the co-word map −, and the intellectual focus on
the relevance and importance of earlier work on these topics − as reflected in the co-citation map
(Braam et al., 1989; Braam, 1991).
Braam et al. (1989) stress that the combined use of the co-word and the co-citation technique for one
and the same bibliometric data set offers an excellent tool to investigate the relationship that exists
between two different types of intellectual focus in scientific research: shared attention to a common
body of base literature, on the one hand, and shared attention to research topics, on the other.
Besides the combined use of different techniques for the mapping of science, Tijssen (1992) also
stresses that such maps should preferably be used in combination with other information sources, such
as opinions of independent experts in the scientific (sub-) fields under study. However, this use of
bibliometric maps in combination with an expert's opinion may lead to a dilemma in the use of science
maps:
" … if the expert's 'mental map' and the bibliometric map look alike, it is often argued that the
map adds nothing to the knowledge; if, on the other hand, the structures differ considerably, the
validity of the map of science is questioned. In both cases the map may be discarded as useless.
On the other hand, if maps visualise features of the structure or development of a field that is
surprising to experts in the field it sometimes triggers criticism or disbelief. "(Tijssen, 1992, p.
35)
Moreover, maps of science are, by definition, only artificial ‘snapshots’ of abstract structures within the
science system, which in addition are in a constant state of ongoing development (Tijssen, 1992).
IV.6.3. Utility of mapping exercises for science policy purposes
Let us assume that a mapping exercise by means of co-citation and/or co-word analysis indeed makes it
possible to display the cognitive structure of scientific (sub-) fields. In that case it would prove very
interesting, from a science policy point of view, to explore the research activity of a particular country
or region in the various (sub-) fields identified in the resulting map of science. Investigating the
distribution of publications, research institutes, or countries over these research areas can do this.
80
Mombers et al. (1985) employed co-citation analysis to assess Dutch participation in science. In
particular, analyses were performed by identifying the Netherlands contribution7 to both the cores − i.e.
the cited − and the citing literature. Useful inferences could be made from these analyses. For example,
if a particular cluster showed a strong Dutch participation in the core, this indicated a strong interest in
the past. If at the same time one also observed a substantial Dutch share in the citing literature, this
indicated a continuous strong interest.
If a significant Dutch participation was observed only in the citing literature, the interest in that
particular (sub-) field was either marginal or had only started recently. An alternative explication for the
latter case could be that older Dutch publications on the subject under study had little impact and
consequently had not been cited enough (Mombers et al., 1985).
A similar approach was adopted by Noyons et al. (1999) in a combined performance/mapping study
used to evaluate the activities of IMEC, a Belgian research institute in microelectronics. First, a
mapping exercise based on classification codes was performed. As a result of this exercise, the
cognitive structure of microelectronics, as defined by the publications of seven of the major institutes
active in this field, was displayed, with clusters representing the various sub-domains of the field.
Subsequently, the relative activity of the research institute in terms of number of publications and their
impact was assessed, compared to both the world average and the performance of selected benchmark
institutes.
As such, this combined procedure can be used for monitoring research performance on a micro-level,
while at the same time taking into account recent developments in the field. In this way, an assessment
of the performance of any actor, being either a research institute, a university department, or a country,
can be made, relative to its peers and from a dynamic perspective at once (Noyons et al., 1999).
Engelsman & Van Raan (1991) presented a study focussing on knowledge diffusion in fields of
technology using mapping techniques. They constructed maps on three levels. The first level, the macro
level (co-classification - and co-word analysis offered some interesting insights on the role of Japan in
chemical, mechanical and electronical industry. At the meso level, maps were used to visualise
structures of activity (performed for the emerging ‘crossroad’ technology of Optomechatronics). At the
micro level specific fields were analysed revealing interesting insights on innovations and technological
improvements. They also introduced the concept of ‘technological peripheries’, identifying the most
closely linked fields.
Finally, some comments on the usability of maps. As Noyons & Van Raan (1998) mention, for a long
time maps were perceived as interesting overviews but difficult to interpret, or just to read.
Furthermore, it seemed that the richness of information on the maps was not sufficient, in the sense that
traditional overviews would also provide the same information. The conclusion based on these
shortcomings is that mapping, as a policy support tool, had not yet reached a mature state. However,
according to Noyons & Van Raan (1998) mapping techniques and thus maps, especially those
developed at CWTS (Centre for Science and Technology Studies), have been improved. They mention
the following improvements: multi level mapping, ‘map external’ information included in the maps,
graphical user interface to browse through maps and added information, and animated representation of
positioning of sub-domains in main map of fields. Their applicability has been improved.
7
A Netherlands contribution was defined as an article originating from a research institute, company or university
situated in the Netherlands.
81
Part III: Science and Technology - Examination of the interaction
CHAPTER 9 - LINKING SCIENCE TO TECHNOLOGY
I. General introduction
In the previous two chapters we discussed the most important possibilities and drawbacks for measuring
science and technology in view of their close relation to research and industrial development. Many
scholars have pointed out that in several technological areas, it is becoming increasingly difficult to
differentiate between science and technology because of the multi- and interdisciplinary nature of
knowledge production and the increased number of technology fields linked to science development
(Meyer-Krahmer, 2000). In the literature, the approximation of scientific development towards
technological evolution is coined by terms such as: “Science base of technologies” (Meyer-Krahmer &
Schmoch, 1998; Van Vianen et al., 1990), “Scientification of technologies” (Narin et al., 1997) or
“Science-Driven Market” (Grupp, 1998). It thus becomes more important to focus on the interaction,
and not solely on the two systems independently.
With De Solla Price (1965) and Rosenberg & Birdzell (1990), there has been quite some qualitative
understanding of the linkage between science and technology. Yet until the ’90s there has been very
little quantitative data to specifically characterise this relationship or to pinpoint the subject, national,
international and temporal aspects of the coupling between science and technology. The increasing
number of constraints and developments that governments are facing - constraints on public
expenditure, pressure on the peer review groups, emergence of new scientific and technological areas
etc. – more than ever before, lead to the necessity of ex-ante prioritisation and ex-post evaluation in
science and technology. As such, a tool that enables science interactions and science bases of
technologies would surely contribute to facing this challenge.
The central role, but also the changing nature of ‘knowledge’ in technological progress has already been
discussed extensively in part I. Patents and scientific publications are in their function of ‘carriers of
relevant knowledge’, useful ‘proxies’ of scientific and technological activities (for more details, see part
II). Due to the information contained in these documents, it becomes possible to model the relational
aspects of science and technology and to delve deeper in to the social and cognitive aspects of the
interrelation. Obviously, the presence of cross citation between both spheres is the vehicle towards this
integrated analysis. However, a number of trivialities have to be kept in mind and adequately dealt with
in respect of the purposes for using them.
II. Background on the science and technology interaction
II.1. Importance of the S&T interaction
Many scholars have put forward the importance of scientific advancement for technological
development. The interface between both is the nursery of research and development activities and is
undoubtedly a major driving force for the economic development of our world. Mansfield (1991) has
added a quantitative underpinning to this; from his survey data of 76 large US companies, he estimates
that 11% of industrial innovations and 9% of the process innovations would not have been developed in
time without the contribution of academic research.
82
In the early 1960s, Toynbee (1963) and De Solla Price (1965) already extensively discussed the science
– technology interaction. Toynbee’s well-know metaphor of this interaction as a pair of dancers typifies
very well the assumed importance of their simultaneity and co-development. Using this metaphor two
decades later, Narin et al. (1989) analyse the relation between patents in biotechnology and papers in
bioscience. They point out that not only ‘both partners’ dance together in the same rhythm, but that they
are also locked in an embrace from which it is virtually impossible to separate them. At the same time
they argue that, based on the increased number of articles cited on the front-page of patents in the last
years, technologies have become increasingly ‘scientized’. A refinement has been made in this
proposition in 1997, when Schmoch (1997), but also Narin et al. (1997) and others, have indicated that
especially modern technological areas are often highly scientific.
The (increased) importance of science for technological development has also been reflected in the
categorization of manufacturing industries. It was Pavitt (1984) who introduced the science based sector
in 1984, obviously implying that also from an economic classification point of view, a broad category
of technological activities around numerous companies exist, in which innovations are directly linked to
technological advances made possible by scientific progress. Furthermore, innovation-oriented
activities of the relevant companies are usually formalised in R&D laboratories in which investments in
research and development appear to be disproportionally high. According to Pavitt (ibid.) the electronic
industry, large parts of organic chemistry industry, the pharmaceutical and biotechnology industries, air
and space travel and the military technology industry can be classified in to the science-based sector.
Due to the science – technology interface, market economies of western nations have achieved
unprecedented prosperity. As such, science and technology are of vital importance for the performance
of a national economy, as numerous discussions concerning the international competitiveness of
national economies currently show (cfr. part I). The need for a better understanding of the interaction is
obvious, in view of the dynamic of the science and technology interaction, the growing complexity of
the interrelation (cfr. infra), the importance and necessity for more transparency in science funding and
return on funding. Internationally, within the last few years, a growing interest has been developed in
trying to foresee and understand future developments.
II.2. Science and technology interrelation: who leads and who follows?
A. The changes in the knowledge production and diffusion system
Continuing the discussion on knowledge creation and diffusion touched upon in part I, but now
extended by the context of the more general science and technology interaction, we shall look into the
factors that determine the direction in which both Toynbee’s (1963) ‘dancers’ move. Understanding this
mechanism is of great importance if one attempts to ‘intervene’ in the interaction process (numerous
actions are thinkable).
Most of the studies dealing with the subject are based on the so-called linear model of knowledge
production, diffusion and utilization (cfr. section 3.IV.). It is assumed that scientific progress is directly
followed by a technological application. According to Meyer (2000a), it appears that the work of Narin
and his colleagues is primarily based on the ‘linear’ understanding of the innovation process. This is
also reflected in the explanatory approach of science and technology interaction that they apply (based
83
on paper citation analysis), which implies that a scientific paper cited in a patent has a causal
implication, specifically that science directly leads to technological development.
The state of the art on the knowledge production and diffusion paradigm has to be aligned with the view
on science and technology interactions, and with the analytical approach of studying that interaction.
Meyer-Krahmer (2000) identifies two phenomena (‘new paradigms’) that indicate a structural change in
both the production and the diffusion of knowledge. The first one is the ‘increasing number of
technology fields with close linkages between basic research and industrial application’. The second
phenomenon is the ‘growing importance of multi- and interdisciplinarity, which is a reflection of the
fact that separation of technologies becomes more and more difficult and that the overlapping areas are
often highly dynamic. Both paradigms will be discussed briefly.
The increasing number of technology fields with close linkages between basic research and
industrial application
According to Schmoch et al. (1993), since the sixties a growing number of retrospective studies have
tried to evaluate the dependency of past innovations on scientific work. The first of these large-scale
studies was the so-called “Project Hindsight” of the US Department of Defence. The study assessed the
contribution of science and technology to the development of 20 weapon systems. One of the
conclusions was that basic research seemed to be rather unimportant, whereas applied research had the
highest payback. As a reaction to these negative findings, the National Science Foundation launched a
second study under the acronym “TRACES”. The findings of this study provided evidence to support
the ‘science-pull’ model of innovations.
§
§ Increasing inter- and transdisciplinarity
A second major observation is the growing inter- and transdisciplinarity between S&T, but also within
each of the two systems. Many large national studies have been performed in order to monitor and
identify key future technologies related to economic potentials. Examples are the Delphi inquiry
“Future Technology in Japan”, carried out by the Science and Technology Agency (STA), the study of
the German government “Technology at the Threshold of the 21st Century” and the study performed by
the French Ministry of Industry (1995) “The Key Technologies for the French Industry”. Al these
studies aimed at identifying future relevant developments in the existing science and technology
landscape. In the study performed for the German government (Grupp & Schmoch, 1992b), it was
stressed out that technology at the beginning of the next century couldn’t be separated according to
conventional disciplines (see table 4). Whatever the paths of technology emergence, they are all linked
together.
Table 4 - characteristics of traditional and modern production of scientific and technological knowledge
Traditional systems
Discipline based
Internally driven
Individually dominated
Modern systems
Transdisciplinary
Practically oriented
Network dominated
Source: Gibbons et al. (1994)
According to Gibbons et al. (1994), the mode of organisation of science and technology in the most
advanced sectors, is leading to the above-illustrated transition. In view of the social embeddedness of
science, the linear model of knowledge production and diffusion is inadequate for capturing the science
and technology interaction in all its aspects. In a study performed by Grupp (1998), the interplay
84
between science and technology shows that scientific activities, at least in the area of laser technology,
do not precede technical endeavours but almost simultaneously turn to the new paradigm, confirming,
what is also argued by Meyer (2000a), that the S&T relation is reciprocal instead of linear. Figure 8
illustrates the difference between the ‘linear interpretation’ (model A) of the science and technology
relation and the network model (cfr. section 4.IV.) or ‘two-branched model’ (model B), a term coined
by Rip (1992). The two-branched model is based on two kinds of activities:
(1) Exploitation understood as technological development, pilot processes and feedback.
(2) Exploration to increase understanding.
The figure illustrates that basic science research, or science, produces a number of scientific research
papers, indicated here as copyright symbols (ã). Technology at a certain point in time materialised in
patents illustrated by trademark symbols (â). The linear approach is being represented in A and the
network model or two-way model in B.
Figure 8 - (A) Linear interpretation (B) Network model or Two-way interpretation
A
B
Science
C
Science
C
C
C
C
C
C
C
C
C
C
C
Highly
mediated
R
R
R
R
R
R
R
R
R
R
Technology
R
R
R
R
R
Technology
Source: Meyer (2000a)
B. Discussion of the science – technology interaction in the light of cross-citation analysis
The model of Rip (1992), and its discussion by Meyer (2000a), clearly illustrates how the S&T
interrelation can be grasped and discussed by patent citation analysis. Figure A describes one way of
interpreting the data of Narin and of many other researchers by looking into the S&T relationship in a
rather ‘direct’ way (assuming that science and technology are represented by paper and patents, and that
the cross-citations represent the interaction). The linear approach as such implies a continuum that
stretches from scientific research to technological application. On the contrary, figure B shows that the
S&T interrelation is highly mediated, even though one might find occasionally direct relationships
between a patent and the scientific paper it cites
Meyer (2000a), and Grupp & Schmoch (1992a) have also looked into the relation between paper
references cited in patents and the interpretation in terms of science – technology interaction. Meyer
(ibid) studied this issue in 10 cases (patents) by looking into the motives of inventors for citing nonpatent literature (cfr. section 9.IV.1.1.). The cases concerned USPTO patents in the young and emerging
field of Nanotechnology. Researchers seem to integrate scientific and technological activities
increasingly by working on one subject matter and generating scientific papers as well as technological
output. Grupp & Schmoch (ibid) also concluded that there are several motives for a patent examiner to
cite scientific papers in a patent as prior art, and that science cannot always be assumed to trigger
technological development (see also Tijssen & Buter, 1998). Several writers in the literature on
innovation and technical change have emphasised that information contained only in scientific papers
85
will not suffice to implement the technology in question (see for e.g. Pavitt, 1987; Rosenberg, 1990).
This suggests that there must be more mechanisms in the linkage between science and technology, and
that the interplay is not so straightforward. The early discovery of food preservation in tin-coated steel
by Nicholas Appert in 1810, and the explanation of this process much later in 1873 with the discovery
of the role of micro-organisms in food spoilage (i.e. the birth of the science of bacteriology) provides an
early example of science lagging behind technological development (technology-pull).
III. Implications from a policy perspective
Keeping in mind the importance of the interaction and cross-fertilization between science and
technology, and the different aspects of this interaction, Meyer-Krahmer (2000) discerns several
implications for science and technology policy making. Before turning to these implications, let us first
briefly look into the context in which the policy interest becomes manifest.
As Steinmueller (2000) pointed out, in the past two decades, wide-ranging socio-economic and
technological transformations have caused European governments to reformulate their policies
concerning government-supported scientific activity. This reformulation has been accompanied by
shifts and even by complete turnarounds in research funding across fields of inquiry as well as in the
type of research conducted (basic versus applied). The present constraints on public expenditures in
general, the enormous investments involved in sustaining the econo-techno-scientific complex, and the
actual debate on the effectiveness of government supported scientific research, all augment the need for
more accountability and effectiveness in the area of publicly funded research (Hanusch, 1999; Ziman,
1994; Moed, 1989). The dependence of technological development on scientific progress has been
subject to much examination and speculation (see for instance the analyses and the discussions reported
in the European Science and Technology Indicators reports by the EC (European Commission, 1997)),
or even the ongoing discussion on benchmarking Industry – Science relations in Europe).
Let us now return to what Meyer-Krahmer (2000) refers to as “ implications for science and technology
policy making”. A first implication is the need to promote university/industry relationships. A long
tradition exists of different institutional settings – such as personal contact and co-operative research
institutes and network approaches – and financial incentives – such as subsidies or tax incentives for
extramural R&D or R&D co-operation projects –. It seems that transferring individuals is an efficient
way of technology transfer. Increasing the mobility of R&D personnel between industries, universities,
research institutes and government should therefore be one of the top priorities.
Both types of promotion of technology transfer are mainly elements of a technology-push approach.
Demand however is also a very important determinant of industrial innovation and technology
development. In areas where comparative advantages or high national or international demand is
expected, a concentration of R&D activities is needed. Indeed, national technology should not be
restricted to its own national R&D activities, but should also respond to international demand, which
consist a second implication. Even if a country does not have comparative technological advantages, it
can act as an imitator, using effective ways of technological development and avoiding first-mover
strategy with high learning costs.
A third consequence is that new ways of linkages between basic and applied research need to be
developed. Also, the facilitation of linkages between disciplines on the one hand and of trans- and
interdisciplinarity on the other, are crucial. In order to do this, special attention must be paid to:
86
§
§
§
§
§
Organisation of research: problem orientation in the case of well defined social or industrialtechnical problems, requiring new ways of project organisation and management.
A better integration of the long-term application-oriented basic research in order to respond to
future needs in a better way.
Team research: besides the currently predominant orientation of academic research towards
individualised research setting, team research must be strengthened.
Improved intra- and inter-sectoral mobility of researchers: on an international level and also
between science and industry.
Increased flexibility of research structures: more rapid take-up of new developments through:
- Flexibilization of the present rigid public service rules and budget laws
- Deregulation of the academic administration
- Networking of research institutions for a limited time, especially in an international
framework
A fourth consequence relates to university training. The increasing link between science and
technology, as well as trans- and interdisciplinarity, lead to an increasing demand for Ph.Ds in
engineering and natural sciences. Firms will invest in countries in which this academic human potential
is available.
A last remark that has to be kept in mind is that, in the past, government policy emphasised basic and
public R&D, leaving industrial and applied R&D over to firms (cfr. chapter 3). However, due to the
increasing interdependence between public and industrial R&D, industrial R&D may not be able to
increase without an increase of public R&D. It seems that the traditional institutional delineations have
to be reconsidered.
IV. Exploration of the science – technology interaction
In this section, we present a number of available approaches for investigating the interrelation between
science and technology (S&T). Besides the discussion of the different approaches, we shall also discuss
the main advantages and disadvantages. The focus however lies on S&T interaction analysis based on
non-patent references – cross citations from technology to science.
With respect to the quantitative description of science, both input-related and output-related indicators
can be used (cfr. figure 2). Frequently used input indicators include the volume of research budgets and
the number of scientists involved. However, since indicators of research budgets and personnel are
usually only available on a highly aggregated level, the quantitative description of science usually
occurs in terms of scientific output. Scientific output is generally measured through the number of
scientific articles published by a particular researcher or research group. As described elaborately in the
section on bibliometrics (cfr. chapter 8), a multitude of derived indicators can be constructed on the
basis of publication output measures. Examples of derived indicators include productivity and impact
measures. Analogously, technology can be analysed in terms of input indicators, such as the amount of
research expenditures or the size of the research staff, and output indicators, such as the number of
patents and other derived indicators (Schmoch, 1997).
Grupp (1998) states that it is one of the most difficult tasks to establish a measurement specification for
the S&T interaction. He states that: “In view of the poorly developed state of research, it is hardly
surprising that, unlike the case of the patent statistics treatment of technological spillover effects, there
87
are hardly any economic expressions to identify the science base of technology” (ibid, p. 331). One of
the exceptions, however, is the situation in which the patent examination leads to references to scientific
literature indicating and delineating the so-called ‘prior art’.
In order to describe the relationship between science and technology in a quantitative way, we can
either focus on the existing relationships between some of the aforementioned indicators, or we can
construct relevant derived indicators ourselves. However, since the main point in the analysis of the
science – technology interaction is the demarcation of the knowledge movements between both spheres,
one should aim to identify those proxies that best represent science and technology. Moreover, input
indicators such as research budgets and personnel are usually only available on a relatively highly
aggregated level, and hence are not suitable for analysis of particular technology areas, mainly output
indicators based on patents and publications are used for this purpose (Schmoch, 1997). Patents and
publications, due to their knowledge (information) carrying character, are best suitable for identifying
the cognitive and social networks underlying the science and technology interaction. However, by using
these proxies only a certain type of science – technology interaction can be grasped, namely the intacit
and profound variant of the interaction present in patents and publications.
In order to grasp the more tacit forms of science – technology interaction, other approaches need to be
applied. As a consequence, we distinguish between direct, explicit forms of S&T interactions, and
indirect, implicit forms. As explained, the first type implies that the science relation is a clear and direct
one (Noyons & Van Raan, 1994). This is however not always the case, implying that other approaches
that can reveal more implicit forms of interaction need to be applied. These approaches, “…do not
identify links between science and technology which are actually present, but rather links which could
be there, or even should be there” (ibid). Based on this distinction, we can discern between several
approaches for investigating the link between science and technology (Schmoch, 1997; Noyons & Van
Raan, 1994). The table below present the different analytical approaches.
Table 5 – Overview of the different approaches for analysing the S&T interrelation
Direct/explicit S&T interrelation
q
Indirect/implicit S&T interrelation
Citations to scientific publications in patents
q
Patents of scientific institutions
q
Publications of industrial enterprises
q
Co-activities
scientific
(joint
activities)
institutions
and
between
industrial
enterprises
q
Parallel
observation
of
patents
and
publications
q
Cartographical
occurrences
of
approach
based
publication
and
on
co-
patent
keywords
In the following sections, these approaches will be discussed more in detail. The direct S&T linkage
approach will be discussed first, followed by the more indirect linkage approaches.
88
I. Direct/explicit S&T interrelations
IV.1. Citations to scientific publications in patent documents
In the section on bibliometrics (cfr. section 8.IV.) the concept of citation analysis was already
introduced in a purely scientific context. We mentioned that every scientific publication contains
references to other publications of which the content in the mind of the authors is related in some way
or another to their own article. These references - or citations - can then be used for a variety of
purposes, of which the assessment of research performance of particular actors and the mapping of
scientific (sub-)fields are two of the most common approaches.
In a science-technology interface context, however, citation analysis involves the study of citations to
scientific publications in patents. In principle, the logic applied in the field of bibliometrics - where a
citation is perceived as an information item linking two scientific papers - is simply transposed to a
science-technology context - where a citation is perceived as an information item linking a particular
patent with a particular scientific article. The simple "translation" of this logic from a science context to
a science-technology context however, does not occur without a flaw (cfr. section 9.IV.1.). Assuming
that patents represent technology and publications science, the references to non-patent literature
(NPRs) in patents search reports can be regarded as an indicator of the science base of technology
(Schmoch et al. 1993).
According to Collins & Wyatt (1988), Schmoch et al. (1993), Narin et al. (1997), Meyer (2000a) and
Meyer-Krahmer (2000), studies of patent-to-journal citations are inherently interesting because of what
they reveal about the linkage between science and technology. Furthermore, Grupp (1998) points out
that patent citations are more objective than citations from scientists or technologists in their own
publications. Various studies, among which the above mentioned, verify and also demonstrate that
references to scientific publications are an appropriate indicator for describing the S&T relation.
Before investigating this issue in more detail, a few introductory remarks on patents and patent citations
seem necessary. First of all, a patent document consists of three elementary parts: (1) the title page or
front page with bibliographic information, (2) the text, which includes a description of the invention,
preferred examples in detail as well as drawings, diagrams, and flow charts, and (3) the claims. For
citation studies, most of the relevant information is found on the front page of the patent document
(Meyer, 2000a). It includes the following information: title and number of the patent, name and address
of the inventor(s), name and address of the assignee(s), date of application, issue date, details of
searches carried out by the examiner, abstract and drawing (when applicable) of the invention,
classification code(s) and bibliographic information (Collins & Wyatt, 1988; Meyer, 2000a).
For citation studies, the bibliographic information available on the front page of the patent document is
most important. In particular, we can distinguish between two types of references in a patent document:
(a) patent references, and (b) non-patent references. Obviously, the second type of references is
particularly interesting if one wants to investigate the link between science and technology. In the
following sections, we will subsequently discuss the concept of NPRs and the related citation motives,
the nature of the citation link established by NPRs, a number of indicators that can be constructed, and
the usefulness of this approach from a policy perspective.
89
IV.1.1. Non-patent references (NPRs)
Non-patent references (NPRs) are a mixed set of references to scientific journal papers, meetings,
books, and many non-scientific sources such as industrial standards, technical disclosures, engineering
manuals, and every other conceivable king of published material (Perko & Narin, 1997). In particular,
the references to scientific literature provide empirical evidence that a technical invention is in some
way related - or perhaps initiated and/or stimulated - by research activities (Tijssen, 2001).
The number of NPRs that can be found in patents differs strongly. This can be partly due to the
characteristics of the patenting offices (external factor) or due to the characteristics of certain fields of
technology (internal factor) - cfr. infra. According to Meyer (2000a), only a minority of patents contain
references to non-patent literature. He cites a Norwegian study of the knowledge base of certain
technologies (Iversen, 1998; op. cit. Meyer 2000a) showing that only 30% of Norwegian-oriented US
patents contained NPRs. One explanation for this is the fact that the stock of prior art references of the
Patent Offices mainly consists of patent literature. For example, the German Patent Office has about
10% of non-patent literature to refer to.
For patents to cite substantial numbers of publications in journals, they must be situated in a field that is
young, developing rapidly and with high scientific content. Furthermore, enough patents have to be
granted in a short space of time so that the citations in them are sufficiently numerous to bear statistical
analysis (Collins & Wyatt, 1988). According to these authors, this means that in practice the number of
patents granted within a space of about five years should produce several thousand citations to the
journal literature.
A. Academic citation versus patent citations
Referencing, as one of the widely accepted and utilised norms, confirms and illustrates the social
character of the knowledge creation and diffusion process. Citing occurs not only within the scientific
community but also within the technologic community. As we have seen, there is a profound ‘cross’
citation practice between both communities, mainly from technology to science. Comparison of the
motives between academic citations (cfr. section 9.IV.1.) and patent citations may provide relevant
insights in the differences in citation behaviour between the technology and the science system and also
the value of patent citations, still a subject of discussion.
At first sight, the role of citations in patent documents is intrinsically different from academic citations.
Primarily, this is due to the legal function and character of a patent document and the social process of
patenting, implying the involvement of several parties (examiner, applicant, patent attorneys) during the
patenting process. A patent is a property right based on a sealed claim. For the property right to be
recognised by other competitor companies, all property right details have to be public (Grupp, 1998).
The description of the so-called ‘prior art’ leads to a set of one or more claims. All relevant documents
(patents and other documents) that are known to the inventor, and that present those aspects in which
the product or process has to be considered innovative, have to be mentioned. The role of the examiner
versus that of the applicant differs per patenting system (more about this in the next section).
The patent examiner’s task is to ensure the originality of the claimed invention and to identify the limits
to that originality. His or her citations aim at defining the area in which the invention is truly original,
by assessing the state of the art, and therefore fulfil the criteria for granting a patent. The examiner
refers to the claims of the patent application. Thus, one of the major motives to cite a specific document
90
results directly from his/her responsibility and is related to the academic citation motive: ‘Disclaiming
work, ideas or claims of others’, negative claims and homage.
Two other citation motives within the scientific community also seem to play a role for the patent
examiner to cite scientific literature (and patent documents): ‘Providing background reading’, in view
of the information function of a patent (Grupp, 1998), and ‘Identifying original publications in which an
idea or concept was discussed’, again in view of the examiners obligation to search for novelty and
originality of the patent applied for. As a consequence, documents considered relevant to the examiner,
are cited. There seem to be no other similarities between the citations in both spheres (cfr. section
9.IV.1.).
B. Examiner versus inventor-given references
With respect to the source of NPRs in patent documents, one can distinguish between inventor and
examiner-given references. Nevertheless, both types of references stem from a different logic. Whereas
the task of the examiner is to ensure the novelty of the invention claimed and to identify its limits, the
inventor’s principle task is to identify work "either related to, but significantly different from, or else a
useful step towards, the new invention or a use of the invention" (Collins and Wyatt, 1988, p. 66).
However, the inventor’s task has to be regarded in light of the patenting procedure handling the
application (cfr. infra). Examiner citations, as a result, usually complement, rather than duplicate the
citations given by the inventor, although both examiner and inventor not seldom refer to the same
publications (Collins & Wyatt, 1988; Tijssen, 2001; Meyer, 2000a).
According to a survey by Schmoch (1993), about 8% of all examiner citations originate from the
inventor. In a study performed by Tijssen et al. (2000) this percentage even amounted 35% (see also
Narin et al., 1989). They also indicated that in another 35% of all cases, the inventors introduced most
of the NPRs. As for the citation characteristic, applicants are more likely to cite the journal literature
and examiners to cite reference books, conference proceedings and abstracting journals.
Collins & Wyatt (1988) as well as Meyer (2000a and 2000b) argue that because citations in patents
have a legal character, they are more carefully selected, and as a consequence less superfluous than
citations in scientific publications. However, they are not free of problems either. Collins & Wyatt
(1988) point to a number of these problems. The most important are:
• Examiners tend to restrict their reading to a narrow range of specialties and to be relatively
unfamiliar with the wider literature; they frequently cite in secondary form (abstracts), probably
because of the availability of abstracts databases making the search for prior art easier.
• The use of the same set of citations by one examiner in several different patents, suggests an
occasional tendency to cite by rate, rather than by relevance.
• Both examiners and applicants/inventors may be affected by national chauvinism in their citing
practice.
Furthermore, in view of the limited number of articles cited on the ‘front-page’ of patents, Collins &
Wyatt (1988) question the competence of the patent examiners. They point out that inventors have a
greater tendency to cite the scientific literature than the examiners. According to them, examiners do
not only cite a few articles, but in addition, they may even cite these ‘badly’.
C. Motives of the examiner to cite NPRs
Whereas in the previous section we looked into the general tasks of the examiner (the legal
responsibility) in comparison to the inventor’s tasks, in this section we will look more specifically into
91
the motives for an examiner to cite NPRs, and not patents for example. Based on interviews and
technical discussions with patent examiners, lawyers, and others in the field, Grupp & Schmoch (1992a)
infer six reasons for examiners to assign an NPR to the prior art description. Each reason influences the
degree of science involvement in the invention process. These reasons, together with an indication of
science involvement, are presented in the textbox below.
Reasons for assigning NPRs to a patent
Reflection Science
Involvement
I. Prior art in science and technology is still not covered by any patent document.
Medium/High
II. The patent examiner wants to cite non-patentable research results (e.g. formulae,
High
theories, discoveries, software development, and medical cures). He has to cite literature
documents because patent documents are not available.
III. The specialist area subject researched is developing so quickly that preceding patent
Medium
documents from foreign patent offices are still not accessible to examiners owing to the
publication periods (but only their own documents even prior to publication). If the
inventor has published scientific papers on similar subjects in non-patent literature, then
these documents can be cited.
IV. A company dispenses with a patent application and publishes the result of the
Low
development in the company’s own journal in order to safeguard the novelty claim visà-vis competitors even without a patent application. This document will be involved if it
is relevant.
V. The necessary references to Japanese language documents, incomprehensible to most
None
European and American examiners, may feature, via an English abstract service in the
search report in the form of an NPL reference, whereas in reality they are patent
citations.
VI. References to very simple relationships that fall below the patentability threshold (e.g.
None
how a screw is tightened or a mechanical clock wound) are essential. Schoolbooks and
encyclopaedia’s, therefore NPL sources, are mentioned.
Adapted from Grupp & Schmoch (1992a) – Extended by author
Some of these motives reflect an intensive science involvement, others not so unambiguously. Motives
5 and 6 are judged to be unimportant for science involvement. As a consequence, NPRs of such an
origin should not be involved in the construction of indicators. Citations related to the 4th motive
concern inventions that, through management decision, fall outside the commercial interests of the
producing company. These need to be excluded from indicator construction as well. The 3rd motive
pleads, though not necessarily, for the presence of a science interaction. The 2nd motive reflects a
science connection, with a quite high level of probability. According to Grupp & Schmoch (ibid), the 1st
motive reflects the classic situation in which - following Tijssen (2001) - examiners are probably more
inclined to copy applicant-given NPRs when the prior art in science and technology is not yet, or only
marginally, documented in the form of patents. This may particularly be the case in new technical fields
or in young research-intensive fields where rapid developments have not yet reached the stage of
patenting and are documented only in research papers and technical reports. Two elements are of
interest here. First, according to Grupp & Schmoch (1992a), prior art in the form of patents is still not
discovered. Second, according to Tijssen (2001), this leads to an increase in the number of applicantgiven citations included by the examiner.
92
D. Types of non-patent references
In addition to the distinction that can be made between inventor and examiner citations, one can also in European search reports only - distinguish between different types of patent citations and as such
obtain more insight into the relevance of a citation. As indicated by Schmoch (1993), the examiner has
to look for earlier documents - primarily patents - that have the same or almost the same features as the
patent application. Only if there are no other relevant documents questioning the novelty of the
invention, will the patent application be accepted.
In line with these responsibilities and tasks, we can find two types of patent citations (Schmoch, 1993):
(1) documents of particular relevance, and (2) references concerning the general background.
Documents of particular relevance restrict the claims of inventors. They are marked with the letter "X"
when they, if taken alone, question the novelty or inventiveness of a patent claim. If a document is
considered to question inventiveness of a particular claim, when taken in combination with another
document, it is marked with the letter "Y". Citations documenting the technical background of the
invention are marked with the letter "A" (Meyer, 2000a, 2000b).
The EPO is the only patenting office that marks the different categories of references explicitly.
Different degrees of linkage (or proximity) to the examined patents may be associated with the different
types of cited references. The “A”-type references are general information for the applicant and are
important, first of all, for the assessment of the so-called ‘inventive step’, which forms the second basic
criterion for the approval of a patent. References of type “A” have low linkage proximity, whereas “X”
references have a high degree of linkage. The linkage degree of “Y” references may vary since they are
important only in combination with other references. When considering the “X” and “Y” type of
references as of high-medium proximity, according to Schmoch (1993), then 29% of all citations can be
considered “closely linked to direct knowledge sources”. On the other hand, this implies that almost
70% of all citations have no close linkage to the examined technological invention. Although only 29 %
of all citations are closely related to the patents in question, all references can be considered relevant for
the analysis of the S&T interrelation analysis, or “…patent citation analysis can be helpful for tracing
the knowledge transfer without a distinction between the different types of citation” (ibid, p. 195).
IV.1.2. Differences in citation intensity
As Meyer (2000a, 2000b) pointed out, the patenting process is mainly a socially embedded process,
during which many parties and considerations, and even interests (cfr. supra) play a part. The entrance
of NPRs during this process is also influenced by this factor. Of particular interest is to look into the
causes of varying numbers of NPRs between technology areas, countries etc. In other words: are there
any plausible explanations for the fact that NPR-intensities vary? This is an important question, as its
answer directly influences the interpretational possibilities, and the validity of this type of analyses. In
view of the discussion carried out above, we can say that the occurrence and frequency of NPRs is a
function of a) patenting procedures, b) the technology field under analysis, and c) the country of
analysis. A fourth rather subjective factor can also be added, namely the decision making process of the
patent examiner. We will discuss these factors more in detail below.
A. Influence of patenting procedures
Procedural differences between national patenting systems, and specifically between the European
patent office and the United States patent office, also influence the availability of NPRs (Meyer,
93
2000b). It is in this context that Schmoch et al. (1993) identified another reason why EPO and USPTO
comparisons are important: reliability. If the differences in the NPR structure between both patenting
systems are major, then the reliability of the adopted approach needs to be called into question or a
proper explanation needs to be found.
Let us review a number of differences between the EPO and the USPTO (cfr. section 7.I.2). According
to Schmoch et al. (1993), in science-intensive fields like biotechnology, the USPTO is very slow in
granting because of an insufficient number of qualified staff in these areas. This implies that the number
of EC-assigned patents at the EPO with equivalents at the USPTO is in reality much higher.
Another interesting aspect is the difference in the number of citations between the EPO and the USPTO
patent data. Again, we can find the answer by taking a closer look at the two examination procedures
(Meyer, 2000a, 2000b). At the EPO, only the main claim is being examined. Most important is that the
main claim meets the three criteria: novelty, non-obviousness and industrial applicability. In contrast, at
the USPTO, all claims including dependent claims are investigated in detail. The most important reason
for differences in the number of citations is the so-called “duty of disclosure” in the US. Meyer (ibid)
explains: “In Europe, it is up to the applicant to introduce prior art known to him in the examination
procedure or to refrain from doing so, whereas, US law stipulates that the applicant has to cite any prior
documents known to him to the USPTO as long as the application is under examination” (p. 424). Not
complying with this requirement is considered as fraud by the USPTO and can be used as a ground for
invalidating the patent (Rainer Bertram, personal communication; in: Meyer, 2000b, p. 424). Table 6
provides an overview of the differences between the European and the US examination practices.
Table 6 – Differences between the European and the US examination practices
Issue
EPO
USPTO
“Someone skilled in the art”,
Specialist is well educated, so
Specialist is less educated, so he
the “average specialist”.
for him, a less detailed
needs a very detailed description
description is sufficient.
and many references to other
documents.
Searches, search reports
It is said that, in many cases, the
The quality of US researchers is
EPO report is better, due to a
broader access to relevant
limited by their focus in
English-language documents.
material.
US examiners are under time
pressure.
Education requirements of
Generally higher than those of
Generally lower than those of
patent examiners
their US counterparts.
their European counterparts.
Claims
Focus on umbrella of claims.
Many claims.
References
No duty of disclosure; Focus on
Duty of disclosure: all relevant
relevant citations.
documents have to be indicated
by the applicant party.
Source: Meyer (2000b)
According to Meyer (2000b), the differences between the EPO and the USPTO patenting system at the
micro-level, might affect the macro interpretation of patent citations. As USPTO patent practice
includes the examination of both major and other claims, a greater variety of linkage and a greater
frequency should be expected. USPTO data seem to be more appropriate if the analysis aims at the
widest possible cognitive web of interconnecting science and technology areas.
94
B. Field-specific influences
Research has shown that differences in citation behaviour occur between technology domains. MeyerKrahmer and Schmoch (1997) observed, for example, that pharmaceutical patents cite scientific
publications to a much bigger extent than mechanical and automobile patents. Meyer (2000a) indicates
that electronics-related patents have a higher level of NPRs than materials-related patents.
Narin and Olivastro (1992) also found significant variations in the number of NPRs present in patents
belonging to different technology fields. High science-citing fields included "drugs and medicine" with an average of 3,19 NPRs per patent - and, to a lesser extent, "chemistry and chemical products"
(0,77 NPRs per patent) and "professional and scientific instruments" (0,54 NPRs per patent). Low
science-citing domains included "machinery" (0,12 NPRs per patent) and "transportation" (0,01 NPRs
per patent) (Narin and Olivastro, 1992). According to Collins and Wyatt (1988), patents will cite
substantial numbers of scientific papers if they are in a field that is relatively young, developing rapidly
and with a strong scientific component. To quote Meyer (2000a, p. 426): “Different fields have a
different nature of interaction”. In addition, one should also keep in mind that not all ‘technology’ is
patented. A well-known example is software, which for reasons of market dynamics and actual
exclusion from patenting, has consequently been a field with little patenting.
As such, we can conclude that different fields have a different nature of interaction between science and
technology. In some fields the intensity of the interrelation is stronger - indicated by a higher number of
NPRs per patent - when compared to other fields. According to Meyer (ibid), this would also argue for
sector-specific technology transfer policies.
In addition to the fact that the number of NPRs appears to vary significantly according to the
technology domain the patent belongs to, Narin et al. (1997) found that the degree of science linkage
also is very subject specific. Their study shows that drugs and medicine patents invented in all countries
cite almost exclusively to papers in the scientific fields of clinical medicine and biomedical research.
Similarly, chemical patents appear to cite chemistry papers heavily, while electronics patents heavily
cite to physics and engineering papers.
C. Country specific influences
In literature it has been frequently discussed whether the differences in citation patterns between
countries, and specifically between the US and other countries, are due to actual country differences
such as language. On the one hand, it has been pointed out that there are intrinsically national
differences in the science bases of countries, but on the other hand, an investigation of the science
intensity of the technological landscape of Germany, performed by Grupp and Schmoch (1992b), also
discussed in Meyer-Krahmer (2000), showed that the link with science is an internal characteristic of
the technological areas under review, and that there are hardly any differences between countries. It is
also shown that a country must have a certain scientific level if it wants to ‘participate’ in a sciencelinked area.
In addition, Narin et al. (1997) found a strong national component in the linkage between science and
technology. They found that each country's inventors in the USPTO-system cite their own country's
papers two to four times as often as expected, when adjusted for the size of the country's scientific
publication rate. This strong domestic component that exists in the science-technology linkage shows
that each country's inventors are preferentially building upon their own domestic science.
95
IV.1.3. Interpretation of the science – technology linkage based on citations
In several studies – e. g. Grupp and Schmoch (1992a) – it has been indicated that NPRs can be seen as
an indication of the “science relatedness” of a technology field. Meyer (2000a, 2000b) investigated the
nature of the citation link established by an NPR, by looking into a sample of ten patents in detail. His
findings clearly confirm the idea that there is no direct relationship (causality) between cited paper and
citing patent. It appeared that, in the cases he analysed, the cited literature source rarely seemed to be
the original source of the idea leading to the invention. Only in one of the ten cases, one can draw an
“antecedent” direct cognitive link between a patent and a particular scientific article. Instead, the
relationship can be more characterised as highly mediated.
In a study performed by Tijssen & Buter (1998), slightly different outcomes were shown. In a survey
involving 50 Dutch inventors, with USPTO granted patents, it appeared that in 94% of the patents, and
thus of the inventors questioned, ‘in-house’ scientific research played a “very important” or “reasonably
important” role. The conclusion that can be drawn is that a citation link (a patent citing a scientific
paper) at least ‘connects’ a patent to a paper. As Meyer (2000a, p. 421) puts it: “It tells whether the
invention touches an area where there has been no patent before, but a scientist has published something
relevant”.
What does this mean for the use of citation links for establishing the S&T interaction? NPRs provide an
important tool for analysing the S&T interaction and for identifying and revealing cognitive webs of
interactions between both spheres. The evidence is there. The studies performed by the CHI (Computer
Horizons Incorporated) concluded that the technology areas whose patents cited to scientific papers
were, in fact, rated by their peers as far more science dependent that areas of technology which did not
relate to science (Narin & Olivastro, 1998). In 1993, Schmoch et al. (1993) confirmed these findings in
a study performed for the European Commission focussing on the evaluation of EC programmes in the
areas of agro related biotechnology and electronics & information technology. They identified similar
science intensive technology fields as CHI. One of the conclusions in the above-mentioned study is that
the operationalization of science intensity, through references to scientific publications, can be a starting
point for many interesting investigations, but that, also in view of what has been presented in this
report, a careful methodology is necessary to obtain valid results.
According to Meyer (2000a), several writers in the literature on innovation and technical change have
emphasised that information contained only in scientific papers will not suffice to implement the
technology in question (see for e.g. Pavitt, 1987; Rosenberg, 1990). This suggests more mechanisms in
the linkage between science and technology, and a not so straightforward interplay. Rabeharisoa (1992)
states that there is no global, unequivocal answer to the question if articles, cited on the front-page of
patents, trace links between different scientific and technical entities (cfr. section 9.IV.1.3.). However,
the individual case rules: some examiners will cite scientific articles consistently and others will not
(cfr. 9.IV.1.1.). The author continues to argue that the reliability and also the relevance of ‘front-page’
cited articles is a ‘local’ question and furthermore depends on the domain chosen and on the database
used. In view of the previous discussion, relying upon journal citations becomes a delicate affair.
Naturally, this has implications for the way in which S&T linkage results should be interpreted. As this
discussion is related to the direction of the knowledge flows between science and technology, we also
refer to chapter 7 for more details.
96
IV.1.4. Relevance of direct linkage approach from a policy perspective
According to Meyer (2000a), the method of patent citation analysis is still valid as a policy tool. The
method is still useful for policy makers in order to illustrate the science-relation of technological fields.
More strongly expressed, even if the use of citation analysis for the illustration of the S&T relationship
only reveals those science fields ‘touched’ by specific fields of technology, this can only be regarded as
being of enormous value for the support of policy decisions in the S&T area, especially when
combining the S&T link with other analytical tools.
There are three major applications of patent citation analysis that we would like to discuss more in
detail (Meyer, 2000c): following the general science orientation of fields over time (revealing a web of
science and technology linkages); measuring the intensity of science and technology interaction; and
tracking potential knowledge flows between scientific and technological fields. The first application
allows for observing potential governance shifts in certain S&T areas. Areas in which instrumentation
developed in early stages preceded and enabled scientific exploration, a development that gave the
opportunity for further technological development (a so-called ‘Rosenberg pattern’). An example of
such a field is nanoscience and –technology (Meyer & Persson, 1998; Braun et al., 1991). Identification
of shifts from ‘applied’ to more ‘basic’ oriented technologies, or in the opposite direction, may have
important policy implications concerning support and organisation structures.
The second application, measuring the science – technology interaction, illustrated in the present study,
measures the relevant interaction between science and technology fields. As already mentioned, patent
citations can be used as a unit of analysis for establishing the link between both spheres. Science –
technology interactions may reveal key science domains for certain technologies and provide an
impression of how intensively scientific domains are related to technologic domains, and how this
interaction co-develops. By broadening the approach, it also becomes possible to identify specific areas
in which science and technology interactions could be promising but are not present; potential areas for
future activity can be identified enabling policy makers to pinpoint them and intervene.
The third application of patent citation analysis concerns the possibility of tracking and examining
potential knowledge flows between specific (sub-) fields of science and technology. Analysis of
patterns in which (sub-) fields interact, will increase the understanding of the science and technology
interaction. The level of examination of the knowledge flows can vary from high (level of (sub-) fields
of science and technology), medium (countries, regions or organisational entities such as universities,
companies etc.) to low (identification of potential key actors). Patent citation analysis can give
important clues as to the location of the knowledge flows and the occurrence or absence of knowledge
flows in certain (sub-) fields, countries, and regions. All three applications will be utilised in our study,
albeit in view of the necessary global focus.
IV.1.5 Science – Technology linkage indicators
In this section we shall discuss the different indicators – based on NPRs - and thec subsequent analyses
that can be performed. Depending on the aspect of the linkage they focus on, we can distinguish
between two major types of science-technology linkage indicators: those describing (1) the magnitude
or intensity of the linkage; and those describing (2) the nature of the linkage. After having described
both types, we shall elaborate on a number of drawbacks, or better put, issues on S&T linkage approach
and linkage statistics.
97
A. Indicators describing the magnitude or the intensity of the S&T interaction
This first type of indicators is based on simple counts of the number of non-patent references that occur
in a given set of patents. Under the basic assumption that the degree of science intensity of a particular
technology domain is reflected in the number of references to scientific publications given in patents,
one can interpret such counts as a measure for the magnitude or intensity of the linkage between science
and technology. In a next step, the average number of NPRs per patent (and specifically the citation to
scientific papers) can be calculated in order to compare the magnitude of the science-technology
linkage between various countries or regions.
Narin et al. (1997), in their analysis of USPTO-patents over the period 1985-1995, observe a steady
increase in science linkage across all countries and technologies. This is illustrated in Table 7 on a
country-by-technology basis and in Figure 9 on a country-by-country basis.
Table 7 − Evolution in the average number of science references in USPTO patents
Technology area
1985
1990
1995
Chemical Patents
0.74
1.30
3.18
Drug and medicine patents
2.17
3.78
8.66
Electrical component patents
0.45
0.73
1.00
Scientific instruments patents
0.41
0.58
1.27
Source: adapted from Narin et al. (1997)
Figure 9 − Increase in science-technology linkage as reflected in the number of NPRs in USPTO patents for five
countries
S cience References per US P T O P atent (sm oothed)
1,6
US
1,4
1,2
UK
1,0
0,8
France
Germ any
0,6
Japan
0,4
0,2
0,0
1985
1986
1987
1988
1989
1990
1991
1992
1993
1994
1995
T ime
Source: Narin et al, 1997
Tijssen & Buter (1998), in a study on the importance of Dutch scientific research for technological
innovations, concluded that Dutch science is increasingly used in technological innovations, based on
the average number of NPRs per patent. The average number of citations increased with 42% (2.9 per
patent). Practice has shown that the distribution of the number of NPRs per patent is very skewed. A
98
large number of patents contain no examiner-given NPRs, whereas only a few patents contain a lot of
NPRs. As a result, a relatively high value for the average number of NPRs per patent could be caused
by only a few patents containing a (very) high number of NPRs. In order to assess this problem, Van
Vianen et al. (1990) have introduced two indices that provide us with information on the low-frequency
end − patents containing not a single NPR − and the high-frequency end of the distribution − patents
containing many NPRs.
The first index, related to the low-frequency end of the distribution, is the proportion of patents
containing zero references to scientific literature (P(0)). The second index, corresponding to the highfrequency end of the distribution, is the proportion of patents containing five or more references to
scientific literature (P(5)).
Grupp & Schmoch (1992b) introduced a more sophisticated indicator for measuring the magnitude or
intensity of the linkage between science and technology. Their "mean NPL index" or "NPLM-index" is
also based on simple counts of NPRs. For a particular technology domain j and country i the mean
NPL index is calculated as follows (Grupp & Schmoch, 1992a; Meyer-Krahmer, 2000):
NPLM ij (t ) =
with
Pij (t ) =
NPL (Pij , t ) =
NPL (Pij , t )
Pij (t )
the number of patents in year t for country i in
technology domain j
the number of NPRs in year t in patents from
country i in technology domain j
This NPLM index gives the mean value of the number of references to scientific literature in the patents
from a particular country in a particular technology domain. One of the disadvantages of the mean
index, according to Van Vianen et al. (1990), is that the mean is very sensitive, in a bad sense, for very
‘skewed distributions’ (i.e. in case that a lot of patents contain no NPRs and that only a few patents
contain a high number of NPRs). For the visualisation of the results, this NPLM index can be
transferred into a relative, standardised form (Schmoch, 1997):
æ NPLM ij å Pij ö
÷
RNPLij = 100 × tanh ln ç
ç å NPL (Pij ) ÷
è
ø
The RNPL index gives the deviation of individual countries and technology domains from the overall
average. By taking the logarithm and the tangens hyperbolicus, the RNPL index becomes symmetric
within the boundaries of +100 and −100, with 0 as neutral point. Positive values for the RNPL index
correspond with science-technology linkage intensity above average, whereas a negative value reflects
linkage intensity below average (Schmoch, 1997; Meyer-Krahmer, 2000).
Schmoch (1997) used the RNPL index to investigate the science-technology linkage intensity for 30
different technology domains. Calculations were based on the total number of EPO-patents for the
period 1989-1992. Figure 10 illustrates the outcome of this study.
Schmoch (1997) encountered the highest science-technology linkage for the domain of biotechnology,
the linkage intensity for domains such as chemistry, microelectronics, and information technology was
also above average. The areas characterised by a relatively weak science-technology linkage − as
reflected by the RNPL of a particular domain − are generally in the area of mechanical and civil
engineering.
99
Figure 10 − Intensity of the science-technology linkage for 30 technology areas
(EPO 1989-1992)
100
90
80
Sem iconductors (66)
70
Pharm aceutics (68)
60
O rganic Chem istry (61)
Food (55)
50
Optics (39)
40
Telecom m unications (32)
30
Basic m aterials chem istry (21)
20
Surfaces (17)
10
Polym ers (5)
0
Environm ental technology (-17)
M aterials processing (-28)
Biotechnology (85)
-10
-20
-30
-40
-50
Data Processing (42)
M aterials (37)
Audiovisual technology (27)
Control technology (20)
Nuclear technology (8)
Electrical energy (-12)
Chemical engineering (-2 6)
M achine tools (-46)
Engines (-57)
Food processing (-61)
Handling, printing (-66 )
-60
-70
Therm al processes (-64)
M edical technology (-70)
Transport (-80)
-80
Consum er goods (-88)
-90
Space technology (-78)
M echanical elem ents (-85)
-100
Civil engineering (-9 3)
Source: Schmoch, 1997
B. Indicators describing the nature of the S&T interaction
The second type of indicators - those describing the nature of the linkage - are not based on the simple
counts of non-patent references that occur in a given set of patents, but on the characteristics of these
non-patent references. Patents are then classified into groups on the basis of the nature and
characteristics of the NPRs included in the patent.
More insight into the nature of the S&T linkage can be obtained by studying several characteristics of
the NPRs and the patents involved. This can, for instance, be achieved by investigating which
technology domains refer to which science domains, resulting in a sort of "citation matrix". This higher
level of analysis enables to model interaction between areas of technology and science. Other inferences
on the nature of the science-technology linkage can be made by studying the age distribution of NPRs,
the type of journals referred to, the author affiliation of the papers referred to, and so on. The most
important aspects that can be investigated are described below.
B.1. Modelling interrelations between science and technology areas
First of all, the nature of the linkage is often investigated in terms of the science fields that are cited by
the NPRs in the patents. As such, we can see which technology fields are linked with which scientific
fields, thus arriving at a so-called "linkage scheme" or "citation matrix". This can be done for all
patents in general, or for the patents belonging to specific technology domains in particular. Patents are
assigned to a particular technology domain on the basis of their IPC classification codes. Scientific
papers are assigned to a scientific discipline based on the journal they were published in. Journals are
100
assigned to one or more scientific disciplines based on the journal’s general contents. According to this
procedure, cognitive links between citing patents and cited papers can be investigated.
Tijssen (2001) for example examined the nature of the linkage between citing patents and Dutch cited
papers at a rather aggregated level. Papers were classified into major scientific disciplines according to
the ISI-defined journal sets ("subject categories") and the definitions used in the Science and
Technology Indicators Report issued by the Dutch Observatory of Science and Technology. Citing
patents were classified into one or more product fields, in accordance with the associated US-based SIC
codes best corresponding to the origin of technical knowledge. Patents were further classified by
aggregating those SIC groups into five generic fields, corresponding to broad industrial sectors. The
distribution of citations to Dutch research papers on USPTO patents, by citing sector and cited scientific
discipline (in percentage of total per sector) for patents granted in 1995-1996, as found by Tijssen
(2001), is presented in table 8. Contributions with a percentage share of less than 5% in a sector were
not shown.
The results, according to Tijssen (2001), display an intricate web of interrelationships where each sector
is characterized by its own distinctive multidisciplinary "science linkage" profile. This also illustrates
that technological advances in the industrial sectors in question depend on a wide range of scientific
disciplines. As such, we can see that the Pharmaceuticals and Medicines sector are primarily linked to
scientific fields like Biomedical Research, Clinical Medicine and Biochemistry and Biology. The
Instruments and Control sector, on the other hand, heavily relates to scientific fields such as Electrical
Engineering, Physics, but also Clinical Medicine. Notice, however, that this "citation matrix" is based
on citations to Dutch research papers only.
101
Table 8 − Distribution of citations to Dutch research papers on USPTO patents by citing technology domain and
cited scientific discipline (USPTO 1995-1996)
TECHNOLOGY DOMAINS
Pharmaceuticals
and medicines
Chemicals
and materials
Biomedical
Research
29%
17%
Clinical Medicine
19%
17%
Biochemistry and
biology
28%
Chemistry
SCIENTIFIC DISCIPLINES
Pharmacology
7%
Instruments
and control
Electrical and
electronics
Machinery
and transport
19%
7%
26%
6%
19%
20%
14%
17%
5%
23%
22%
9%
Biotechnology
20%
Chemical
Engineering
6%
Physics
18%
7%
9%
Biomedical
Technology
9%
5%
6%
Electrical
Engineering
24%
5%
Mathematics and
computer science
7%
Source: adapted from Tijssen (2001)
By incorporating the citing countries into the analysis, the linkage scheme can be analysed on a more
detailed level. Narin and Olivastro (1992), for example, investigated the linkage between science and
technology domains by determining the scientific fields that are cited in patents for a number of major
product field categories. Figure 11, for example, shows the average number of NPRs to SCI journal
papers, per cited science field and citing country for the technology domain of computing and
communications. The data relate to USPTO patents issued between 1987 and 1988.
102
Figure 11 − Cited scientific fields and citing countries for the technology domain of computing and
communications
Citations per patent
0, 3
0,2
0, 1
Other
En & Tech
Phy sics
Chem
0
France
UK
C ited Field
C M & BR
Germ any
Japan
US
C oun try
Legend: CM & BR = Clinical Medicine and Biomedical Research; Chem = Chemistry; En & Tech = Engineering and Technology
Source: adapted from Narin and Olivastro, 1992
B.2.Investigating other aspects of the linkage between science and technology
As we already mentioned, other aspects of the science-technology linkage can be investigated as well.
This kind of analysis is always based on the classification of NPRs occurring in the patents under study,
according to their intrinsic nature.
Van Vianen et al. (1990), McMillan et al. (2000), and Schmoch et al. (1993) for example, investigated
the nature of the linkage between science and technology by examining (a) the age distribution of
NPRs, (b) the type of journals (basic versus applied) patents refer to, (c) the institutional breakdown of
cited research, and (d) the country of origin for the cited papers. Several other analyses could be
performed in a similar way. We will briefly describe and illustrate each of the aforementioned
analyses.
Age distribution of NPRs
The nature of the linkage between science and technology can be described by examining the age
distribution of references to scientific literature. In order to determine this age distribution, Vianen et al.
(1990) calculated the age of all references, whereby "age" was defined as the difference between the
year in which the patent was granted and the year of publication of the cited paper. Notice that the "age"
of a NPR is not completely comparable to a reference in a scientific publication, since the time lag
between applying for and granting a patent is on average between 18 and 24 months (in certain areas
even more). Furthermore, when attempting to capture possible knowledge transfers like in the case of
q
103
patent citation examination and ‘age’ of the citation, the time lag between submitting an article to a
journal and its publication after a referee procedure has also to be taken into account (Schmoch et al.,
1993). Van Vianen et al. (1990) examined the age distribution of non-patent references in the domain of
chemical technology. The results are displayed in figure 12. The data relate to USPTO patents issued in
1985.
Figure 12 − Age-distribution of non-patent references in the domain of chemical technology
10%
9%
8%
% of total
7%
6%
5%
4%
3%
2%
1%
0%
0
5
10
15
20
25
30
age
Source: Van Vianen et al, 1990
Van Vianen et al. (1990) found that the age distribution was very negatively skewed. Most of the cited
papers were relatively recent ones, with a median age of four years. 50% of the non-patent references
appeared to be less than 10 years old, while less than 10% was older than 30 years.
They argue that the age distribution of NPRs gives an indication of the topicality of the technology
domain represented by the patents. They argue that a so-called "high-tech" field is based on very recent
science, whereas a more "established" technology domain would refer to somewhat older scientific
publications (see also Narin & Noma, 1985). By comparing the age distributions of NPRs for different
technology domains or different countries, one should get an insight in the "recentness" of the science
part that is linked to a particular technology domain or a particular country's technological system.
From a science and technology policy perspective, the authors argue, the question whether or not a
country is involved in rather "high-tech" areas might be of interest, since these kinds of areas are
generally supposed to present opportunities for economic growth.
104
The type of cited journals: the continuum “basic to applied”
Another way of examining the nature of the linkage between science and technology is by investigating
whether science that is cited in a set of patents tends to be more basic or more applied. In this respect,
the National Science Foundation and Computer Horizons Inc. (CHI) jointly developed a classification
scheme for the journals covered by the Science Citation Index (SCI). This classification scale ranges
from 1 to 4, with a score of "1" indicating that a journal is "very applied" and a score of "4"
corresponding with a "very basic" research journal.
McMillan et al. (2000) analysed the nature of the linkage between science and technology in the domain
of biotechnology. They also investigated the degree to which biotechnology firms rely on basic or
applied research. They first selected a sample of 119 biotechnology companies. Up to 1997, these 119
companies acquired 2334 patents in total, which contained 23.286 NPRs on their front pages. These
23.286 NPRs were eventually traced back to 6884 different SCI-covered publications. According to the
aforementioned classification, publications were classified into four categories on the basis of the
journal they were published in. The result is displayed in table 9.
q
Table 9 – Research orientation of Biotechnology firms
Level of journal
Number of papers
Percentage
Unknown
1 (=very applied)
2
3
4 (= very basic)
12
150
541
1749
4432
0.3%
2.1%
7.9%
25.4%
64.3%
Total
6884
100.0%
Source: McMillan et al. (2000)
The findings of McMillan et al.'s (2000) study clearly show the degree to which biotechnology firms
rely on very basic research. This in turn indicates the need for biotechnology firms to develop and
maintain a continued internal basic research ability in order to evaluate these external sources − see in
this respect the notion of "absorptive capacity".
q
Institutional breakdown of the cited research: public versus private science
McMillan et al. (2000) also investigated whether the domain of biotechnology is predominantly linked
with public or private science. They found that 71.6% of NPRs originating from patents granted to the
119 biotechnology companies in their sample, were to papers written solely at public science
institutions, such as universities, medical schools and research institutes. 11.9% of the NPRs were to
papers being joint efforts of public and private institutions, while only 16.5% of the cited papers
originated entirely from private companies. The results are displayed in table 10.
Table 10 – Type of institutional origin of cited NPRs of biotechnology patens
Author institution type
Number of citations
Percentage
Public
Private
Public and private
8568
1971
1422
71.6%
16.5%
11.9%
Total
11961
100.0%
Source: McMillan et al. (2000)
105
These findings indicate that the biotechnology industry is primarily linked with science originating from
public science institutions. Likewise, Tijssen's (2001) results on the study of Dutch papers cited in
USPTO patents also illustrate the overwhelming importance of public science in the cited research
literature. This is particularly the case in the pharmaceuticals and medicines sector, where he found that
more than 70% of the papers originate from the public science sector. Finally, Narin et al. (1997) found
similar results for the United States. Their results show that 70% to 80% of the NPRs were to
publications (co-)authored by researchers working at universities or public sector research institutes.
Institutional breakdown of the cited research: identification of knowledge producers
Another frequently performed analysis is the one where papers, referred to by NPRs, are classified
according to the institutional entity − rather than the type of institute − the author(s) of the cited papers
belong to. In McMillan et al.'s (2000) study, the importance of public science to the domain of
biotechnology was further demonstrated by the involvement of prestigious universities and laboratories
in the cited papers. They identified the top 10 US author institutions for the sample of science
references originating from the patents of 119 US biotechnology companies they investigated. Their
results are displayed in table 11.
q
Table 11 – Institutional origin of NPRs in biotechnology patents
Institution names
Number of papers
National Institutes of Health (Total)
Harvard University
National Cancer Institute
UC-San Francisco
Stanford University
US Veterans Administration
University of Washington
Massachusetts Institute of Technology
National Institutes of Health (unspecified)
Massachusetts General Hospital
646
510
296
208
181
177
163
163
137
127
Source: McMillan et al. (2000)
Country of origin
An analogous analysis can be performed for the country of origin of the author(s) of the cited papers.
On the basis of such an analysis, inferences can be made on the country specificity of the sciencetechnology linkage. It can, for example, be investigated whether there is a strong "national component"
in the science-technology linkage for a particular country. This would mean that a country's technology
is, to a higher than proportional extent, linked with research performed in the same country.
q
Narin et al. (1997) uncovered a very strong national component in the science-technology linkage. They
found that each country's inventors in the USPTO system cite their own country's papers two to four
times more often than expected, when adjusted for the size of a country's scientific publication rate. It
appeared, for example, that although only 7% of all papers in the SCI were authored by researchers at
German institutions, 17% of all NPRs on German-invented USPTO-patents referred to German
scientific publications. In addition, this appeared to be a general phenomenon for all of the five
countries − Japan, France, United Kingdom, Germany and United States − examined by Narin et al.
(1997). Similarly, McMillan et al. (2000) found a very strong national bias in the US biotechnology
industry. Likewise, Tijssen (2001) discovered a large domestic component in NPR linkages for Dutch
science-technology linkages.
106
These results suggest that there is a strong domestic component in the science-technology linkage,
showing that each country's inventors preferentially link with domestic science. An interesting
observation is that the same national preference is apparent, not only in citation behaviour from patents
to papers, but also from papers to papers and patents to patents. Narin et al. (1997) conclude from all
these citing phenomena that there are very strong national ties between scientists within a country and
inventors within a country, and between national inventors and scientists. This implies that a strong
national scientific base and a strong national technology base go hand in hand, at least in areas where
science and technology are tightly linked, for example in domains such as biotechnology, agricultural
chemicals and plastics.
B.3. Issues around S&T linkage approach and statistics
Classification ambiguity in regard of S&T linkage
A clearly classified observation structure is required for showing the connection between science and
technology (Grupp & Schmoch, 1992a). Since no general valid classification of technology is available,
it is an important advance that in the field of patent analysis there is a structured, uniform and
internationally agreed upon classification system, the so-called International Patent Classification (IPC).
Grilliches (1990) pointed out that the IPC is primarily based on technological and functional principles,
in function of the patenting system, and that it is only rarely related to economists’ notions of products
of well-defined industries.
q
In the case of economic research, quite some attempts have been made to link IPC-categories to
economic categories of interest through concordance schemes (cfr. section 7.II.2.2.). However, the IPC
classification does not always ally with a policy maker’s notion of a technology area. Therefore several
attempts have been made in the past to converge the IPC classification towards broader technology
areas.
In a study of Grupp & Schmoch (1992a) a classification scheme of technology has been constructed
based on IPC-codes (4th revision). The Fraunhofer (FhG-ISI) technology classification consists of 28
broad technological sectors, defined in 3-, and occasionally, in 4-digit IPC codes, being representative
and homogeneous in terms of modern technology. The classification scheme has recently been
updated/extended in co-operation with Observatoire des Sciences et des Technologies in France (OST)
and the INPI.
Also, for establishing the science content in terms of its intrinsic characteristics a type of classification
is necessary. De Bruin & Moed (1993) comment on this task as follows: “An appropriate delimitation
of scientific sub fields constitutes one of the key problems in bibliometrics”. According to the authors,
there are several methods for classifying science sub fields. The most important ones are co-citation
analysis, co-word analysis, the use of indexing systems produced by professional indexing or
abstracting services, and the use of a classification of scientific journals into subject categories –
nearing science sub fields. An example of a (meta-) classification used in several scientometric studies
(see for example Collins & Wyatt, 1988; Narin & Olivastro, 1992; Malo & Geuna, 1999), is the
classification produced by Computer Horizon Incorporated (CHI). This classification breaks several
thousands of journals covered by the SCI down into four different levels, depending on the research
type they contain, varying from ‘very’ basic to ‘very’ applied.
107
The journal classification of CHI has its difficulties and specifically in the field of physics and
engineering (Collins & Wyatt, 1988, p. 66). According to Moed (1996) the SCI-ISI classification seems
to be based to a large extent on information in the titles of the journals, while the above-mentioned CHI
classification system is primarily based upon analyses of journal-to-journal citations. Furthermore,
while the SCI classification is based on subject categories, which are and can be linked to science
domains, the FhG-ISI classification is based on the type of science performed. This illustrates that
classification systems can be intrinsically different, even when they are based on the same type of
source data.
Problems with linkage statistics
Before turning to the indirect linkage approaches, we shall briefly elaborate on three issues or problems,
inherent to the use of NPRs in the direct linkage approach. Some of these issues have been touched
upon previously from a different point of view. The following will be discussed:
1) Database shortcomings: the need for unification and standardisation
2) Skewed distribution of NPRs
3) Complexity of the data
q
Each of these items will be discussed in detail below.
1) Database shortcomings: the need for unification and standardisation
In chapters on patents and scientific publications (7 and 8), we have already dealt with specific
problems around databases. In the context of the direct linkage approach and the use of NPRs, however,
the heterogeneous format of NPRs and in many cases the incompleteness of the data (see also Schmoch
et al. 1993) also play a significant role. As Grupp & Schmoch (1992a) mention, the patent databases
usually have not been set up for statistical inquiries, but rather for patent lawyers, employees of patent
offices and interested industries, who wish to clarify and research individual cases. When conducting
statistical inquiries, double counting, incomplete data and spelling mistakes have to be taken into
account. The question of reliability and in particular of reproducibility calls for a series of
standardisations. In the table below we present a sample of NPRs originating from the USPTO and EPO
data. Note the heterogeneity among the references.
Table 12 – Examples NPRs
Examples of USPTO journal references
1.
2.
3.
Truelove, B. et al., Journal of Nematology, vol. 9, No. 4, Oct. 1977, pp. 326-330
Rodriguez-Kabana, R. et al., Nematropica, vol. 8, No. 1, (1978) pp. 26-31.’Ceramics In Surgery’ by R. P. Welsh et
al., Journal Biomedical Materials mposium, vol. 2 (part 1), pp. 231-249, 1972
‘The Experimental Use of Heterologous Umbilical Vein Grafts as Aortic Substitutes’, Singapore Medical Journal, vol.
3, No. 1, Mar. 1962
Examples of EPO journal references
1.
2.
3.
0203CHEMICAL ABSTRACTS, vol. 85, 1976, Columbus, Ohio, USA, BHATT S.B. et al. ‘A convenient method for
the preparation of alfa-arylaminophenylacetic acids’, page 607, Abstract no. 123514h & Curr.Sci. 1976, 45(15), 547
0202JOURNAL OF THE CHEMICAL SOCIETY 1975, pages 1865-1868
0203L'ONDE ELECTRIQUE, vol. 55, no. 1, Jan. 1975 Paris J. BESSON, ‘Plaques photovoltaiques au sulfure de
cadmium’, pages 21-24. * Page 21, column 1, paragraph 3 - page 21, column 2, paragraph 2; page 23, column 1,
paragraph 2 - page 24, column 1, paragraph 2; figure 2
108
Collecting patent citations to scientific literature is a difficult task (Van Vianen et al., 1990). An
important part of the data handling is ‘unification’ of the references: correction of journal abbreviations,
page numbers, volumes, authors into a standard format (see also Narin et al. 1997).
2) Skewed distribution of NPRs
Noma & Narin (1984) performed a study based on citation and referencing data from biotechnology
patents and bioscience papers. They showed that there is a very skewed distribution of cited material in
both patents and papers, with a relatively small number of highly cited patents and papers (see also Van
Vianen et al., 1990; Schmoch et al. 1993; Schmoch, 1993). In other words, a large number of patents
contain no references, whereas just a few patents contain many references to scientific literature. An
important question when dealing with (very) skewed distributions is whether the value of the mean
(average number of NPRs per patent) is a representative value.
3) Complexity of the data: multi-authorship
The last problem deals with the data complexity. Narin et al. (1997) point out that the data used in
linkage statistics are extremely complex. First of all, there are many different ways in which counts can
be made (normal, adjusted, etc. – cfr. section 8.IV.3.3.). A scientific publication may be cited in several
patents, and, on the other hand, a given patent may cite a number of different scientific publications. A
paper often has authors from more than one institution, just like patents may have different inventors
from different countries or institutions. The combination of the abovementioned factors gives rise to
various complexities in counting and data-representation.
109
II. Indirect/implicit S&T interrelations
The relation between science and technology cannot always be studied in a direct and
straightforward manner. Scientific knowledge, used for technological application, is not always cited
in patents. Noyons et al. (1994) found that the presence of fewer or no NPRs in patent is not
necessarily an indicator of a lesser science intensity of the individual patents, but more an indicator
of the technological nature of individual patents. As such, they conclude: “patents with no or only a
few NPRs cannot be regarded as significantly less ‘science intensive’ than those with many NPRs”.
In that regard, Meyer-Krahmer and Schmoch (1998) point out that a weak (direct) science linkage of
a technology does not imply a low university – industry interaction.
IV.2. Patents of scientific institutions and publications of industrial enterprises
It is generally assumed that the main output of universities and research institutes consists of scientific
publications, whereas the main output of industry consists of products and processes − represented by
patents. However, academics can also take out patents and researchers in industrial enterprises can also
publish scientific articles. This is illustrated in Figure 13.
Figure 13 − Output versus Actor
ACTOR
OUTPUT
University
Industry
Scientific
publications
MAINLY
Secondarily
Patents
Secondarily
MAINLY
Source: adapted from Godin (1996)
As a result of this general frame of mind, the number of studies dealing with articles published by
industrial enterprises, on the one hand, and patents taken out by scientific institutions, on the other, has
been limited in amount. In addition, the few studies that have examined this subject tend to be rather
limited in scope, focusing on one technology or one industry − often pharmaceuticals − at a time
(Godin, 1996).
A first indicator of industry–university collaboration is the share of patents of scientific institutions
(Meyer-Krahmer & Schmoch, 1998). A patent only makes sense for a scientific institution if it is
interested in the commercial exploitation of a new finding and collaboration with an industrial partner
(Meyer-Krahmer & Schmoch, 1998). According to Schmoch (1997), a high share of patents on the part
of scientific institutions can be considered a good indicator for a close relationship of scientific and
industrial laboratories in the technology field under analysis. He indicates that at the German Patent
Office, the average share of patents held by scientific institutes is about 5%. Narrowly defined
technology domains are often too small to perform any meaningful analysis on them. Nevertheless, this
approach is easily applicable in countries where universities and research institutes appear as applicants
and owners of patents, which is the case in the United States.
110
By simply generating applicant lists, one can proceed by identifying the patents of scientific
institutions. In Germany or Switzerland for example, the professors can freely dispose of the intellectual
property rights, and therefore appear themselves as applicant or sell their rights to firms. In some
databases, they can be identified by the academic title “Professor” (Schmoch, 1997). Hicks (1995) has
shown that companies publish scientific papers in order to signal scientific competence to academic
research. Therefore publications of companies are appropriate indicators for the relationship between
science and technology. A variant in the form of direct research co-operation between scientific and
industrial institutions is documented by co-publications (in fact: co-authorship). It is also interesting to
point out that, in the study of Meyer-Krahmer & Schmoch (1998), it appeared that academic researchers
rank collaborative research distinctively higher than contract research, which they perceive as rather
one-directional. The high ranking of informal contacts supports the relevance of knowledge exchange.
Let us now consider the relation for the other angle: scientific publications (activities) of industry. As
Godin (1996) mentions, for much of the last 35 years, basic science has been considered by economists
as a "public good". Firms were thought to have relatively little incentive to do basic research, since
research is expensive and risky and, once available, every firm can enjoy the results freely. As a
corollary, firms are also recognised to publish relatively few scientific articles. Publications by firms are
considered merely as a by-product of industrial activities and, if firms do publish, it is believed they
mostly publish applied results, not basic ones (Godin, 1996).
Recently, however, these assumptions have come under challenge. Since knowledge is complex involving "learning-by-doing" -, tacit - knowledge cannot be fully codified -, cumulative - it builds on
what has already been learned - and firm specific, merely relying on extramural research is not an
effective substitute for in-house research (Mowery, 1983; Godin, 1996). Rosenberg (1990) identified a
number of reasons why firms perform basic research in-house. One of the most important reasons is that
firms often need to do some amount of basic research, in order to understand better how and where to
conduct research of a more applied nature. In addition, doing basic research in-house is indispensable in
order to properly monitor and evaluate research being conducted elsewhere. Another important reason
for conducting basic research in-house is that it serves as some kind of a ticket of admission to an
information network. Indeed, the most effective way to stay connected with the scientific network is to
be a participant in the research process (Godin, 1996).
In accordance with these arguments, Nelson (1990) argued that firms have incentives to publish
scientific articles: to (1) attract customers, (2) establish legal rights, (3) attract capital, (4) inform
suppliers, and (5) gain reputation in order to relate to the scientific community and attract scientists and
engineers. Although the number of scientific publications by industrial scientists is still relatively small
compared to those from academics, the production of scientific articles by industry has increased by
50% in the last decades. In addition, some of these industrial publications have a very high impact on
the research community. This is especially the case in some high technology industries, such as
aerospace, chemicals, pharmaceuticals and electronics.
Research has shown that companies primarily publish scientific papers in order to signal scientific
competence to academic researchers (Hicks, 1995). Indeed, this appears to be the most effective way
for getting involved in the communication of scientific communities and gaining access to the most
recent research results. As a result, publications of industrial enterprises can be considered appropriate
indicators for the relationship between science and technology. Indicators in this respect might be the
relative importance of industrial publications in a particular scientific domain and its evolution over the
time (Schmoch, 1997).
111
An enormous disadvantage of the mentioned approach is the lack of possibilities for automatic
processing. Databases do not display special codes for universities, research institutions, and industrial
patenting. Consequently, manual processing, which is time consuming and thus expensive, is necessary.
In addition, an increasing number of the organisations that researchers work for no longer fit into the
traditional university/industry dichotomy anymore.
IV.3. Tendency to integrate scientific and technological activities
Meyer (2000a, 2000b) indicates that there is a increasing tendency for researchers to integrate scientific
and technological activities by working on one subject matter, but by generating scientific papers as
well as technological outputs − in the form of patents. He describes a number of science-technology
interactions he encountered when studying the front pages of a limited number of patents in the field of
Nanotechnology. They are presented in Table 13.
Table 13 - Types of science – technology interaction as reported in the cases studied
Types of science – technology interaction as reported in cases
Close personal science – technology linkage: individuals active both in academic research
and industry.
Doctoral candidate working in university-based corporate research institute. His work on the
same subject matter has led to both scientific results and technological output.
Public research institute is actively involved in both patenting and publishing. Not infrequently,
patents and papers result from the same project.
Invention patented was developed by a scientist who was active in a company and produced
patents as well as scientific research papers, based on the same project.
Data source: Meyer (2000a)
As a result, investigating the publication as well as patenting activity of an actor that is involved in both
kind of activities obviously is another way to explore the relationship between science and technology.
In this respect, Schmoch (1997) suggests searches in patent databases with the target of identifying
relevant inventors. In a second step, one could search publication databases for scientific articles
published by these inventors. Noyons et al. (1994), for example, found that most inventors in the field
of medical lasers are also actively engaged in publishing. They found that, at least for their sample, the
number of publications related to the area of the invention significantly increased before the patent
application.
Rabeharisoa (1992) encountered similar results when analysing the field of fuel cells. She found that in
times when the development of fuel cell technology was successful, inventors published articles
describing their technical advances and the market prospects of the technology in question. In addition,
stages of technical stagnation, reflected by a decreasing number of patents, were accompanied by a shift
of the inventor-authors towards more fundamental research in electrochemistry (Schmoch, 1997).
Obviously, the major drawback of this approach is the limited number of patents and publications
originating from the same actors.
IV.4. Co-activities (joint activities) between scientific institutions and industrial
enterprises
Direct co-operation between scientific and industrial institutions obviously is another indicator for a
strong science-technology interface in a particular domain. Co-operation between institutions can take
112
the form of either scientific collaboration, which usually results in co-authorship, or technological
collaboration, resulting in co-inventorship or co-ownership with respect to patents.
By investigating the number of articles by academic researchers that are co-published by industrial
colleagues on the one hand, and the number of patents by industrial enterprises that are either coinvented by or co-assigned to academic partners on the other hand, we should clearly get a better view
on the relationship that exists between science and technology.
Noyons et al. (1994) found in this respect that co-publications of universities and companies increased
in the period before the patent registration and decreased afterwards. As such, it can be argued that the
technology transfer between academic and industrial researchers is well reflected in publications linked
to patents (Schmoch, 1997).
Obviously, also here the major drawback pertains to the size of the sample involved. In addition, this
approach is also very labour-intensive, since databases do not display particular codes for universities,
research institutions and industrial enterprises. As a result, the records of the sample have to be edited
in a manual way (Schmoch, 1997). In addition, an increasing number of the organisations that
researchers work for no longer fit into the traditional university/industry dichotomy anymore.
IV.5. Parallel observation of patents and publications
The second approach, the parallel observation of patents and publications, compares the co-evolution
of patents and publications in time (Schmoch, 1997). According to Rappa & Debackere (1992), the
parallel development of publications and patents can be considered a strong indication for a close
interaction between academic and industrial researchers. “The comparison of time series for longer
observation periods supports the understanding of the interaction between industrial and academic
research and thereby the process of technology generation” (Schmoch, 1997, p. 110). An example of
parallel observation in the field of Neural Network is shown below.
Number
Figure 14 – Parallel observation of patents and publications in the domain of neural networks
5000
4500
4000
3500
3000
2500
2000
1500
1000
500
0
Publications
Patents
80
81
82
83
84
85
86
87
88
89
90 91
Year of publication/application
Source: Schmoch (1997)
The simultaneous take-off of both graphs around 1987 is due to the (re-) discovery of a learning
algorithm that had major implications for the performance of neural networks. In the last period,
starting around 1990, we see that scientific publications start evolving differently than the patents. This
113
might imply a breach in the close S&T interaction in the field of neural networking. A methodological
problem of this approach is the equivalent definition of a technology field in patent and publication
databases. The problems around classification systems, as previously discussed (cfr. supra), are a
limiting factor.
As a variation of the comparative approach, Schmoch (1997) proposes to start the search in a patent
database and to identify relevant inventors in the field. In a second step, publications of these inventors
are searched for in publication databases. Rabeharisoa (1992) and Noyons et al. (1994) use a similar
approach by examining the S&T relation through the identification of authors who are also inventors
(inventor - author relationships).
IV.6 Cartographical approach based on co-occurrences of publication and patent
keywords
The last indirect approach for analysing the S&T relationship that will be discussed in this report is
mapping. Mapping techniques and the utility of maps for science and technology policy have already
been discussed in the chapter on bibliometrics (chapter 7). In this section, we shall briefly discuss
mapping as a technique for illustrating and analysing the S&T interaction.
According to Noyons & Van Raan (1994) inventor - author relations occur in closely linked science and
technology fields, which is not always the case. By using a cartographic approach, the link between
science and technology is analysed on the basis of a cognitive overlap. This approach does not identify
S&T links that are actually present, but rather links that could be there, or even should be there. In the
study performed by Noyons & Van Raan (ibid.), a group of experts is used for the definition of the field
of Optomechatronics, the field under study. The same definition has been used to identify relevant
publications on the science side and also relevant patents on the technology side. From the set with
selected publications a list of most frequently occurring controlled (classification terms of the database
producer) and uncontrolled terms (terms given by the author reflecting the content) was generated.
From the patent set, the top-100 of indexed words was generated. Two maps were constructed, by using
co-occurrence frequencies, based on the same definition. It appeared that the cluster structure of the two
maps is more or less identical, implying a relationship between the scientific and technological
development.
Korevaar & Van Raan (1992) also explored the possibility to map the interface between science and
technology by combining bibliometric data on inventors and scientists. The findings represent the major
features of the field of Catalysis well, both for the science side and for the technology side, as was
validated by several internationally renowned scientists. It appears that mapping of the science and
technology interaction results in interesting insights, offering an alternative for those situations where a
direct linkage does not occur or where a direct linkage approach is not desirable. As no methodology is
completely unambiguous, the choice for a certain methodology has to be made in regard of the specific
aims and characteristics of each study.
114
-
Sectoral breakdown
and analysis, Number
start-ups in the field,
Institutional
specification
Normalisation of
absolute cooccurrence values by
using the ‘cosina’
formula, check on
suitability of data
Patent counts,
Publication counts, #
Organisations and
citations per institution
type
Patent and Publication
counts, Specialisation
Indices (RLA)
2.
3.
Controls
(normalisation)
Citation counts,
Publication counts,
Impact ratio’s
Variables
1.
Study
§
§
§
Type: indirect (cooccurrence)
Type: direct (through
NPRs)
Type: -
Linkage
115
§
§
§
§
§
§
§
§
§
§
§
§
§
§
§
Data source(s): INSPEC
T-field selection/class, keyword based
search, IPC
S-field selection/class, keyword based
selection
Data standardisation: General: two time periods, mapping of
internal dynamics
Data source(s): SCI
T-field selection/class.: S-field selection/class.: construction of a
classification based on journals (similar to
SCI class.)
Data standardisation: extensive (author
name, publication year, journal title,
volume, starting page number)
General: extensive standardisation process,
algorithm to identify citations to target
articles, full counting method, correlation
coefficients
Data source(s): EPO
T-field selection/class.: Keyword based
selection
S-field selection/class.: CHI ‘Level’
classification of journals
Data standardisation: General: number of organisations and
citations per institution type
Characteristics
Comparable
analysis of S&T
Explicit linkage
to Science
(NPR),
Identification of
major institutes,
policy
implications
Standardisation
approach,,
matching
approach,
construction of
science domains
+/+
-
-
-
-/-
APPENDIX I: KEY CHARACTERISTICS OF SEVERAL EMPIRICAL STUDIES ON S&T ANALYSIS
Correlation
# NPRs per patent
# NPRs per patent
Patent counts, NPR
counts, Publication
counts, Inventors being
Authors count
Patent counts, NPR
Counts, Citation counts,
Age distribution
Patent counts, NPR
Counts, References,
NPRs per patent
4.
5.
6.
§
§
§
Type: direct (through
NPRs)
Type: direct (through
NPRs)
116
Type: indirect but based
on NPRs (through
inventor – author
relationships)
§
§
§
§
§
§
§
§
§
§
§
§
§
§
§
Data source(s): USPTO
T-field selection/class.: Keyword based
selection
S-field selection/class.: CHI ‘Level’
classification of journals
Data standardisation: General: From – to citation analysis,
fractional counting, comparison of
examiner and applicant citations
Data source(s): USPTO, EPO
T-field selection/class.: S-field selection/class.:
Data standardisation: complete
standardisation of all relevant fields (more
or less complete reference)
General: matching to SCI, front page
references are used,
Data source(s): EPO, SCI
T-field selection/class.: randomly selected
patents – most closely linked publications,
expert assessment, inventor authored
publications
S-field selection/class.: CHI ‘Level’
classification of journals
Data standardisation: General: analysis of most closely related
publications, expert analysis of the patents,
introduction of the CICA indicator (coinventors vs. co-authors), division in
subsets of patents based on # NPRs
EPO – USPTO
comparison in
NPRs
Field
characteristics
for good linkage
results
Expert
validation,
showed that less
NPRs is not
necessarily an
indicator of a
lesser science
intensity of the
individual
patent
-
-
-
Two different
mapping methods
(Cosina – inclusion
index, single linkage
clustering –
waverage)
-
# NPRs per patent
Patent and publication
counts
Counts of ‘Other
references’, Patent
counts, Year
Patent counts, NPR
counts, Citation counts,
Publication counts
7.
8.
9.
Type: indirect (mapping;
co-occurrence)
Type: direct (through
NPRs)
117
Type: direct (through NPRs)
§
(S&T linkage has been created
based on inventor-author
relations and thus creating a
map)
§
§
§
§
§
§
§
§
§
§
§
§
§
§
§
§
Data source(s): Chemical Abstracts, WPI/L,
EPAT
T-field selection/class.: IPC definition of
field of Catalysis, Chemical Abstract class
and keyword selection
S-field selection/class.: Data standardisation: General: Bibliometric cartography, 2 time
periods of analysis, Co-occurrence
frequency (Cosina, MDS, inclusion index)
Data source(s): USPTO, TECH-LINE, SCI
T-field selection/class: Concordance, SIC
product groups
S-field selection/class.: CHI ‘Level’
classification of journals
Data standardisation: General: unification of each NPRs and
assignation to a specific category, adjusted
count’ (partial), assigned patents, division
between SCI and non-SCI reference
Data source(s): SCI based SLID database,
USPTO
T-field selection/class.: S-field selection/class.: Data standardisation: divided in different
categories, unification of format
General: matching to SCI-SLID, patents
issued, identification of author addresses
-
-
Expert
validation,
Comparison of
two mapping
techniques,
Internal
dynamics
insight enables
policy decisions
24%
unmatched
references
due to
imperfectio
ns
-
Interpretatio
n of the
maps
Distribution of
references to
scientific literature
(P0 and P5),
Validation of the
Activity Index (χsquare criterion)
-
# NPRs per patent
Patent counts, NPRs,
Field of technology,
Patent class, Age
distribution,
‘Prominence’ of the
Journal in question,
Activity Index, Science
Intensity Index
Publications counts,
Occurrence counts, Year,
Country
Patent counts, NPRcounts, different
variables around type
and characteristics of
NPR (counts, journal
influences, time
distribution etc.)
10.
11.
12.
§
§
Type: direct (NPRs)
Type: linkage between
fields of science
118
Non patent references,
Comparing age distribution in
patent references
Type: direct (through NPRs)
§
§
§
§
§
§
§
§
§
§
§
§
§
§
§
Data source(s): USPTO, EPO (through
WPIL), SCI
T-field selection/class.: IPC, keyword
selection (combined)
S-field selection/class.: CHI ‘Level’
classification of journals
Data standardisation: unification of NPRs,
(format, id, supplementation)
General: -
Data source(s): INSPEC (PACS codes)
T-field selection/class.: Keyword selection
of publications
S-field selection/class.: Data standardisation: General: mapping, Co-classification
(classification code), Co-word (descriptors
in a publication), Activity index
(normalised)
Data source(s): USPTO, SCI
T-field selection/class.: US classification,
SIC product groups
S-field selection/class.: CHI ‘Level’
classification of journals
Data standardisation: format unification,
parsing and identification of the specific
fields
General: analysis of the characteristics cited
literature, examiner references, P(0) and
P(5) in order to analyse the skewness of
distribution
Patent search
technique,
analysis of the
citation
characteristics,
thorough
analysis of the
findings
Insight in the
development of
the field
Validation of
the Activity
Index,
Alternative:
inventor =
scientific
author, suggest
an approach for
validating the
distribution of
NPRs, language
barrier
-
-
-
(Study mainly based
on the results of other
studies)
NPR counts, Science
Intensity Index, # of
patents,
14.
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
§
§
Type: direct/indirect
(university – industry
interactions, NPRs)
Type: direct (NPRs)
§
§
§
§
§
§
§
§
§
§
Data source(s): USPTO, SCI
T-field selection/class: SIC product groups,
selection based on nationality and NPR
S-field selection/class.: SCI classification
of Journals is Science disciplines
Data standardisation: General: survey with 50 random inventors
on the reference motives
Data source(s): EPO
T-field selection/class: S-field selection/class.: Data standardisation: General: identification of patents of
universities through keyword ‘Professor’,
They argued
that a weak
science linkage
of a technology
does not imply a
low univ. –
industry
relationship
Analysis of the
role of public
research
institutes,
validation study
-
-
119
Luwel, M. (1999), Is the Science Citation Index US-biased
Malo S. & A. Geuna (1999), Science-Technology Linkages in an Emerging Research Platform: The Case of Combinational Chemistry and Biology
Hinze, S. (1997), Bibliometric Mapping of Microsystems Engineering and International Actors in the Field
Noyons, E.C.M. et al. (1994), Exploring the science and technology interface: inventor-author relations in laser medicine research
Collins, P. & S. Wyatt (1988), Citations in patents to the basic research literature
Narin, F. & D. Olivastro (1997), Linkage between patents and papers: an interim EPO/US comparison
Korevaar, J.C. & A.F.J. Van Raan (1992), Science Base of Technology: Bibliometric mapping as a tool for national science and technology policy
Narin, F. & D. Olivastro (1992), Status report: Linkage between technology and science
Narin, F., K.S.D. Hamilton & D. Olivastro (1997), The increasing linkage between US technology and public science
Van Vianen, B.G., H.F. Moed & A.F.J. van Raan (1990), The Assessment of the Science Base of Recent Technology
Noyons, E.C.M. & A.F.J. Van Raan (1995), Structuring the dynamics of Neural Network Research
Schmoch, U. et al. (1993), Indicators of the Scientific Base of European Patents
Tijssen, R.J.W & P.K. Butter (1998), Het Belang van Nederlands wetenschappelijk onderzoek voor technologische innovaties: kwantitatieve analyse van octrooien
Meyer-Krahmer, F. & U. Schmoch (1998), Science-based technologies: university – industry interactions in four fields
Corresponding references:
#NPRs per patent,
validation based on
EPO patents
NPR-counts, patent
counts, publication
counts
13.
REFERENCES
Arrow, K. (1994), “The production and distribution of knowledge”, In: G. Silverberg and L.
Soete, (eds.), The Economics of Growth and Technical Change, 1994
Arundel, A., and I. Kabla (1998), “What percentage of innovations are patented? Empirical
estimates for European firms”, Research Policy, 27, pp. 127 – 141
Bain, D. and C. Rinaldini (1989), “Pitfalls of indicator based evaluation”, Commission of
the European Community Joint Research Centre, Ispra, Italy, In: Van Raan et al. (1989)
Braam, R.R. (1991), “Mapping of Science: Foci of Intellectual Interest in Scientific
Literature”, Doctoraatsdissertatie Rijksuniversiteit Leiden, DSWO Press.
Braam, R.R., H.F. Moed and A.F.J. Van Raan (1989), “Comparison and Combination of
Co-Citation and Co-Word Clustering”, In: Van Raan A.F.J., Nederhof A.J., Moed H.F., Eds.
(1989)
Braun, T., W. Glänzel, and A. Schuber (1997), “The Bibliometric Assessment of UK
Scientific Performance: Some Comments on Martin’s ‘Reply’”, Scientometrics, 20, 2, pp.
359-362
Bud, R. (1994) “The Uses of Life: A History of Biotechnology,” Cambridge: Cambridge
University Press.
Callon, M., J. Law and A. Rip (1986), “Mapping the Dynamics of Science and
Technology”, MacMillan Press Ltd., London
Callon, M., J.P. Courtial, W.A. Turner and S. Braun (1983), “From Translations to
Problematic Networks: an introduction to Co-word Analysis”, Social Science Information, 22,
pp. 191-235
Caniels, M.C.J. (1997), “The Geographic Distribution of Patents and Value Added Across
European Regions”, Working Paper, Maastricht Economic Research Institute on Innovation
and Technology (MERIT), Maastricht
Collins, P., and S. Wyatt (1988), “Citations in patents to the basic research literature”,
Research Policy, 17, pp. 65-74
David. P. and D. Foray (1995), “Accessing and Expanding the Science and Technology
Base”, STI Review, pp. 13-68, OECD, Paris
Debackere, K., M. Luwel, R. Veugelers (2000), “Patent Data as a Tool to Monitor S&T
Portfolio’s”, Research Report 0040, Department of Applied Economics, Catholic University
of Leuven, Belgium
120
Deeds, L., DeCarolis, D. en Coombs, J. E. (1999), “Dynamic Capabilities and New Product
Development in High-technology Ventures: An Empirical Analysis of New Biotechnology
Fims”, Journal of Business Venturing, 15, pp. 211-229
Deeds, D. L., DeCarolis, D. en Coombs, J. E. (1997), “The Impact of Firm-specific
Capabilities on the Amount of Capital raised in an Initial Public Offering; Evidence from the
Biotechnology Industry”, Journal of Business Venturing, 12, pp. 31-46
De Bruin, R.E., and H.F. Moed (1993), “Delimination of scientific subfields using cognitive
words from corporate addresses in scientific publications,” Scientometrics 26, pp. 65-80
De Solla Price, D. J. (1963), "Little Science, Big Science", Columbia University Press.
De Solla Price, D. J. (1965), “Is Technology Historically Independent of Science? A study in
Statistical Historiography”, Journal of Technology and Culture, 6, pp. 553 – 568
Dosi, G. and S. Fabiani (1994), “Convergence and Divergence in the Long-term Growth of
Open Economics”, In: Silverberg & Soete (eds.)
Drucker, P.F. (1993), “Post-Capitalist Society”, Oxford: Butterworth Heinemann
Egghe, L., R. Rousseau (1990), “Introduction to Informetrics”, Quantitative Methods in
Library, Documentation and Information Science, pp. 203-290, Elsevier, Amsterdam
Engelsman, E.C., and A.F.J. Van Raan (1991), “Mapping of Technology, A first
exploration of knowledge diffusion amongst fields of technology”, Centre for Science and
Technology Studies (CWTS), Leyden University
Engelsman, E.C., and A.F.J. Van Raan (1994), “A patent-based cartography of
technology”, Research Policy, 23, pp. 1-26
European Commission (1997), “Second European Report on S&T Indicators”, Final Report,
EUR 17 639 EN, Brussels/Luxemburg
French Ministry of Industry (1995), “The Key Technologies for the French Industry”,
Ministère de l’Industrie, de la Poste et des Télécommunications
Garfield, E. (1979), “Citation Indexing. Its Theory and Application in Science, Technology
and Humanities”, Wiley, New York
Gauthier, E. (1998), “Bibliometric analysis of scientific and technological research: a user’s
guide to the methodology”, Observatoire des Sciences et des Technologies, Science and
Technology Redesign Project, Canada, ST-98-08
121
Gibbons, M., C. Limoges, H. Nowotny, S. Schwartzman, P. Scott and M. Trow (1994),
“The New Production of Knowledge”, Sage, London
Glänzel, W. (1996), “The Needs for Standards in Bibliometric Research and Technology”,
Scientometrics, 35, p. 167-176
Glänzel, W. and U. Schoepflin (1992), “Little Scientometrics, Big Scientometrics … And
Beyond?”, Scientometrics 30, pp. 375 – 384
Grilliches, Z. (1984), (ed.), "R&D, Patents and Productivity", University of Chicago Press,
Chicago
Grilliches, Z. (1990), “Patent Statistics as Economic Indicators: A Survey”, Journal of
Economic Literature, 28, pp. 1661-1707
Grilliches, Z., A. Pakes and B.H. Hall (1987), "The value of patents as indicators of
inventive activity.", In: Dasgupta and SP. Stoneman (eds.), "Economic Policy and
Technological Performance", Cambridge University Press, pp.97-124
Grupp, H. (1994), “The Measurement of technical performance of innovations by
technometrics and its impact on established technology indicators”, Fraunhofer Institute for
Systems and Innovation Research (FhG-ISI), Karlsruhe, Germany, Research Policy, 23, pp.
175-193
Grupp, H. (1998), "Foundations of the Economics of Innovation", Edward Elgar,
Cheltenham, UK; Northampton, MA, USA
Grupp, H. and B. Schwitalla (1989), “Technometrics, Bibliometrics, Econometrics, and
Patent Analysis: Towards a Correlated System of Science, Technology and Innovation
Indicators”, In: Van Raan et al. (1989)
Grupp, H. and U. Schmoch (1992a), “Perception of Scientification of Innovation as
Measured by Referencing between Patents and Papers”, In: H. Grupp (ed.), Dynamics
Science-based Innovations, Springer Publishers, Berlin/Heidelberg, pp. 73 – 128
Grupp, H. and U. Schmoch (1992b), “Wissenschaftsbindung der Technik. Panorama der
internationallen Entwicklung und sektorales Tableau für Deutschland; Physica”, Heidelberg
Healey, P., H. Rothman and P.K. Hoch (1986), “An experiment in science mapping for
research planning”, Research Policy, 15, pp. 233 – 251
Hicks, D. (1987), “Limitations of Co-Citations Analysis as a Tool for Science Policy”, Social
Studies of Science, 17, p. 295 – 316
122
Hicks, D. (1995), “Published Papers, Tacit Competencies and Corporate Management of the
Public/Private Character of Knowledge”, Industrial and Corporate Change, No. 4, pp. 401424
Hicks, D., B.R. Martin and J. Irvine (1986), "Bibliometric Techniques for Monitoring
Performance in Technologically Oriented Research: The Case of Integrated Optics", R&D
Management, Vol.16, No.3, pp.211-223
Hinze, S. (1997), “Mapping of Structures in Science and Technology: Bibliometric Analyses
for Policy Purposes”, Doctoraatdissertatie Rijksuniversiteit Leiden, DSWO Press
Katz, J.S. and B.R. Martin (1997), “What is research collaboration?”, Research Policy, 26,
pp. 1-18
Katz, J.S. and D.M. Hicks (1996), “A Systemic View of British Science”, Scientometrics,
35 (1), pp. 133 – 154
Korevaar, J.C., and A.F.J. Van Raan (1992), “Science base of technology. Bibliometric
mapping as a tool for national science and technology policy”. Part I: Recent developments in
catalysis and environmental chemistry, Research Report to The Netherlands Ministry of
Education and Science, Zoetermeer. Report CWTS-92-08, Center for Science and
Technology Studies (CWTS), Leiden, 73 pp.
Laestadius, S. (1998), “The relevance of science and technology indicators: the case of pulp
and paper”, Research Policy, 27, pp. 385 – 395
Luwel, M. (1999a), “Is The Science Citation Index US-Biased”, Scientometrics, Vol. 46, No.
3, pp. 549-562
Luwel, M. (1999b), “Bibliometrisch profiel van het Vlaams onderzoek”, Ministerie van de
Vlaamse Gemeenschap: Administratie Wetenschap en Innovatie
Luwel, M., E.C.M. Noyons and H.F. Moed (1999), “Bibliometric assessment of research
performance in Flanders: Policy background and implications”, R & D Management 29 (2),
pp. 133-141
Malo, S. and A. Geuna (1999), “Science-Technology Linkages in an Emerging Research
Platform: The Case of Combinatorial Chemistry and Biology”, Scientometrics, 33, pp. 381422
Mansfield, E. (1969), “The Economics of Technological Change”, Norton & Company,
London, Great Britain
Mansfield, E. (1991), “Academic Research and Industrial Innovation”, Research Policy, 20
(1), pp. 1 - 20
123
Martin, B., A. Salter, D. Hicks, K. Pavitt, J. Senker, M. Sharp and N. von Tunzelmann
(1996) “The Relationship Between Publicly Funded Basic Research and Economic
Performance - A SPRU Review”, Report prepared for HM Treasury, SPRU, University of
Sussex, Falmer, Brighton, UK
Martin, B.R. (1996), “The Use of Multiple Indicators in the Assessment of Basic Research”,
Scientometrics, 36 (3), pp. 343 – 362
Martin, B.R., Irvine, J. (1983), “Assessing Basic Research. Some Partial Indicators of
Scientific Progress in Radio Astronomy”, Research Policy, 12, pp.61-90
McCulloch, R. (1980), “International Indicators of Science and Technology: How does the
US compare?”, Scientometrics, 2, 5-6
McMillan, G.S., Narin, F. and Deeds, D.L. (2000), “An analysis of the critical role of public
science in innovation: the case of biotechnology”, Research Policy, 29, pp. 1-8.
Meyer, M. (2000a), “Does science push technology? Patents citing scientific literature”,
Research Policy, 29 (3), pp. 409-434
Meyer, M. (2000b), “Patent citations in a novel field of technology – What can they tell
about interactions between emerging communities of science and technology”,
Scientometrics, 48 (1), pp. 151 – 178
Meyer, M. (2000c), “Patent Citation Analysis as a Policy Planning Tool”, Working Paper INN1E496,
SPRU University, Sussex, UK
Meyer, M. and O. Persson (1998), “Nanotechnology – interdisciplinarity, patterns of
collaboration and differences in applications”, Scientometrics, 42, 2, pp. 195-205
Meyer-Krahmer, F. (2000), “Increasing role of Basic Research for Innovation: The Case of
Science Based Technologies. Empirical Evidence and Institutional Consequences”, ISI
Karlsruhe, Conference paper
Meyer-Krahmer, F. and U. Schmoch (1997), “Science-based technologies: universityindustry interactions in four fields”, Research Policy, 27, pp. 835 – 851
Moed, H.F. (1989), "The Use of Bibliometric Indicators for the Assessment of Research
Performance in the Natural and Life Sciences", Doctoraatsdissertatie Rijksuniversiteit Leiden,
DSWO Press
Moed, H.F. (1996), “Differences in the construction of SCI based bibliometric indicators
among various producers: A first overview”, Scientometrics, 35, pp. 177-191
Moed, H.F. and M. Vriens (1989), “Possible Inaccuracies Occuring in Citation Analysis”,
Journal of Information Science, 15, pp. 283-295
124
Moed, H.F., Burger, W.J.M., Frankfort, J.G., Van Raan, A.F.J. (1985), "The use of
bibliometric data for the measurement of university research performance", Research Policy,
14, pp.131-149
Moed, H.F., R.E. De Bruin, and Th.N. Van Leeuwen (1995), “New bibliometric tools for
the assessment of national research performance: database description, overview of indicators
and first applications”, Scientometrics, 33, p. 381-422
Mohnen, P. (1992), “The relationship between R&D and productivity growth in Canada and
other major industrialized countries”, Economic Council of Canada
Mombers, C., A. Van Heeringen, Van Venetië, C. Le Pair (1985), "Displaying Strengths
and Weaknesses in National R&D Performance through Document Co-citation",
Scientometrics, 7 (3-6), pp. 341-355
Narin, F. and K.S. Hamilton (1996), “Bibliometric Performance Measures”, Scientometrics,
36, pp. 293 – 310
Narin, F., and D. Olivastro (1992), “Status Report: Linkage Between Technology and
Science”, Research Policy, 21, pp. 237-330
Narin, F., and E. Noma (1985), “Is technology becoming science?”, Scientometrics, 7 (3-6),
pp. 369-381
Narin, F. and E. Noma (1987), “Patents as indicators of corporate technological strength”,
Research Policy, 16, pp. 143-155
Narin, F., K.S. Hamilton, and D. Olivastro (1997), “The Increasing Linkage Between US
Technology and Science”, Research Policy, 26, pp. 317 – 330
Narin, F., M. Rosen and D. Olivastro (1989), “Patent citation analysis: new validation
studies and linkage statistics”, In: Van Raan et al. (1989)
Nederhof, A.J. (1991), “Delimitation of a medical research topic: interaction with experts in
selecting a database and constructing a search strategy”, Research Evaluation 1, pp. 149-154
Nederhof, A.J., Van Raan, A.F.J. (1993), "A bibliometric analysis of six economics
research groups: A comparison with peer review", Research Policy, 22, pp.353-368
Nelson, R.R. (1994), “What has been the Matter with Neoclassical Growth Theory?”, In:
Silverberg & Soete (eds.)
Nelson, R.R. and S.G. Winter (1974), “Neo-classical vs Evolutionary Theories of Economic
Growth: Critique and Prospectus”, Economic Journal, pp. 886 – 905
125
Nonaka, I. and H. Takeuchi (1995), “The Knowledge-Creating Company. How Japanese
Companies Create the Dynamics of Innovation”, New York, Oxford: Oxford University Press
Noyons, E.C.M. and A.F.J. Van Raan (1998), “Advanced mapping of science and
technology”, Scientometrics, 41 (1-2), pp. 61-67
Noyons, E.C.M., A.F.J. Van Raan, H. Grupp, and U. Schmoch (1994), “Exploring the
science and technology interface: Inventor-author relations in laser medicine research”,
Research Policy, 23, pp. 443-457
Noyons, E.C.M., and A.F.J. Van Raan (1994), “Bibliometric cartography of scientific and
technological developments of an R&D field. The case of optomechatronics”, Scientometrics,
30, pp. 157-173
Noyons, E.C.M., M. Luwel and H.F. Moed (1998), “Assessment of Flemish R&D in the
field of information technology; A bibliometric evaluation based on publication data and
patent data, combined with OECD research input statistics”, Research Policy, 27, pp. 285-300
Noyons, E.C.M., M. Luwel, and H.F. Moed (1999), “Combining mapping and citation
analysis for evaluative bibliometric purposes”, J. of the American Society for Information
Science (JASIS), 50 (2), pp. 115-131
OECD (1992), “Technology and The Economy, The Key Relationships”, Paris
OECD (1994a), “The Measurement of scientific and technological activities. Using Patent
Data as Science and Technology Indicators”, Patent Manual 1994, Paris
OECD (1994b), “Main Definitions and Conventions for the Measurement of Research and
Experimental Development (R&D)”, A Summary of the Frascatti Manual 1993, Paris
OECD (1997), “Bibliometric Indicators and Analysis of Research Systems: Methods and
Examples”, STI working papers 1997/1
Pakes, A. (1986), "Patents as Options: some estimates of the value of holding European
patent stocks", Econometrica, 54, pp.755-784
Pakes, A., Grilliches, Z. (1984), "Patents and R&D at the Firm Level: A First Look", In:
Grilliches, Z. (Ed.), R&D, Patents and Productivity, Chicago, University of Chicago Press
Pavitt, K. (1984), “Sectoral patterns of technical change: towards a taxonomy and a theory”,
Research Policy, 13, pp. 343 – 373
Pavitt, K. (1985), “Patent Statistics as indicators of inventive activities: possibilities and
problems”, Scientometrics, 7, pp. 77-99
126
Pavitt, K. (1987), “The Objectives of Technology Policy”, Science and Public Policy, 14, pp.
182 – 188
Pavitt, K. (1997), “Academic Research, Technical Change and Government Policy”, In
Krige, J. and Pestre, D. (eds.), “Science in the Twentieth Century”, Amsterdam: Harwood
Academic Publishers, pp. 143-158
Perko, J.S. and F. Narin (1997), “The Transfer of Public Science to Patented Technology: A
Case Study in Agricultural Science”, Journal of Technology Transfer, 22, 3, pp. 65 – 72
Pritchard, A. (1969), "Statistical bibliography or bibliometrics?", Journal of Documentation,
25, pp. 348-349
Rabeharisoa, V. (1992), “A special mediation between science and technology. When
inventors publish scientific articles in fuel cells research”, pp. 54 – 72, In: Grupp (1992a)
Rappa, M.A. and K. Debackere (1992), “Technological communities and the diffusion of
knowledge”, R&D Management, 22, pp. 209 – 211
Rip, A. (1992), “Science and technology as dancing partners”, In: Kroes, P., M. Bakker
(Editors), Technological Development and Science in the Industrial Age, Kluwer, n.p., pp.
231-270
Rosenberg, N. (1990), “Why Do Firms Do Basic Research (With Their Own Money)?”,
Research Policy, No. 19 (1990), pp. 165-174
Rosenberg, N. (1992), “Scientific Instrumentation and University Research”, Research
Policy, No. 21, pp. 381-390
Rosenberg, N. and L.E. Birdzell Jr. (1990), “Science, technology and the Western miracle”,
Scientific American, 263 (5), pp. 42-54
Rosenberg, N. (1982), “Inside the black box: technololgy and economics”, Cambridge
University Press, Cambridge
Salter, A. J. and B. R. Martin (1999), “The Economic Benefits of Publicly Funded
Research: A Critical Review”, Science Policy Research Unit, University of Sussex, UK
(forthcoming in Research Policy)
Schankerman, M., Pakes, A. (1985), "The Rate of Obsolescence and the Distribution of
Patent Values: Some Evidence from European Patent Renewals", Revue Econ., 36(5), pp.917941
Scherrer, F.M. (1996), “Patents: what do we know; what must we learn?”, November,
Luxembourg
127
Schmoch, U. (1993), “Tracing the knowledge transfer from science to technology as reflected
in patent indicators”, Scientometrics, 26 (1), pp. 193-211
Schmoch, U. (1997), “Indicators and the relations between Science and Technology”,
Scientometrics, No 38, pp. 103-116
Schmoch, U., E. Strauss, H. Grupp and T. Reiss (1993), “Indicators of the Scientific Base
of European Patents”, Report to the European Commission, EUR 15 330 EN,
Brussels/Luxemburg
Senker, J. (1993), “The Contribution of Tacit Knowledge to Innovation”, AI & Soc. (1993),
No. 7, pp. 208-224
Senker, J. (1995), “Tacit Knowledge and Models of Innovation”, Industrial and Corporate
Change (1995), Vol. 4, No. 2, pp. 425-447
Senker, J. et al. (1999), “Changing Structure, Organisation and Nature of European PSR
Systems, European Comparison of Public Research Systems”, EU (TSER) report
Silverberg, G. and L. Soete (eds.), (1994), “The economic growth and technical change:
technologies, nations, agents”, Elgar Aldershot
Smith, L.C. (1981), "Citation Analysis", In: Potter, W.G. (ed.), “Bibliometrics”, Library
Trends, 30 (1), Summer
Sneath, P.H.A. and R.R. Sokal (1973), “Numerical taxonomy: the principles and practice of
numerical classification”, Freeman San Fransisco (Calif.)
Soete, L.G. and S.M.E. Wyatt (1983), “The Use of Foreign Patenting as an Internationally
Comparable Science and Technology Output Indicator”, Scientometrics, 5 (1), pp. 31-54
Steinmueller, W.E. (1994), “Basic Research and Industrial Innovation”, Chapter 5 in
Dodgson, M. and Rothwell R. (eds.), The Handbook of Industrial Innovation, Edward Elgar,
Aldershot, pp. 54-66
Steinmueller, W.E. (2000), “Economics of Science”, In: Pavitt, K., E. Steinmueller and J.
Calvert & B. Martin (2000), “International Encyclopedia of the Social and Behavioural
Sciences: 3 papers", Paper no. 44 of SPRU Electronic Working Papers Series, Science Policy
Research Unit, University of Sussex, UK
Stoneman, P. (1987), “The economic analysis of technology policy”, Clarendon Oxford,
Oxford
Tidd, J. and Brocklehurst, M. (1999), “Routes to Technological Learning and
Development: An Assesment of Malaysia’s Innovation Policy and Performance”,
Technological Forecasting and Social Change, 62, pp. 239-257
128
Tijssen, R.J.W. (1992), "Cartography of Science: Scientometric Mapping with
Multidimensional Scaling Methods", Doctoraatsdissertatie Rijksuniversiteit Leiden, DSWO
Press, 307 pp.
Tijssen, R.J.W. and R.K. Buter (1998), “Het belang van Nederlands wetenschappelijk
onderzoek voor technologische innovaties: kwantitatieve analyse van octrooien”, Center for
Science and Technology Studies (CWTS), Leiden, The Netherlands
Tijssen, R.J.W., and E. Van Wijk (1999), “In search of the European Paradox: an
international comparison of Europe's scientific performance and knowledge flows in
information and communication technologies research”, Research Policy, 28 (5), pp. 519-543
Tijssen, R.J.W., R.K. Buter and Th.N. Van Leeuwen (2000), “Technological relevance of
science: validation and analysis of citation linkages between patents and research papers”,
Scientometrics, 47, pp. 389-412
Toynbee, A. J. (1963), “Introduction: The Genesis of Civilisations, A Study of History”, 12
vols, 1, 3, New York
Trajtenberg, M. (1987), “Patents, Citations and Innovations: Tracing the Links”, Working
Paper No. 31 – 87, Foerder Institute for Economic Research, Tel-Aviv University, Ramat
Aviv, Israel
Trajtenberg, M. (1990), “A penny for your Quotes. Patent Citations and The Value of
Innovations”, Rand Journal Economics, 21 (1), pp. 172 – 187
Van Amelsvoort, P. (1992), “Het vergroten van de bestuurbaarheid van produktieorganisaties”, Proefschirft ter verkijging van de graad van doctor aan de Technische
Universiteit Eindhoven, ST-groep, Oss, The Netherlands
Van Den Berghe, H., Houben, J.A., De Bruin, R.E., Moed H.F., Kint, A., Luwel, M. and
Spruyt, E.H.J. (1998), "Bibliometric Indicators of University Research Performance in
Flanders", Journal of the American Society for Information Science (JASIS), Vol.49 (1), pp.
59-67
Van Raan, A.F.J. (1993), “Advanced bibliometric methods to assess research performance
and scientific development. Basis principles and recent practical applications”, Report
CWTS-93-05, Center for Science and Technology Studies (CWTS), Leiden University, The
Netherlands
Van Raan, A.F.J. (1997), “Scientometrics: State of the Art”, Scientometrics, 38 (1), pp. 205218
129
Van Raan, A.F.J. and H.P.F. Peters (1989), “Dynamics of a field analysed by co-subfield
structures”, Scientometrics, 15, pp. 607 – 620
Van Vianen, B.G., H.F. Moed and A.F.J. van Raan (1990), “An Exploration of the Science
Base of Recent Technology”, Research Policy, 19, pp. 61-81
Verspagen, B. (1994), “Technology and Growth; the Complex Dynamics of Convergence
and Divergence”, In: Silverberg & Soete (1994)
Verspagen, B., T. van Moergestel and M. Slabbers (1994), “MERIT concordance table:
IPC – ISIC (rev. 2)”, MERIT Research Memorandum 94-004, University of Limburg
Maastricht, The Netherlands
Ziman, J. (1994), “Prometheus Bound. Science in a Dynamic Steady State”, Cambridge
University Press, Cambridge
130