Quantity Does Not Equal Quality in Evaluating a

Quantity Does Not Equal Quality in
Evaluating a Scientist’s Real Importance as
a Key Opinion Leader
Research Report
Social Network Analysis
Finds that 50% of
Key Scientific Leaders
Are “Low Publishers”
In Scientific Journals
Key Takeaway: Finding key opinion leaders and
effectively engaging thought leadership creates competitive advantage. The top
– down approach – to analyze an entire medical research area using social
network analysis – far surpasses the
current bottom – up method that relies on standard publicity/publishing metrics.
Who Should Read This: Medical and Biomedical
executives seeking new thinking from 3rd party thought leadership and
advocates in scientific advisory board and medical affairs KOL capacities.
Specifically:
• KOL Relationship Management
• Opinion Leader Relations
• Medical and Scientific Affairs
• Scientific Advisory Board Managers
• Senior Marketing Managers
• Medical Science Liaisons
• Medical Research Executives
• Advocacy Development
• Market Research
• Brand and Product Management
Why This is Important: Pharmaceutical and biotechnology
companies of all sizes are affected by ongoing changes in legislation surrounding
the appropriate use of industry thought
leadership in development and marketing
of medical products. Companies that evolve their advocacy strategies using undiscovered relationships and powerful people often find their go-to-market
strategies positively impacted.
Philip Topham, MBA
[email protected]
LNX PHARMA © 2010
Quantity Does Not Equal Quality in
Evaluating a Scientist’s Real Importance as
a Key Opinion Leader
Research Report
Social Network Analysis Finds that 50% of All Key Scientific Leaders
Are “Low Publishers” In Scientific Journals
Executive Summary
This study questions current methodology used extensively in pharmaceutical and biotech
industries to find Key Opinion Leaders (KOLs) and influential thought leaders.
Lnx Pharma research shows that bibliometrics as a means of discovering and ranking potential
product advocates fails to find over 50% of the truly respected experts, compared to using the
more advanced Large Scale Social Network Analysis (LS_SNA). The complex relationship web amongst researchers contains a wealth of information that can
be extracted using LS-SNA. Researchers leave ample evidence for their preferred working
relationships through their publications, which follow the rigor of scientific method and evidence
based discovery. Consequently it is only natural to use research publications to identify experts or
key opinion leaders (KOLs). Yet for all the wealth of information, no amount of counting
publications will determine “respect.” A person could publish 10 times or 100 times and still
not be respected. And yet, some companies continue to utilize this outdated and ineffective
technique.
The following white paper challenges the generally-held notion that people who are “productive”
are in fact well connected and meaningful to the community. We found that in a large number of
cases, the opposite is true.
Background
KOLs as “Influencers”
With such high costs and risks in developing new drugs1 and potentially catastrophic results2
from not understanding both the science and the marketplace, selecting Key Opinion Leaders (i.e.
expert consultants) has always been an incredibly important part of the advocacy continuum. But,
until recently, KOLs were often used primarily as extensions to the sales force or medical affairs
team. Thus choosing KOLS was often based more on their prescription patterns than on their
expertise.
A common practice was to provide gifts to high prescribers and lavish honoraria to speakers to
promote a product3. There are many documented examples of trial results that were tampered
with or doctored to make the outcomes look better because the principal investigator stood to
make substantial earnings4. The public and regulatory backlash from such “influence peddling”
has not only moved the regulatory pendulum but fundamentally shifted the center.
LNX PHARMA © 2010
2
Worldwide regulations and guidelines5 now require separation between Commercial Activities and
Medical Affairs. Naturally this increases the need to find more KOLs.
Outcome: Because of policy changes, and also in response to economics in the industry, many
top pharmaceutical and biomedical executives have a new and pressing need to re-evaluate their
current strategy for finding and recruiting outside thought leadership in all stages of drug discovery
and development.
Opportunity: Advanced methods of social network analysis make it possible to not only discover
new and different KOLs, but also to discover insights about their communities at both a qualitative
and quantitative level.
KOL defined
Key Opinion Leaders6 – Highly respected medical experts within their domain, by
which their thoughts and actions have a greater (asymmetric) effect on their peers
with regards to adopting a new idea, product or service. In other words, KOLs have
a large impact on the diffusion of innovation.
The Art of Advocacy: Finding the Right Thought Leaders
With pharmaceutical regulatory oversight for KOL identification and engagement increasing
significantly, it is more important than ever to understand each method for KOL identification to
ensure compliance while maintaining effectiveness of the business goals and objectives. Little
public research has been done to analyze and compare the different methods7 of KOL
identification.
This research paper compares the Large Scale Social Network Analysis method8 (LS-SNA) with the
bibliometric method.
The LS-SNA method has been used to evaluate more than 20 different disease areas
including:
LNX PHARMA © 2010
•
•
•
•
•
•
•
Obesity
Blood Coagulation Disorders Infectious Diseases
Autism
Cardiology
Oncology - Breast
Oncology - Lung
•
•
•
•
•
•
•
Oncology - Ovarian
Oncology - Colorectal
Rheumatology
Central Nervous System
Diagnostics & Biomarkers
Insomnia
Diabetes
3
The Incumbent: Bibliometric KOL Evaluation
The bibliometric approach counts accomplishments within a particular expertise domain.
Accomplishments can include the number of publications written, clinical trials managed,
speeches given, training sessions given, advisory board participation, editorial board leadership
positions held and the like. The bibliometric process can be as simple or elaborate as desired but
still relies on counting things. Often, weightings are given to various accomplishments to denote
more importance to the person looking for specific expertise. For example, a company recruiting
KOLS with clinical trial matters may emphasize and count clinical trials more heavily than participation on an editorial review board. To find and qualify expertise, counting scientific peer review
journal articles is a common practice9.
High Publishers and Low Publishers
The quantity of articles a person writes during a period of time (e.g. the last five years) is counted.
People with high article counts are generally considered to have higher expertise. Low publishers
are likely to be new or less well known individuals. But this method does not account for the highly
respected and well known expert with low publications.
How can a “low publisher” be a highly respected expert? There are many reasons and
situations that could create such a pattern, such as:
1.
2. 3.
4.
Rising stars: individuals early in their careers who have published very impactful papers but have not yet generated a volume of work;
Individuals on sabbatical or semi-retired who have established respect through previous works but are now publishing at a lower volume;
Individuals who join or leave the industry and thus have a hiatus in their
academic research frequency; or
Highly respected KOLs with prestige in another – typically related – field, yet few publications in the field being studied.
Let’s call these individuals the “Invisible Experts.”
How can companies find Invisible Experts? Typically, companies must either limit themselves to the
candidate pool of highly published experts or rely on other methods to find these hidden experts
– such as asking their core group of experts for additional recommendations. This approach is
known as “snowball sampling”10.
The limitations of such an approach should be clear to the reader (geographical influences, limited
to top-of-mind awareness, selection bias, survey design bias, fitting the results to meet prior
perceptions, etc.), and yet it is one of the most commonly used methods of growing an advocacy
or KOL network.
What is a Social Network? Social Networks are much more than Facebook or
Twitter. As people (or organizations) interact they form groups and cliques. These
interactions together are a social network. Social networks can be based on
friendships, family relationships, political relationships, business transactions, joint
ventures, medical needs, shared values (clubs and organizations), schools and much more.
What is social network analysis? Social network analysis – SNA for short – is the
formal method for mapping social interactions, and then applying mathematical
models to understand people and the social structures they form.
LNX PHARMA © 2010
4
The Contender: Social Network
Analysis KOL Discovery &
Evaluation
Friendship Network Example
The social network analysis11, 12, aapproach is based on social science and
anthropology principles that began in the
1800s. They were used more formally
in the early 1900s to study familial and
other relationships in small communities.
Researchers literally mapped connections
between one family member and another
(see Figure 1 - Friendship Network
Example.) These maps led to greater
understanding than a list of names
Figure 1
alone. But it wasn’t until the 1970s that
mathematical models were developed
sufficiently to systematically extract meaning
from such social networks13. Because the computing power required increases logarithmically with
the size of the community, these models were limited to small communities and could not be used
to study entire research communities. Today with powerful computing
it is possible to analyze large and very large networks .
Social Prestige
Unlike the bibliometric method, social network analysis models “social
prestige.” The adage “birds of a feather tend to flock together”15
explains much about social networks: a person prefers to work with
people with similar interests and compatible value systems. For
example, consider that many consumer brands seek Olympic Gold
winners and athletic superstars (e.g. Kobe Bryant, Michael Phelps,
and Tiger Woods) for endorsement advertising. Companies know that
some of that “social respect” transfers through association. But when
the famous person is caught doing something perceived as immoral
or illegal16, social norms generally dictate that the endorsements be
canceled and the sponsorships withdrawn. The relationships are
severed because the Olympian’s “social prestige” was damaged and
thus significantly diminished.
Even within science, researchers make choices about relationships.
For example, a brain cancer researcher who is given a choice
between working with a prestigious cancer researcher at a famous
institution versus an unknown researcher with limited resources is
more likely to choose the famous researcher. This choice confirms
and reinforces that the famous researcher has more “social respect
and prestige.”
Figure 2
The social network analysis approach examines and calculates
metrics17 across the entire network of relationships. Different combinations of these measures reflect the role a person holds within the community. In his famous book
The Tipping Point: How Little Things Can Make a Big Difference18, Malcom Gladwell popularizes
this social network principal by identifying three types of people: The Maven (or technical expert),
The Connector (or well connected expert), and the Sales Person (or promoter).
In more practical terms, when we attend a professional society meeting, the more important
people are typically on stage and central to the event; they typically hold formal titles like
LNX PHARMA © 2010
5
Chairman, Vice Chair or Committee Member etc. Although it is a relatively trivial matter through
observation to figure out who’s most important, it would be a more daunting task to pick important
people out of the general audience.
Bibliometrics might solve this problem by counting the number of times a person attends
professional society meetings. But clearly this method might erroneously nominate a faithful
member who attends every meeting but never truly contributes.
Social network analysis takes the guesswork out of the process by looking at all the relationships.
LS-SNA makes it easy to identify, compare and rank people. Method
Natural questions that might arise for teams responsible for selecting KOLs:
How do the two methods compare? (Bibilometrics vs. LS-SNA)
How often are highly respected people actually low publishers?
Does counting publications find all the key people?
This method section outlines the process used to answer the questions above.
Data
Social network analysis projects
covering a variety of research topics
(see Table 2) were included in the
comparison. Each data set represents
5 years (2005-2009) of publication19
data.
Data Preparation
Each data set is prepared by cleaning
and disambiguating names. Both the
problems with synonyms and
homonyms are resolved. Homonyms
deal with names that are spelled
exactly alike; such as “Robert Smith”
and “Robert Smith.” If the authors are
the same person they are merged into
a single master person record. If the
authors are different people then two
master person records are created. A
similar process handles synonyms,
names with different spellings may
refer to the same person (e.g. “Bob”
is a common substitution for the name
“Robert”).
Network Components
Figure 3
Network Creation
Each researcher is represented as a node on a graph. A line (also called an edge) is drawn on the
graph for each co-authorship relationship. For example if John Smith and Alice Jones have written
a research paper together, then a line is drawn between the two to represent their collaboration.
See Figure 3.
LNX PHARMA © 2010
6
Social network analysis
After a graph of all researchers is created, the various centrality measures are calculated20. The
process naturally identifies the “core community”21, the largest contiguously connected group of
researchers.
Ranking
The various social network analysis centrality measures are combined into single SNA scores22
which provide an ordinal ranking from top to bottom for each of the researchers. Additionally, each
researcher is ranked according to their publication volume.
Identifying KOLs
Researchers with either a combined SNA
score or high publication score in the top
2% of researchers in the disease area
are selected for comparison.23, 24
Publication and SNA rank score comparison
Categorizing KOLs
Each KOL candidate is put into one of
three categories:
•
•
•
High Publishers with Low SNA – researchers with high publication
rank.
Low Publishers with High SNA – researchers with low publication rank but having a high SNA score.
High Publishers with High SNA –
researchers with high publication rank and a high SNA score.
Comparison
The KOL data is collated and
compared.
Figure 4
Findings and Discussion
Community Size
As expected, community size varies by interest area, with some areas attracting larger quantities of
researchers (and presumably resource funding) than other areas. For example, “Area D: Soft Tissue
Cancer” has more than four times the researchers than “Area C: Insomnia.”
Community Cohesion
Core community size relates to the community’s cohesiveness; the ability of all researchers to
connect to each other through contiguous connections. Communities with limited or focused
research doctrines are more cohesive than communities with several research doctrines25.
This difference in communities can impact KOL engagement strategy. In very cohesive
communities there are fewer differences of opinion and thus it’s harder to overcome “group think,”
but once a new idea gets established it spreads through the many connections quickly.
Across the eleven areas (see Table 2 – Research areas and disease topics) cohesiveness in the core
community varies from 16.2% to 69.4% of the entire community.
LNX PHARMA © 2010
7
KOL Comparison - High Publishers versus High SNA Scores
Table 1 – Averages for Categorized KOL Candidates26
High Pubs
High Pubs
Low Pubs
Total
Low SNA
High SNA
High SNA
All Eleven Areas
692
15.8%
1155
26.4%
2528
57.80%
4375
100%
Excluding Vaccine (area K)
691
15.88%
1143
26.27%
2517 57.85%
4351
100%
Adopting LS-SNA as Methodology for KOL Discovery
Failure to adopt the LS-SNA method would mean companies are missing 50% of the KOLs who
could help them most. The possible alternative of loosening the publication criteria could mean an
increasing burden of sifting though ever greater numbers of experts who may be missing social
prestige.
Changing processes from any old method to a newer method requires careful change
management. Companies should include publication statistics with LS-SNA projects until they are
comfortable with the new method. Fortunately, combining publications counts with the LS-SNA
process is negligible additional work and can be easily performed at the same time compared to
the LS-SNA process alone.
Conclusion
Bibliometrics alone fails to find over 50% (57.8%) of the respected experts compared to Large
Scale Social Network Analysis. Three categories of experts emerge from community analysis and can be summarized as
follows:
Insular Experts (High Pubs – Low SNA) : Many publications compared to their peers, yet are lacking the social prestige necessary to be collegially accepted. They may be
acknowledged and somewhat respected, but they are not included in meaningful social interactions. About 16% of KOLs fall into this category.
Visible KOLs (High Pubs – High SNA): The combination of many publications with many author-to-author interactions create high visibility for these people in all dimensions. These KOLS are often currently engaged and competition for their services fierce.
Invisible KOLs (Low Pubs – High SNA): With strong roles (high SNA metrics) within the community but lacking evidence of publications, these individuals are often overlooked by those in key positions as thought leadership team builders. As stated, Invisible KOLs are often “rising stars;” or individuals on sabbatical or semi-retired; or individuals that join or leave an industry; or highly respected KOLs with prestige in a different field. About 58% of what would be the considered most desirable and respected KOLs fall into this category.
LNX PHARMA © 2010
8
Companies need outside thought leadership for a number of reasons:
Innovation: Infuse company researchers with new ideas (i.e. to avoid group think).
Expertise: Use the wisdom of crowds to identify promising research pathways and
reduce risks.
Strategy: Identify new markets and adjacent markets.
Tactical: Help design clinical trial studies.
Collaboration: Create opportunities for sharing costs and reducing risks.
Education: Quickly train staff when needed (e.g. for a newly licensed product in a
disease area unfamiliar to the licensee)
Using standard bibliometric KOL identification methods has a very high failure rate. Using this
method, about 38%27 of “experts” fail to meet the definition for “well respected.” In other words,
about 38% do not appear to have the real social prestige necessary to be considered “well
respected experts.”
Although the vetting process may filter out these 38%, it will be at great expense, time and effort. However, it is the intangibles such as “starting on the wrong foot” with the wrong KOLs in an
advocacy team that may ultimately damage one’s credibility the most, and potentially cause
additional costs and increased risks.
THEORY: People who are very “productive” in terms of publications and publicity
are very connected and meaningful to the community.
CONCLUSION: This report shows that not all KOLs are alike, and that high
publication volume does not always indicate the highest level of social prestige and
respect.
Some High Publishers’ ideas may be meaningful, but comprehensive LS-SNA
reveals that they don’t always have the relationships within the community
necessary to evangelize ideas as easily to peers and colleagues. If someone is an
isolated producer, it’s difficult for ideas to flow. Conversely, there are researchers
and doctors that companies might dismiss because there is lower evidence of their
importance as traditionally measured by publishing counts, but the LS-SNA method
finds them highly connected in roles such as advisors and trusted colleagues, or
subject matter experts to the community.
LNX PHARMA © 2010
9
Research areas and disease topics
Research Area
Total Unique
Researchers
Core
Researchers
Core
Community
Total Articles
Articles for
Core
Community
Article % for
Core
Community
Average
authors
per
Article
A
Cardiovascular
20,370
9,091
44.6%
7,249
3,864
53.3%
5.26
B
Infectious Diseases
32,945
11,837
35.9%
9,244
4,033
43.6%
5.52
C
Insomnia
9,211
3,396
36.9%
3,425
1,423
41.5%
4.13
D
Soft Tissue Cancer
40,500
13,046
32.2%
10,610
5,252
49.5%
5.53
E
Nosocomial Infections
3,128
1,372
43.9%
991
538
54.3%
4.96
F
Oncology - Blood
30,242
20,984
69.4%
10,693
6,693
62.6%
6.26
G
Oncology - Blood
26,685
14,798
55.5%
7,199
4,257
59.1%
6.56
H
Anemia
7,854
2,438
31.0%
2,326
826
62.6%
6.26
I
Oncology - Various
Products
16,224
5,183
31.9%
4,503
1,591
35.3%
4.97
J
Uterine Cancer
11,644
5,352
46.0%
3,462
1,507
43.5%
5.56
K
Vaccine
3,703
601
16.2%
1,123
193
17.2%
3.90
202,506
88,098
43.5%
60,825
30,177
49.6%
5.55
Totals & Averages
Table 2
The largest two rows from each metric is bolded.
Research Area – each area is defined by collecting medical articles28 based on keywords and
extracting the co-author relationships
Total Unique Researchers – after data preparation and name disambiguation, this represents the
total number of unique names
Core Researchers – the total number of unique researchers contiguously connected to each other
Core Community – the ratio of core researchers to total unique researchers expressed as a
percentage
Total Articles – the total number of articles collected for the research area
Articles for Core Community – the number of articles written by researchers in the giant
component
Article% for Core Community – the ratio of total articles attributable to the core researchers
compared to total articles expressed as a percentage
Average Authors per Articles – the ratio of authors (not unique researchers) to the total number of
articles
LNX PHARMA © 2010
10
Categorized KOL Candidates /Sorted by High Publication subtotal, highest to smallest/
Research Area
High
Pubs
High
Pubs
Low
Pubs
Low
SNA
High
SNA
High
SNA
High Pubs
High Pubs
Low SNA
High SNA
Subtotal
%
High SNA
Total
Low Pubs
Total %
K
Vaccine
1
12
11
24
4.2%
50%
54.2%
45.8%
100%
G
Oncology - Blood
82
224
313
619
13.2%
36.2%
49.4%
50.6%
100%
F
Oncology - Blood
143
296
536
975
14.7%
30.4%
45.1%
55%
100%
E
Nosocomial
Infections
10
19
36
65
15.4%
29.2%
44.6%
55.4%
100%
H
Anemia
42
111
196
349
12%
31.8%
43.8%
56.2%
100%
B
Infectious Diseases
104
172
366
642
16.2%
26.8%
43%
57%
100%
I
Oncology - Various
Products
54
66
172
292
18.5%
22.6%
41.1%
58.9%
100%
D
Soft Tissue Cancer
109
115
343
567
19.2%
20.3%
39.5%
60.5%
100%
C
Insomnia
35
45
123
203
17.2%
22.2%
39.4%
60.6%
100%
J
Uterine Cancer
67
48
204
319
21%
15%
36%
63.9%
100%
A
Cardiovascular
45
47
228
320
14.1%
14.7%
28.8%
71.3%
100%
692
1155
2528
4375
15.8%
26.4%
42.2%
57.8%
100%
Totals & Averages
Table 3
About: Lnx Pharma
Lnx Pharma is a division of Lnx Research, LLC, a privately held company based in Orange,
California, dedicated to leadership in the analysis of knowledge-creating communities. Lnx
Research utilizes proprietary social network analysis methods and technologies to identify and
understand “The Invisible College” of key opinion leaders in order to answer questions and build
strategies within knowledge communities for government and commercial clients in multiple
industries.
For white papers on this topic, or more specific information on Lnx Pharma,
email [email protected], or see www.lnxpharma.com
LNX PHARMA © 2010
11
End Notes
DiMasi, J.A., H.G. Grabowski. The Cost of Biopharmaceutical R&D. Managerial and Decision Economics. 28, 469-479
(2007)
2
“Pfizer to pay $2.3 billion to resolve criminal and civil health care liability relating to fraudulent marketing and the
payment of kickbacks.” U.S. Department of Health and Human Services and U.S. Department of Justice. Stop Medical
Fraud. http://www.stopmedicarefraud.gov/pfizerfactsheet.html
3
Johnson, Carrie. In Settlement, A Warning To Drugmakers. Washington Post (September 3, 2009)
4
Sang-hun, Choe. Disgraced Cloning Expert Convicted in South Korea. The New York Times (October 27, 2009 sec.
International/Asia Pacific.)
5
United States Office of the Inspector General - http://www.oig.hhs.gov/authorities/docs/03/050503FRCPGPharmac.pdf
Association British Pharmaceutical Industry - Code of Practice http://www.abpi.org.uk/links/assoc/PMCPA/Code06use.pdf
European Federation of Pharmaceutical Industries and Associations - http://www.efpia.eu/Content/Default.
asp?PageID=559&DocID=3484
6
For purposes of this paper, this definition will be used as the baseline and generally includes Thought Leader, Key
Scientific Leader, Clinical Practice Leader and many others. The specific definition varies greatly from one company to
another and even within the same company, with some companies creating elaborate definitions to identify and match
with specific consulting roles; but most fundamental is the need for a “well respected expert.”
7
Whitepaper #2. Finding Key Opinion Leaders Using Large Scale Social Network Analysis, A Comparative Analysis of
Methods for Finding Key Opinion Leaders” http://www.lnxpharma.com/our-story/resources/
8
Whitepaper #1. Finding Key Opinion Leaders Using Social Network Analysis http://www.lnxpharma.com/our-story/
resources
9
http://en.wikipedia.org/wiki/Bibliometrics
10
http://en.wikipedia.org/wiki/Snowball_sampling
11
Hanneman, Robert A., and Mark Riddle. Introduction to Social Network Methods. University of California, Riverside.
(2005) http://www.faculty.ucr.edu/~hanneman/nettext/.
12
Marin, Alexandra, and Barry Wellman. Social Network Analysis: An Introduction. University of Toronto, June 11, 2009.
(Forthcoming in Handbook of Social Network Analysis. Edited by Peter Carrington and John Scott. London: Sage,
2010)
13
Freeman, L. C. Centrality in social networks: Conceptual clarification. Social Networks, 1(3), 215-239. (1979)
14
“Large” is a relative term - as improvements in computing power increase, today’s large is tomorrow’s small. At
present, for academic researchers,” large generally means 5,000 to 10,000 researchers. For commercial processes,
typically “large” means 100,000+ researchers.
15
Miller McPherson, Lynn Smith-Lovin, and James M Cook, BIRDS OF A FEATHER: Homophily in Social Networks.
Annual Review of Sociology Vol. 27: 415-444. (2001)
16
Steven N. Durlauf and Lawrence E. Blume (Eds), ‘Social Norms’ in New Palgrave Dictionary of Economics, Second
Edition, London: Macmillan (forthcoming).
17
Some of the more common network centrality measures are betweeness centrality, first degree centrality, closeness
centrality, and Eigenvector centrality.
18
Gladwell, Malcolm. The Tipping Point: How Little Things Can Make a Big Difference. 1st ed. Little, Brown and
Company. (2000)
19
Data source http://www.pubmed.gov
20
David Knoke and Song Yang, Social Network Analysis. vol. 154, 2nd ed., Series: Quantitative Applications in the
Social Sciences 07 Sage Publications Ltd. (2008)
21
In SNA terms, the core community is known as the giant component and represents the largest contiguously
connected group of people.
22
The SNA score uses the proprietary LnxPharma score, which combines several centrality measures into a single
composite number.
23
For both Oncology (Blood) areas and Anemia the top 1% were used for the cutoff. In all cases the cutoff percent
used for SNA scores was also the same cutoff value used for high publication scores.
24
In a typical bibliometric search, the cutoff value is gradually increased until the desired number of researchers is
obtained.
25
Oxoby, Robert. Understanding Social Inclusion, Social Cohesion and Social Capital. Department of Economics,
University of Calgary. (2009)
26
For the eleven areas, some might consider Area K – Vaccine at 24 individuals to be a small sample size; thus the
averages were calculated with and without Area K. This had a negligible impact, changing from 57.80% to 57.85%.
27
38% is calculated by of taking the as a percentage “High Pubs+Low SNA” compared to “All High Publishers”
28
http://www.pubmed.gov
1
LNX PHARMA © 2010
12