Quantity Does Not Equal Quality in Evaluating a Scientist’s Real Importance as a Key Opinion Leader Research Report Social Network Analysis Finds that 50% of Key Scientific Leaders Are “Low Publishers” In Scientific Journals Key Takeaway: Finding key opinion leaders and effectively engaging thought leadership creates competitive advantage. The top – down approach – to analyze an entire medical research area using social network analysis – far surpasses the current bottom – up method that relies on standard publicity/publishing metrics. Who Should Read This: Medical and Biomedical executives seeking new thinking from 3rd party thought leadership and advocates in scientific advisory board and medical affairs KOL capacities. Specifically: • KOL Relationship Management • Opinion Leader Relations • Medical and Scientific Affairs • Scientific Advisory Board Managers • Senior Marketing Managers • Medical Science Liaisons • Medical Research Executives • Advocacy Development • Market Research • Brand and Product Management Why This is Important: Pharmaceutical and biotechnology companies of all sizes are affected by ongoing changes in legislation surrounding the appropriate use of industry thought leadership in development and marketing of medical products. Companies that evolve their advocacy strategies using undiscovered relationships and powerful people often find their go-to-market strategies positively impacted. Philip Topham, MBA [email protected] LNX PHARMA © 2010 Quantity Does Not Equal Quality in Evaluating a Scientist’s Real Importance as a Key Opinion Leader Research Report Social Network Analysis Finds that 50% of All Key Scientific Leaders Are “Low Publishers” In Scientific Journals Executive Summary This study questions current methodology used extensively in pharmaceutical and biotech industries to find Key Opinion Leaders (KOLs) and influential thought leaders. Lnx Pharma research shows that bibliometrics as a means of discovering and ranking potential product advocates fails to find over 50% of the truly respected experts, compared to using the more advanced Large Scale Social Network Analysis (LS_SNA). The complex relationship web amongst researchers contains a wealth of information that can be extracted using LS-SNA. Researchers leave ample evidence for their preferred working relationships through their publications, which follow the rigor of scientific method and evidence based discovery. Consequently it is only natural to use research publications to identify experts or key opinion leaders (KOLs). Yet for all the wealth of information, no amount of counting publications will determine “respect.” A person could publish 10 times or 100 times and still not be respected. And yet, some companies continue to utilize this outdated and ineffective technique. The following white paper challenges the generally-held notion that people who are “productive” are in fact well connected and meaningful to the community. We found that in a large number of cases, the opposite is true. Background KOLs as “Influencers” With such high costs and risks in developing new drugs1 and potentially catastrophic results2 from not understanding both the science and the marketplace, selecting Key Opinion Leaders (i.e. expert consultants) has always been an incredibly important part of the advocacy continuum. But, until recently, KOLs were often used primarily as extensions to the sales force or medical affairs team. Thus choosing KOLS was often based more on their prescription patterns than on their expertise. A common practice was to provide gifts to high prescribers and lavish honoraria to speakers to promote a product3. There are many documented examples of trial results that were tampered with or doctored to make the outcomes look better because the principal investigator stood to make substantial earnings4. The public and regulatory backlash from such “influence peddling” has not only moved the regulatory pendulum but fundamentally shifted the center. LNX PHARMA © 2010 2 Worldwide regulations and guidelines5 now require separation between Commercial Activities and Medical Affairs. Naturally this increases the need to find more KOLs. Outcome: Because of policy changes, and also in response to economics in the industry, many top pharmaceutical and biomedical executives have a new and pressing need to re-evaluate their current strategy for finding and recruiting outside thought leadership in all stages of drug discovery and development. Opportunity: Advanced methods of social network analysis make it possible to not only discover new and different KOLs, but also to discover insights about their communities at both a qualitative and quantitative level. KOL defined Key Opinion Leaders6 – Highly respected medical experts within their domain, by which their thoughts and actions have a greater (asymmetric) effect on their peers with regards to adopting a new idea, product or service. In other words, KOLs have a large impact on the diffusion of innovation. The Art of Advocacy: Finding the Right Thought Leaders With pharmaceutical regulatory oversight for KOL identification and engagement increasing significantly, it is more important than ever to understand each method for KOL identification to ensure compliance while maintaining effectiveness of the business goals and objectives. Little public research has been done to analyze and compare the different methods7 of KOL identification. This research paper compares the Large Scale Social Network Analysis method8 (LS-SNA) with the bibliometric method. The LS-SNA method has been used to evaluate more than 20 different disease areas including: LNX PHARMA © 2010 • • • • • • • Obesity Blood Coagulation Disorders Infectious Diseases Autism Cardiology Oncology - Breast Oncology - Lung • • • • • • • Oncology - Ovarian Oncology - Colorectal Rheumatology Central Nervous System Diagnostics & Biomarkers Insomnia Diabetes 3 The Incumbent: Bibliometric KOL Evaluation The bibliometric approach counts accomplishments within a particular expertise domain. Accomplishments can include the number of publications written, clinical trials managed, speeches given, training sessions given, advisory board participation, editorial board leadership positions held and the like. The bibliometric process can be as simple or elaborate as desired but still relies on counting things. Often, weightings are given to various accomplishments to denote more importance to the person looking for specific expertise. For example, a company recruiting KOLS with clinical trial matters may emphasize and count clinical trials more heavily than participation on an editorial review board. To find and qualify expertise, counting scientific peer review journal articles is a common practice9. High Publishers and Low Publishers The quantity of articles a person writes during a period of time (e.g. the last five years) is counted. People with high article counts are generally considered to have higher expertise. Low publishers are likely to be new or less well known individuals. But this method does not account for the highly respected and well known expert with low publications. How can a “low publisher” be a highly respected expert? There are many reasons and situations that could create such a pattern, such as: 1. 2. 3. 4. Rising stars: individuals early in their careers who have published very impactful papers but have not yet generated a volume of work; Individuals on sabbatical or semi-retired who have established respect through previous works but are now publishing at a lower volume; Individuals who join or leave the industry and thus have a hiatus in their academic research frequency; or Highly respected KOLs with prestige in another – typically related – field, yet few publications in the field being studied. Let’s call these individuals the “Invisible Experts.” How can companies find Invisible Experts? Typically, companies must either limit themselves to the candidate pool of highly published experts or rely on other methods to find these hidden experts – such as asking their core group of experts for additional recommendations. This approach is known as “snowball sampling”10. The limitations of such an approach should be clear to the reader (geographical influences, limited to top-of-mind awareness, selection bias, survey design bias, fitting the results to meet prior perceptions, etc.), and yet it is one of the most commonly used methods of growing an advocacy or KOL network. What is a Social Network? Social Networks are much more than Facebook or Twitter. As people (or organizations) interact they form groups and cliques. These interactions together are a social network. Social networks can be based on friendships, family relationships, political relationships, business transactions, joint ventures, medical needs, shared values (clubs and organizations), schools and much more. What is social network analysis? Social network analysis – SNA for short – is the formal method for mapping social interactions, and then applying mathematical models to understand people and the social structures they form. LNX PHARMA © 2010 4 The Contender: Social Network Analysis KOL Discovery & Evaluation Friendship Network Example The social network analysis11, 12, aapproach is based on social science and anthropology principles that began in the 1800s. They were used more formally in the early 1900s to study familial and other relationships in small communities. Researchers literally mapped connections between one family member and another (see Figure 1 - Friendship Network Example.) These maps led to greater understanding than a list of names Figure 1 alone. But it wasn’t until the 1970s that mathematical models were developed sufficiently to systematically extract meaning from such social networks13. Because the computing power required increases logarithmically with the size of the community, these models were limited to small communities and could not be used to study entire research communities. Today with powerful computing it is possible to analyze large and very large networks . Social Prestige Unlike the bibliometric method, social network analysis models “social prestige.” The adage “birds of a feather tend to flock together”15 explains much about social networks: a person prefers to work with people with similar interests and compatible value systems. For example, consider that many consumer brands seek Olympic Gold winners and athletic superstars (e.g. Kobe Bryant, Michael Phelps, and Tiger Woods) for endorsement advertising. Companies know that some of that “social respect” transfers through association. But when the famous person is caught doing something perceived as immoral or illegal16, social norms generally dictate that the endorsements be canceled and the sponsorships withdrawn. The relationships are severed because the Olympian’s “social prestige” was damaged and thus significantly diminished. Even within science, researchers make choices about relationships. For example, a brain cancer researcher who is given a choice between working with a prestigious cancer researcher at a famous institution versus an unknown researcher with limited resources is more likely to choose the famous researcher. This choice confirms and reinforces that the famous researcher has more “social respect and prestige.” Figure 2 The social network analysis approach examines and calculates metrics17 across the entire network of relationships. Different combinations of these measures reflect the role a person holds within the community. In his famous book The Tipping Point: How Little Things Can Make a Big Difference18, Malcom Gladwell popularizes this social network principal by identifying three types of people: The Maven (or technical expert), The Connector (or well connected expert), and the Sales Person (or promoter). In more practical terms, when we attend a professional society meeting, the more important people are typically on stage and central to the event; they typically hold formal titles like LNX PHARMA © 2010 5 Chairman, Vice Chair or Committee Member etc. Although it is a relatively trivial matter through observation to figure out who’s most important, it would be a more daunting task to pick important people out of the general audience. Bibliometrics might solve this problem by counting the number of times a person attends professional society meetings. But clearly this method might erroneously nominate a faithful member who attends every meeting but never truly contributes. Social network analysis takes the guesswork out of the process by looking at all the relationships. LS-SNA makes it easy to identify, compare and rank people. Method Natural questions that might arise for teams responsible for selecting KOLs: How do the two methods compare? (Bibilometrics vs. LS-SNA) How often are highly respected people actually low publishers? Does counting publications find all the key people? This method section outlines the process used to answer the questions above. Data Social network analysis projects covering a variety of research topics (see Table 2) were included in the comparison. Each data set represents 5 years (2005-2009) of publication19 data. Data Preparation Each data set is prepared by cleaning and disambiguating names. Both the problems with synonyms and homonyms are resolved. Homonyms deal with names that are spelled exactly alike; such as “Robert Smith” and “Robert Smith.” If the authors are the same person they are merged into a single master person record. If the authors are different people then two master person records are created. A similar process handles synonyms, names with different spellings may refer to the same person (e.g. “Bob” is a common substitution for the name “Robert”). Network Components Figure 3 Network Creation Each researcher is represented as a node on a graph. A line (also called an edge) is drawn on the graph for each co-authorship relationship. For example if John Smith and Alice Jones have written a research paper together, then a line is drawn between the two to represent their collaboration. See Figure 3. LNX PHARMA © 2010 6 Social network analysis After a graph of all researchers is created, the various centrality measures are calculated20. The process naturally identifies the “core community”21, the largest contiguously connected group of researchers. Ranking The various social network analysis centrality measures are combined into single SNA scores22 which provide an ordinal ranking from top to bottom for each of the researchers. Additionally, each researcher is ranked according to their publication volume. Identifying KOLs Researchers with either a combined SNA score or high publication score in the top 2% of researchers in the disease area are selected for comparison.23, 24 Publication and SNA rank score comparison Categorizing KOLs Each KOL candidate is put into one of three categories: • • • High Publishers with Low SNA – researchers with high publication rank. Low Publishers with High SNA – researchers with low publication rank but having a high SNA score. High Publishers with High SNA – researchers with high publication rank and a high SNA score. Comparison The KOL data is collated and compared. Figure 4 Findings and Discussion Community Size As expected, community size varies by interest area, with some areas attracting larger quantities of researchers (and presumably resource funding) than other areas. For example, “Area D: Soft Tissue Cancer” has more than four times the researchers than “Area C: Insomnia.” Community Cohesion Core community size relates to the community’s cohesiveness; the ability of all researchers to connect to each other through contiguous connections. Communities with limited or focused research doctrines are more cohesive than communities with several research doctrines25. This difference in communities can impact KOL engagement strategy. In very cohesive communities there are fewer differences of opinion and thus it’s harder to overcome “group think,” but once a new idea gets established it spreads through the many connections quickly. Across the eleven areas (see Table 2 – Research areas and disease topics) cohesiveness in the core community varies from 16.2% to 69.4% of the entire community. LNX PHARMA © 2010 7 KOL Comparison - High Publishers versus High SNA Scores Table 1 – Averages for Categorized KOL Candidates26 High Pubs High Pubs Low Pubs Total Low SNA High SNA High SNA All Eleven Areas 692 15.8% 1155 26.4% 2528 57.80% 4375 100% Excluding Vaccine (area K) 691 15.88% 1143 26.27% 2517 57.85% 4351 100% Adopting LS-SNA as Methodology for KOL Discovery Failure to adopt the LS-SNA method would mean companies are missing 50% of the KOLs who could help them most. The possible alternative of loosening the publication criteria could mean an increasing burden of sifting though ever greater numbers of experts who may be missing social prestige. Changing processes from any old method to a newer method requires careful change management. Companies should include publication statistics with LS-SNA projects until they are comfortable with the new method. Fortunately, combining publications counts with the LS-SNA process is negligible additional work and can be easily performed at the same time compared to the LS-SNA process alone. Conclusion Bibliometrics alone fails to find over 50% (57.8%) of the respected experts compared to Large Scale Social Network Analysis. Three categories of experts emerge from community analysis and can be summarized as follows: Insular Experts (High Pubs – Low SNA) : Many publications compared to their peers, yet are lacking the social prestige necessary to be collegially accepted. They may be acknowledged and somewhat respected, but they are not included in meaningful social interactions. About 16% of KOLs fall into this category. Visible KOLs (High Pubs – High SNA): The combination of many publications with many author-to-author interactions create high visibility for these people in all dimensions. These KOLS are often currently engaged and competition for their services fierce. Invisible KOLs (Low Pubs – High SNA): With strong roles (high SNA metrics) within the community but lacking evidence of publications, these individuals are often overlooked by those in key positions as thought leadership team builders. As stated, Invisible KOLs are often “rising stars;” or individuals on sabbatical or semi-retired; or individuals that join or leave an industry; or highly respected KOLs with prestige in a different field. About 58% of what would be the considered most desirable and respected KOLs fall into this category. LNX PHARMA © 2010 8 Companies need outside thought leadership for a number of reasons: Innovation: Infuse company researchers with new ideas (i.e. to avoid group think). Expertise: Use the wisdom of crowds to identify promising research pathways and reduce risks. Strategy: Identify new markets and adjacent markets. Tactical: Help design clinical trial studies. Collaboration: Create opportunities for sharing costs and reducing risks. Education: Quickly train staff when needed (e.g. for a newly licensed product in a disease area unfamiliar to the licensee) Using standard bibliometric KOL identification methods has a very high failure rate. Using this method, about 38%27 of “experts” fail to meet the definition for “well respected.” In other words, about 38% do not appear to have the real social prestige necessary to be considered “well respected experts.” Although the vetting process may filter out these 38%, it will be at great expense, time and effort. However, it is the intangibles such as “starting on the wrong foot” with the wrong KOLs in an advocacy team that may ultimately damage one’s credibility the most, and potentially cause additional costs and increased risks. THEORY: People who are very “productive” in terms of publications and publicity are very connected and meaningful to the community. CONCLUSION: This report shows that not all KOLs are alike, and that high publication volume does not always indicate the highest level of social prestige and respect. Some High Publishers’ ideas may be meaningful, but comprehensive LS-SNA reveals that they don’t always have the relationships within the community necessary to evangelize ideas as easily to peers and colleagues. If someone is an isolated producer, it’s difficult for ideas to flow. Conversely, there are researchers and doctors that companies might dismiss because there is lower evidence of their importance as traditionally measured by publishing counts, but the LS-SNA method finds them highly connected in roles such as advisors and trusted colleagues, or subject matter experts to the community. LNX PHARMA © 2010 9 Research areas and disease topics Research Area Total Unique Researchers Core Researchers Core Community Total Articles Articles for Core Community Article % for Core Community Average authors per Article A Cardiovascular 20,370 9,091 44.6% 7,249 3,864 53.3% 5.26 B Infectious Diseases 32,945 11,837 35.9% 9,244 4,033 43.6% 5.52 C Insomnia 9,211 3,396 36.9% 3,425 1,423 41.5% 4.13 D Soft Tissue Cancer 40,500 13,046 32.2% 10,610 5,252 49.5% 5.53 E Nosocomial Infections 3,128 1,372 43.9% 991 538 54.3% 4.96 F Oncology - Blood 30,242 20,984 69.4% 10,693 6,693 62.6% 6.26 G Oncology - Blood 26,685 14,798 55.5% 7,199 4,257 59.1% 6.56 H Anemia 7,854 2,438 31.0% 2,326 826 62.6% 6.26 I Oncology - Various Products 16,224 5,183 31.9% 4,503 1,591 35.3% 4.97 J Uterine Cancer 11,644 5,352 46.0% 3,462 1,507 43.5% 5.56 K Vaccine 3,703 601 16.2% 1,123 193 17.2% 3.90 202,506 88,098 43.5% 60,825 30,177 49.6% 5.55 Totals & Averages Table 2 The largest two rows from each metric is bolded. Research Area – each area is defined by collecting medical articles28 based on keywords and extracting the co-author relationships Total Unique Researchers – after data preparation and name disambiguation, this represents the total number of unique names Core Researchers – the total number of unique researchers contiguously connected to each other Core Community – the ratio of core researchers to total unique researchers expressed as a percentage Total Articles – the total number of articles collected for the research area Articles for Core Community – the number of articles written by researchers in the giant component Article% for Core Community – the ratio of total articles attributable to the core researchers compared to total articles expressed as a percentage Average Authors per Articles – the ratio of authors (not unique researchers) to the total number of articles LNX PHARMA © 2010 10 Categorized KOL Candidates /Sorted by High Publication subtotal, highest to smallest/ Research Area High Pubs High Pubs Low Pubs Low SNA High SNA High SNA High Pubs High Pubs Low SNA High SNA Subtotal % High SNA Total Low Pubs Total % K Vaccine 1 12 11 24 4.2% 50% 54.2% 45.8% 100% G Oncology - Blood 82 224 313 619 13.2% 36.2% 49.4% 50.6% 100% F Oncology - Blood 143 296 536 975 14.7% 30.4% 45.1% 55% 100% E Nosocomial Infections 10 19 36 65 15.4% 29.2% 44.6% 55.4% 100% H Anemia 42 111 196 349 12% 31.8% 43.8% 56.2% 100% B Infectious Diseases 104 172 366 642 16.2% 26.8% 43% 57% 100% I Oncology - Various Products 54 66 172 292 18.5% 22.6% 41.1% 58.9% 100% D Soft Tissue Cancer 109 115 343 567 19.2% 20.3% 39.5% 60.5% 100% C Insomnia 35 45 123 203 17.2% 22.2% 39.4% 60.6% 100% J Uterine Cancer 67 48 204 319 21% 15% 36% 63.9% 100% A Cardiovascular 45 47 228 320 14.1% 14.7% 28.8% 71.3% 100% 692 1155 2528 4375 15.8% 26.4% 42.2% 57.8% 100% Totals & Averages Table 3 About: Lnx Pharma Lnx Pharma is a division of Lnx Research, LLC, a privately held company based in Orange, California, dedicated to leadership in the analysis of knowledge-creating communities. Lnx Research utilizes proprietary social network analysis methods and technologies to identify and understand “The Invisible College” of key opinion leaders in order to answer questions and build strategies within knowledge communities for government and commercial clients in multiple industries. For white papers on this topic, or more specific information on Lnx Pharma, email [email protected], or see www.lnxpharma.com LNX PHARMA © 2010 11 End Notes DiMasi, J.A., H.G. Grabowski. The Cost of Biopharmaceutical R&D. Managerial and Decision Economics. 28, 469-479 (2007) 2 “Pfizer to pay $2.3 billion to resolve criminal and civil health care liability relating to fraudulent marketing and the payment of kickbacks.” U.S. Department of Health and Human Services and U.S. Department of Justice. Stop Medical Fraud. http://www.stopmedicarefraud.gov/pfizerfactsheet.html 3 Johnson, Carrie. In Settlement, A Warning To Drugmakers. Washington Post (September 3, 2009) 4 Sang-hun, Choe. Disgraced Cloning Expert Convicted in South Korea. The New York Times (October 27, 2009 sec. International/Asia Pacific.) 5 United States Office of the Inspector General - http://www.oig.hhs.gov/authorities/docs/03/050503FRCPGPharmac.pdf Association British Pharmaceutical Industry - Code of Practice http://www.abpi.org.uk/links/assoc/PMCPA/Code06use.pdf European Federation of Pharmaceutical Industries and Associations - http://www.efpia.eu/Content/Default. asp?PageID=559&DocID=3484 6 For purposes of this paper, this definition will be used as the baseline and generally includes Thought Leader, Key Scientific Leader, Clinical Practice Leader and many others. The specific definition varies greatly from one company to another and even within the same company, with some companies creating elaborate definitions to identify and match with specific consulting roles; but most fundamental is the need for a “well respected expert.” 7 Whitepaper #2. Finding Key Opinion Leaders Using Large Scale Social Network Analysis, A Comparative Analysis of Methods for Finding Key Opinion Leaders” http://www.lnxpharma.com/our-story/resources/ 8 Whitepaper #1. Finding Key Opinion Leaders Using Social Network Analysis http://www.lnxpharma.com/our-story/ resources 9 http://en.wikipedia.org/wiki/Bibliometrics 10 http://en.wikipedia.org/wiki/Snowball_sampling 11 Hanneman, Robert A., and Mark Riddle. Introduction to Social Network Methods. University of California, Riverside. (2005) http://www.faculty.ucr.edu/~hanneman/nettext/. 12 Marin, Alexandra, and Barry Wellman. Social Network Analysis: An Introduction. University of Toronto, June 11, 2009. (Forthcoming in Handbook of Social Network Analysis. Edited by Peter Carrington and John Scott. London: Sage, 2010) 13 Freeman, L. C. Centrality in social networks: Conceptual clarification. Social Networks, 1(3), 215-239. (1979) 14 “Large” is a relative term - as improvements in computing power increase, today’s large is tomorrow’s small. At present, for academic researchers,” large generally means 5,000 to 10,000 researchers. For commercial processes, typically “large” means 100,000+ researchers. 15 Miller McPherson, Lynn Smith-Lovin, and James M Cook, BIRDS OF A FEATHER: Homophily in Social Networks. Annual Review of Sociology Vol. 27: 415-444. (2001) 16 Steven N. Durlauf and Lawrence E. Blume (Eds), ‘Social Norms’ in New Palgrave Dictionary of Economics, Second Edition, London: Macmillan (forthcoming). 17 Some of the more common network centrality measures are betweeness centrality, first degree centrality, closeness centrality, and Eigenvector centrality. 18 Gladwell, Malcolm. The Tipping Point: How Little Things Can Make a Big Difference. 1st ed. Little, Brown and Company. (2000) 19 Data source http://www.pubmed.gov 20 David Knoke and Song Yang, Social Network Analysis. vol. 154, 2nd ed., Series: Quantitative Applications in the Social Sciences 07 Sage Publications Ltd. (2008) 21 In SNA terms, the core community is known as the giant component and represents the largest contiguously connected group of people. 22 The SNA score uses the proprietary LnxPharma score, which combines several centrality measures into a single composite number. 23 For both Oncology (Blood) areas and Anemia the top 1% were used for the cutoff. In all cases the cutoff percent used for SNA scores was also the same cutoff value used for high publication scores. 24 In a typical bibliometric search, the cutoff value is gradually increased until the desired number of researchers is obtained. 25 Oxoby, Robert. Understanding Social Inclusion, Social Cohesion and Social Capital. Department of Economics, University of Calgary. (2009) 26 For the eleven areas, some might consider Area K – Vaccine at 24 individuals to be a small sample size; thus the averages were calculated with and without Area K. This had a negligible impact, changing from 57.80% to 57.85%. 27 38% is calculated by of taking the as a percentage “High Pubs+Low SNA” compared to “All High Publishers” 28 http://www.pubmed.gov 1 LNX PHARMA © 2010 12
© Copyright 2026 Paperzz