PREDICTIVE
DATA-DRIVEN
MONITORING:
WHAT’S IN IT
FOR YOU?
Drew Garty
CONFIDENTIAL
© 2015 PAREXEL INTERNATIONAL CORP.
HUMAN ABILITY TO PREDICT:
“You can’t predict the future.”
Natural Numbers: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13
__
__
Odd Numbers: 1, 3, 5, 7, 9, 11, 13
Prime Numbers: 2, 3, 5, 7, 11, 13
__
Fibonacci Sequence: 1, 1, 2, 3, 5, 8, 13
__
Or can you?
© 2015 PAREXEL INTERNATIONAL CORP. / 2
CONFIDENTIAL
PREDICTIVE ANALYTICS DEFINED
Competitive Advantage
Predictive analytics is the process of identifying patterns harnessing
existing data, knowledge, what-if scenarios, and risk assessments to
predict future outcomes and trends with an acceptable level of reliability.
Optimization
Predict and Adapt
Predictive
Modeling
Sense & Response
Raw
Data
Standard
Cleaned Reports
Data
Generic
Predictive
Analytics
Ad hoc
Reports
What is the best outcome?
What is likely to happen?
Why did it happen?
What happened?
Analytics Maturity
© 2015 PAREXEL INTERNATIONAL CORP. / 3
CONFIDENTIAL
ADAPTIVE MONITORING DEFINED
Tools
Monitoring Tools
An adaptive
approach to clinical
trial monitoring that
directs monitoring
focus and activities to
the evolving areas of
greatest need which
have the most
potential to impact
subject safety and
data quality.*
Process
Execute
Site Monitoring
Team
Monitors
(CRA/CMA)
Distribute
Change
Study
Leadership
Recommendation
and Rationale
Decide on
and Approve
Change
Data
Review
Design &
Data
Clinical
Surveillance
Data
team
Interrogation
Identify and
Propose
Change
Reporting and
Analytics
Process
*Definition taken from :TransCelerate Biopharma.--Position Paper: Risk-Based Monitoring Methodology (pgs. 14-15)
http://www.transceleratebiopharmainc.com/wp-content/uploads/2013/10/TransCelerate-RBM-Position-Paper-FINAL-30MAY2013.pdf
© 2015 PAREXEL INTERNATIONAL CORP. / 4
CONFIDENTIAL
PREDICTIVE ANALYTICS
VALUE CREATION FOR ADAPTIVE MONITORING
Can you predict which sites are likely to experience higher SAE
rates, rates of significant protocol deviations, GCP non-compliance,
and delays in data currency….across a specific protocol, program,
region, compound?
Can you predict which risk avoidance and risk control strategies
implemented by protocol teams most effectively balance cost and
quality in the management of site and project team behaviors that
consume site/data monitoring resources…. thus enabling stable
forecast/allocation of resourcing in a global RBM context?
Before we can predict what will happen, we have to understand what
happened…and why it happened
© 2015 PAREXEL INTERNATIONAL CORP. / 5
CONFIDENTIAL
WHAT’S HAPPENING AT SITE?
MEASURING RISK AND WORKLOAD
© 2015 PAREXEL INTERNATIONAL CORP. / 6
CONFIDENTIAL
HOW EFFECTIVE ARE MONITORING INTERVENTIONS?
ANALYZING RISK AND MONITORING ACTIONS OVER TIME
Monitoring Activity
© 2015 PAREXEL INTERNATIONAL CORP. / 7
CONFIDENTIAL
SMART TRIALS REQUIRE BETTER DECISIONS
The relationship between risk and related mitigation costs is critical to
simultaneously increase quality and patient safety while reducing waste and
operational cost.
Cost
Quality
Cost of Quality
Results of work efforts
=
Total costs
The use of predictive analytics to forecast resource consumption in the
adaptive monitoring space is a critical objective for RBM stakeholders
seeking to get ahead of the outcomes-based reimbursement curve.
© 2015 PAREXEL INTERNATIONAL CORP. / 8
CONFIDENTIAL
SMART MONITORING
Risk & Quality
Management
Cost
Quality
Data
Surveillance
© 2015 PAREXEL INTERNATIONAL CORP. / 9
Resource
Management
Study Design
Data-Driven
Monitoring
CONFIDENTIAL
Innovation &
Optimization
MANAGING RESOURCING, COSTS AND EFFICIENCY
• Measuring and calculating effort and cost
similar to risk
• Includes resource requirements for risk
mitigation and other monitoring activities
• Transparency into trends for resource
forecasting and planning
© 2015 PAREXEL INTERNATIONAL CORP. / 10
CONFIDENTIAL
UNDERSTANDING RESOURCING, COSTS AND
EFFICIENCY
Moving the needle
• Measuring work performance and
effectiveness of monitoring
interventions in reducing risk
• Analyzing by:
• Site
• CRA
• Country
• Study
• Organization
© 2015 PAREXEL INTERNATIONAL CORP. / 11
CONFIDENTIAL
ANALYTIC INSIGHTS: RESOURCE MANAGEMENT
© 2015 PAREXEL INTERNATIONAL CORP. / 12
CONFIDENTIAL
12
A LEARNING ADAPTIVE MONITORING SYSTEM
FUNCTIONALITY
• A “learning” adaptive monitoring solution is one that readily
incorporates lessons from its own activities in real time and applies
that knowledge to drive the application of the most effective and
efficient RBM interventions
• An adaptive monitoring solution must incorporate cumulative
knowledge to predict.
• Adaptive monitoring solution must be able trace the effectiveness
of actions taken in achieving a desired outcome (cost of quality)
A learning system is a predecessor to the use of predictive analytics
in risk-based monitoring
© 2015 PAREXEL INTERNATIONAL CORP. / 13
CONFIDENTIAL
BUILDING AND LEVERAGING CUMULATIVE KNOWLEDGE
Analysis
Design
Execution
Feasibility
Planning &
Setup
© 2015 PAREXEL INTERNATIONAL CORP. / 14
CONFIDENTIAL
Risk
PREDICTING RBM RESOURCE DEMAND
Business Objective:
Determine and predict which risk mitigation methods implemented by clinical monitoring
teams are and will be most effective and efficient.
What-if Scenarios:
• Type of Intervention
• Frequency of Intervention
• Resource (role, level, expertise)
• Therapeutic Area
• Region/Country/Cultural factors
• Site personnel motivators
© 2015 PAREXEL INTERNATIONAL CORP. / 15
CONFIDENTIAL
EXAMPLE STUDY
Sample Summary:
• Respiratory Study (51 enrolling sites; ~400 patients)
• 12 months worth of site risk and monitoring workload data containing:
• ~ 750 Monitoring intervention records (onsite visits, telephone contacts, and emails)
• ~ 37,000 Calculated site risk score records
• ~ 35,000 Monitoring workload calculations
© 2015 PAREXEL INTERNATIONAL CORP. / 16
CONFIDENTIAL
BUILDING A RELIABLE MODEL
Of various models tested, one model was a good fit. Further testing and
analysis continues…..
•
11 sites didn’t fit the model requiring further
investigation
•
© 2015 PAREXEL INTERNATIONAL CORP. / 17
2 sites were significantly influential. Analysis
performed without influential data to confirm
acceptability
CONFIDENTIAL
ANALYSIS
Analysis of bivariate cohesiveness between risk and monitoring
Intervention:
Repeated measurement (in time) models to investigate the effects on site risk:
•
Site workload values over course of study
•
Total number of onsite monitoring visits over
course of study
•
Total number of emails over course of study
•
Total number of calls over course of study
With the unknown assumptions of covariance
structures, multiple types of covariance
structures were attempted (Reference: Kincaid C.,
Guidelines for Selecting the Covariance Structure in
Mixed, SUGI Paper 198-30)
© 2015 PAREXEL INTERNATIONAL CORP. / 18
CONFIDENTIAL
OBSERVATIONS
• There is a strong correlation between:
o Risk and monitoring workload
o Risk and the total numbers of monitoring interventions
o Risk and the type of monitoring interventions taken
− Emails to site are ineffectual means of reducing risk
• An onsite visit is 18X more effective than phone call in reducing risk
scores driven by site characteristics with high workload values
− Protocol deviations, SAEs, open issues, outstanding queries
• Verification that the most effective visit frequency is X in managing
risk unless outstanding workload is above y
© 2015 PAREXEL INTERNATIONAL CORP. / 19
CONFIDENTIAL
WHY IT MATTERS
Flexible monitoring resourcing models presumed by contemporary RBM
methodology hampered by material constraints
• Languages, time zones, site schedules, protocol-specific training requirements
Use of a study’s data/risk assessment alone to drive monitoring
interventions misses opportunities for value creation that accompany
standardization based on prediction
•
Across high volumes of RBM studies and large monitoring forces
Enhances reliability of labor forecasts, resource availability and ability to
commit to fixed volumes of monitoring units
Monitoring plans that initially prescribe monitoring interventions/triggers
and their intervals based on predictive models:
• Improves monitor compliance
• Optimizes the ‘cost of quality’
© 2015 PAREXEL INTERNATIONAL CORP. / 20
CONFIDENTIAL
WHEN A SITE DOESN’T FIT THE PREDICTIVE MODEL:
RISK OVERRIDE AND SITE-SPECIFIC WORKLOAD TUNING
Site Risk Grade Override
Site Specific Workload Fine Tuning
Site Risk Grade Override
© 2015 PAREXEL INTERNATIONAL CORP. / 21
CONFIDENTIAL
THANK YOU
© 2015 PAREXEL INTERNATIONAL CORP. / 22
CONFIDENTIAL