Artificial Intelligence in
Clinical Trial Disclosure and
Transparency
January 28, 2016
3 Key Topics
1.
2.
3.
4.
Mitigating RISK
Silo vs. Collective Learning
Technology Assisting
Humans vs. Humans
Assisting Technology
Artificial Intelligence as an
Enterprise Solution
2
© Copyright 2015 Certara, L.P. All rights reserved.
What are we really talking about?
RISK
3
© Copyright 2015 Certara, L.P. All rights reserved.
Transparency and Disclosure
Policy70
Vendor 1
Anonymization
Report
Lay Summaries
In House
Group B
Vendor 2
AI
Enterprise
Solution
T&D
Registry
Postings
Certification
Vendor 4
Vendor 3
Policy 43
In House
Group A
4
© Copyright 2015 Certara, L.P. All rights reserved.
What is AI and NLP?
Silo Learning
vs
Collective Learning
© Copyright 2015 Certara, L.P. All rights reserved.
We learn in silos
New Learning
6
© Copyright 2015 Certara, L.P. All rights reserved.
The risk of silo learning
EXAMPLE:
In 2016 your company has 5 MAA submissions
planned.
The total documents count needed per Policy
70 is 100 documents (including CSRs, Module 2
Summaries, Module 5 Docs, etc.)
© Copyright 2015 Certara, L.P. All rights reserved.
The risk of silo learning
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
© Copyright 2015 Certara, L.P. All rights reserved.
Learning in a silo
5
*
Learned Redaction
Previously Unknown
© Copyright 2015 Certara, L.P. All rights reserved.
We learn in silos
New Learning
10
© Copyright 2015 Certara, L.P. All rights reserved.
The risk of silo learning
5
5
*
*
5
5
*
© Copyright 2015 Certara, L.P. All rights reserved.
*
The same finding
appears in other reports
within the 100, that others
will not learn.
5
*
The key is collective learning
AI Engine
Define AI and NLP
© Copyright 2015 Certara, L.P. All rights reserved.
What is AI and NLP?
•Natural language processing (NLP) is a field of
computer science, artificial intelligence, and
linguistics concerned with the interactions between
computers and human (natural) languages. As such, NLP
is related to the area of human-computer interaction.
Many challenges in NLP involve natural language
understanding, that is, enabling computers to derive
meaning from human or natural language input, and others
involve natural language generation natural construct an
automated process for developing new documents based
on pre-defined process rules.
© Copyright 2015 Certara, L.P. All rights reserved.
Technology Background
14
© Copyright 2015 Certara, L.P. All rights reserved.
Technology Background
What Does Siri Really do?
15
© Copyright 2015 Certara, L.P. All rights reserved.
What is AI and NLP?
NLP and NLR: Allow for a computer to understand
individual words (textual).
AI + NLP and NLR: Allows a computer to
understand not only the individual words, but the
relation ship of those words to one another
=
CONTEXT
© Copyright 2015 Certara, L.P. All rights reserved.
NLP and AI Working Together
Context is key…
Incase of emergency contact:
812654739929830
17
© Copyright 2015 Certara, L.P. All rights reserved.
What is AI and NLP?
Show examples of NLP
and AI working, numbers
that are a telephone . . .
Do a second one of
Subject ID number
© Copyright 2015 Certara, L.P. All rights reserved.
Tha value of collective learning using AI?
Show examples of NLP
and AI working, numbers
that are a telephone . . .
Do a second one of
Subject ID number
© Copyright 2015 Certara, L.P. All rights reserved.
Tha value of collective learning using AI?
All well and good, but a computer cannot
know everything, it too has to learn, correct?
Yes.
© Copyright 2015 Certara, L.P. All rights reserved.
What is AI and NLP?
Technology
Assisting
Humans
Humans
Assisting
Technology
© Copyright 2015 Certara, L.P. All rights reserved.
Redaction
What Happens When Technology Does Not Understand You?
22
© Copyright 2015 Certara, L.P. All rights reserved.
Redaction Process
Ongoing Collective Learning an Integral Part of the Process
• Create New Redaction Study – Redaction User
• Verify Redaction Documents – ClinGenuity
•
•
•
•
Redaction Management System (CRMS)
Redaction Identification - CRMS
Approve/Deny Redactions – ClinGenuity Quality
Assurance Associate
Publish Documents – CRMS
Final Review - ClinGenuity Quality Assurance
Associate
23
© Copyright 2015 Certara, L.P. All rights reserved.
Redaction Process
Create New Redaction Study
• Redaction User signs
•
•
on to the ClinGenuity
Web Portal
The User assigns a
new study for
Redaction
After the study is
created, the User is
prompted to upload
all study documents
to be redacted
New
Study
Redaction
User
ClinGenuity
Web Portal
ClinGenuity
Database
Study
To Redact
Client Document
Management
System
24
© Copyright 2015 Certara, L.P. All rights reserved.
Redaction Process
Verify Redaction Documents
•
•
•
•
The CRMS assigns the
Redaction Study to the
ClinGenuity Redaction
Processor
The ClinGenuity
Redaction Processor
analyzes all of the
uploaded documents for
the study
Study documents are
verified by the
ClinGenuity QA Associate
After all documents have
been verified the study is
approved to move to the
next queue step
Study
Documents
Redaction
Study
Verified
Documents
Approval
CRMS
ClinGenuity
Database
ClinGenuity
Redaction
Processor
Corrections/
Approval
Next
Queue
Step
Verification
Notification
ClinGenuity
QA
Assocaite
25
© Copyright 2015 Certara, L.P. All rights reserved.
Redaction Process
Identify Redactions
•
•
•
After the study’s
documents have been
verified, the CRMS
assigns the study to the
next step in the redaction
process
The ClinGenuity
Redaction Processor
creates the new output
documents and identifies
all redactions for each
document*
After all documents have
been processed, a
notification is sent to the
ClinGenuity QA Associate
indicating the Study is
ready for review
Previous
Queue
Step
Study
Documents
Marked up
Redaction
Documents
Redaction
Study
CRMS
Redactions
ClinGenuity
Redaction
Processor
ClinGenuity
Database
Review
Notification
ClinGenuity
QA
Associate
*No PII is stored in the ClinGenuity Database just the location of the annotation where
PII is removed and AI information associated with the redaction
26
© Copyright 2015 Certara, L.P. All rights reserved.
Redaction Process
Approve/Deny/Add Redactions
•
•
•
After the study is in a review
state, the ClinGenuity QA
Associate reviews it using the
ClinGenuity Redaction
Reviewer Application
The ClinGenuity Redaction
Reviewer allows the
ClinGenuity QA Associate to
Approve, Deny or Add new
redactions to the study’s
documents
Once all redactions have
been applied, the ClinGenuity
QA Associate approves the
redactions for publishing and
the ClinGenuity Redaction
Reviewer notifies the CRMS
to go to the next queue step
Publish
Approval
ClinGenuity
QA
Associate
Redactions
Publish
Approval
CRMS
Updates
ClinGenuity
Redaction
Reviewer
ClinGenuity
Database
Next
Queue
Step
Marked up
Redaction
Documents
27
© Copyright 2015 Certara, L.P. All rights reserved.
Redaction Process
Publish Documents
•
•
•
When the study is ready to
be published the CRMS
notifies the ClinGenuity
Redaction Processor to
publish the redacted
documents
The ClinGenuity Redaction
Processor generates all of
the redaction documents for
the study
After the documents have
been published, a
notification is sent to the
ClinGenuity QA Associate
informing them the study is
ready for final review
Previous
Queue
Step
Marked up
Redaction
Documents
Redaction
Study
CRMS
ClinGenuity
QA
Associate
Redactions
ClinGenuity
Redaction
Processor
Study Final Review
Notification
ClinGenuity
Database
Redacted
Documents
28
© Copyright 2015 Certara, L.P. All rights reserved.
Redaction Process
Final Review
•
•
•
Following the publishing of
the redacted documents,
the ClinGenuity QA
Associate reviews them
using the ClinGenuity
Redaction Reviewer
Application
After all documents have
been reviewed the
ClinGenuity QA Associate
provides the final approval
for the study
Once the study has been
finalized the ClinGenuity
Redaction Reviewer sends
a study complete
notification to the
redaction user
Final
Approval
Redacted
Documents
ClinGenuity
QA
Associate
Final
Approval
ClinGenuity
Redaction
Reviewer
Redaction
User
ClinGenuity
Database
Study Complete
Notification
29
© Copyright 2015 Certara, L.P. All rights reserved.
The Power of Collective Learning
Humans Assisting Technology
•
Human-aided QC - Teaches the AI –
Engine Gets Smarter Over Time.
•
Every Single Redaction Has to be
Approved or Rejected
•
Over 500,000,000 Redactions Performed
to Date
•
Over 5 Million Pages Redacted per Year
•
Over 100,000 Pages Redacted Every
Week.
30
© Copyright 2015 Certara, L.P. All rights reserved.
Ease of Implementation
Extremely Fast and Easy Implementation
o
Default configuration is TransCelerate compliant out-of-the-box
o
Custom configuration is available
o
o
Review transparency guidance along with company confidentiality policies
o
Determine what information is to be retained and what is to be redacted.
o
ClinGenuity™ AI engine then uses your requirements to configure the system
accordingly.
Custom Process as quick as 2 weeks to full implementation
31
© Copyright 2015 Certara, L.P. All rights reserved.
Enterprise Solution
AI as an
Enterprise Solution
© Copyright 2015 Certara, L.P. All rights reserved.
Enterprise Solution
o
Our AI Engine can Automate the development of other T&D
Offerings Including but Not Limited to:
o Lay Summaries
o CT.Gov Postings
o EUDRA
o Anonymization Reports
o Justification Tables
o And numerous other regulatory documents
not part of T&D.
© Copyright 2015 Certara, L.P. All rights reserved.
Lay Summaries
Lay Summaries
o
o
o
o
o
Exclusive Partnership with CISCRP
Complete offering – From Authoring to Distribution
The Only Clinical AI Lay Summary Offering in the Market
Automate he Authoring of Patient Lay Summaries Using:
o ICF
o Protocol
o CSR Synopsis
Full Service – Scalable Solution
o Consistent
o High Quality
o Efficient
34
© Copyright 2015 Certara, L.P. All rights reserved.
Lay Summaries
Removed Demo Video of Automated
Lay Summary Production Due to
Size
35
© Copyright 2015 Certara, L.P. All rights reserved.
Lay Summaries
Lay Summaries
36
© Copyright 2015 Certara, L.P. All rights reserved.
Lay Summaries
What is Your Biggest Risk with Lay Summaries?
Promotional
Language
37
© Copyright 2015 Certara, L.P. All rights reserved.
Lay Summaries
Lay Summaries
Value in AI automation for Lay Summaries is
NOT merely the speed and efficiency, the real
value is in risk mitigation.
38
© Copyright 2015 Certara, L.P. All rights reserved.
Questions?
39
© Copyright 2015 Certara, L.P. All rights reserved.