CME Evaluation – Summary of Results
Prepared by: Ronald Arthur, Senior Manager, Office of CME
American Association for Cancer Research
Accelerating Anticancer Agent Development and Validation Workshop
May 6-8, 2015
Bethesda, MD
Background
This workshop brings together leaders in clinical and translational cancer research from academia, industry, the National Cancer
Institute (NCI), and the U.S. Food and Drug Administration (FDA), to assist investigators in understanding and improving the process
of cancer drug development. The goal is to expedite the development and validation processes for new anticancer and cancer
prevention agents so they can be made available to patients at an accelerated rate. The 2015 workshop was CME accredited for the
fifth time in its twelve year history. The AACR served as the accredited provider and received 96 Requests for Credit. Below is a
history of CME credit participation:
Year
Requests for Credit
2015
96
2014
96
2013
104
2012
81
2011
63
Collaborating organizations for the workshop are:
The U.S. Food and Drug Administration (FDA), the American Association for Cancer Research (AACR), the American Society for
Clinical Oncology (ASCO), Duke University, and the National Cancer Institute (NCI).
The intent of the evaluation of a CME activity is to analyze changes in the learner’s competence, performance and patient outcomes.
The purpose is to evaluate the effectiveness of the identified needs and whether the desired results were achieved.
AACR Office of CME uses the model developed by Donald Moore, PhD in developing the outcome level of the activity. All postactivity evaluations are assessed for the following outcome levels:
Level 1 Participation
Level 2 Satisfaction
Level 3 Learning
Level 4 Competence
Within three months, a follow up Outcomes Survey will be sent to participants who claimed credit, to ask how successful they have
been in making changes in their practice (Level 5 Performance).
Listed on the following pages is an analysis of the results.
1
Target Audience Stratification
Question # 1: Which statement best describes your practice setting?
Which statement best describes your practice setting?
2.1%
Laboratory Research only
8.3%
Clinical Practice only
20.8%
Clinical Practice and
Laboratory Research
55.2%
Administration
Other (please specify)
13.5%
Which statement best describes your practice setting?
Answer Options
Laboratory Research only
Clinical Practice only
Clinical Practice and Laboratory Research
Administration
Other (please specify)
Response Percent
Response Count
2.1%
8.3%
20.8%
13.5%
55.2%
2
8
20
13
53
answered question
96
Other (please specify)
Academic
Advocate in science
Clinical development
Clinical practice and clinical research
Clinical practice and pharma clinical development
Clinical research (biotech)
Clinical research (x4)
Clinical research and regulatory affairs
Clinical research organization
Clinical research pharma industry
Clinical research professional
Clinical trial development
Clinical trials
Consultant clinical research & drug safety
Ethics committee board member
FDA
FDA regulatory
Industry clinical research
Medicines development
Oncology drug development
Patient advocate
Pharma drug development
Pharma industry R&D
Pharmaceutical company (x2)
Pharmaceutical drug development
Pharmaceutical industry (x3)
Pharmaceutical industry clinical R&D
Pharmaceutical industry drug development
Pharmaceutical regulatory affairs
Policy research
Pre-clinical research/clinical trials translation
Regulatory
Regulatory (FDA) and clinical practice
Regulatory affairs (industry)
2
Government (x4)
Industry - global safety
Industry (x5)
Industry clinical research
Regulatory agency
Regulatory science
Translational & early clinical research, industry
Translational medicine, biotech
Analysis: AACR’s CME Mission statement is to provide activities for: basic, translational and clinical physician-scientists and all
other clinicians involved with the care of cancer patients. Many responses under “other” indicate some type of research practice. The
majority of the audience for this activity is conducting research and or in clinical practice which would meet the CME Mission
Statement. The audience also included many commercial interest employees and FDA employees.
Content Areas
Scope of Knowledge Before CME Activity
Please rate your scope of knowledge in Anticancer Agent Development and Validation prior
to participating in this CME activity.
3.29
0.00
0.50
1.00
1.50
Little
Knowledge
2.00
2.50
Some
Knowledge
Answer Options
Knowledge Level
1
5
2
18
3
29
3.00
3.50
Moderate
Knowledge
4
32
5
12
4.00
4.50
Above Average
Knowledge
Rating Average
3.29
5.00
Extensive
Knowledge
Response Count
96
Analysis: CME activities should have an impact and create change for the learner. This question measures knowledge before
the activity.
Scope of Knowledge After CME Activity
Please rate your scope of knowledge in Anticancer Agent Development and Validation after
participating in this CME activity.
4.01
0.00
0.50
1.00
1.50
2.00
Little
Knowledge
Answer Options
2.50
Moderate
Knowledge
Some
Knowledge
1
2
3
3.00
4
3
5
3.50
4.00
Above Average
Knowledge
Rating Average
4.50
5.00
Extensive
Knowledge
Response Count
0
Knowledge Level
4
16
51
25
4.01
96
Analysis: Response before the activity indicates an average of 3.29 and response after the activity indicates an average of 4.01. This
would indicate that the activity was successful in showing change as a result of the CME activity.
Activity included opportunity to learn interactively from faculty/participants?
4.48
0.00
0.50
1.00
1.50
Strongly
Answer Options
Please rate
Strongly Agree
49
2.00
2.50
Disagree
Agree
44
Neutral
3
Disagree
0
3.00
3.50
4.00
4.50
Neutral
Agree
Strongly Disagree
0
Rating Average
4.48
5.00
Strongly
Response Count
96
Analysis: This question measures satisfaction or the degree to which the expectations of the learner were met. Results indicate strong
satisfaction with networking and collaboration opportunities.
Content covered was useful and relevant to my practice?
4.55
0.00
0.50
1.00
1.50
Strongly
Answer Options
Please rate
Strongly Agree
55
2.00
2.50
Disagree
Agree
39
Neutral
2
3.00
3.50
Neutral
Disagree
0
Strongly Disagree
0
4.00
4.50
Agree
Rating Average
4.55
5.00
Strongly
Response Count
96
Analysis: This question measures satisfaction or the degree to which the expectations of the learner were met. The results indicate
strong satisfaction with the educational content.
4
Average Score
To what extent were the learning objectives met by this CME activity?
Assess the key decision points in determining if
an anticancer agent or combination has clinical
benefit.
4.29
Analyze and more effectively follow FDA safety
and efficacy guidelines during the development
and clinical testing of anticancer and cancer
prevention drugs, and in the long term safety
evaluation of these.
4.41
Design, and participate in oncology clinical trials
with meaningful endpoints and accrual goals.
4.36
1.00
Strongly
Disagree
2.00
3.00
Disagree
Neutral
4.00
5.00
Agree
Strongly
Agree
Analysis: For the learning objectives, the majority of respondents indicated agreement that the learning objectives were met.
The speakers presented sufficient scientific evidence to support the content presented?
4.42
0.00
0.50
1.00
1.50
Strongly
Answer options
Please rate
2.00
2.50
Disagree
3.00
3.50
Neutral
4.00
4.50
5.00
Agree
Strongly
Strongly Agree
Agree
Neutral
Disagree
Strongly Disagree
Rating Average
Response Count
42
52
2
0
0
4.42
96
5
Analysis: Content of CME activities should be based on the best scientific evidence. The responses indicate the speakers met this
requirement.
Meeting Presentations were free from commercial influence or bias?
4.51
0.00
0.50
1.00
1.50
Strongly
Answer Options
Please rate
Strongly Agree
52
2.00
2.50
Disagree
Agree
41
Neutral
3
3.00
3.50
4.50
5.00
Strongly
Agree
Neutral
Disagree
0
4.00
Strongly Disagree
0
Rating Average
4.51
Response Count
96
Analysis: Content of CME activities should be free from commercial influence or bias. The response rating of 4.51 indicates that the
activity met this requirement. Further, many speakers were commercial interest employees and the response confirms that the content
did not relate to the employers’ products and services, as is required to be in compliance with the ACCME Standards for Commercial
Support.
Disclosure of relevant financial relationships were routinely provided?
4.64
0.00
0.50
1.00
1.50
Strongly Agree
61
2.50
Disagree
Strongly
Answer Options
Please rate
2.00
Agree
35
Neutral
0
Disagree
0
3.00
3.50
4.00
4.50
5.00
Neutral
Agree
Strongly
Strongly Disagree
0
Rating Average
4.64
Response Count
96
Analysis: The ACCME Standards of Commercial Support require disclosure to the learners before the activity. All disclosures were
included in the program information and emailed to learners, to meet the disclosure requirement. In addition, speakers are also
requested to disclose before speaking, to assist the learners in their evaluation of the activity. The evaluation response and session
monitoring indicate overall compliance.
6
Based on my participation in the meeting, I am more confident in my ability to care for my patients?
4.16
0.00
0.50
1.00
1.50
Strongly
Answer
Options
Please rate
2.00
2.50
3.00
4.00
Strongly Agree
Agree
Neutral
Disagree
Strongly Disagree
N/A
22
32
9
0
1
32
4.50
Strongly
Agree
Neutral
Disagree
3.50
Rating
Average
4.16
Response
Count
96
Analysis: Learners who are confident after participation in the CME activity are more likely to be successful in making changes in
their practice. For the learners to whom this question applied, 84% felt more confident in their ability after participation in the activity.
Please indicate any barriers you perceive in implementing changes to your practice. (check all that apply)
No barriers
36.5%
Cost
22.9%
Other (please specify)
17.7%
Reimbursement/insurance issues
16.7%
Guidelines
10.4%
Lack of opportunity (patients)
10.4%
Lack of time to assess/counsel patients
6.3%
Lack of administrative support
4.2%
Lack of experience
4.2%
Patient compliance issues
3.1%
Lack of resources (equipment)
3.1%
0.0%
Complexity of trial design.
5.0%
10.0%
15.0%
20.0%
Other (please specify)
Lack of diversity.
7
25.0%
30.0%
35.0%
40.0%
Earlier interaction with HA to define trials.
Inappropriate implementing of regulation in China.
Not currently in clinical practice.
Pediatrics, so limitation to access.
Analysis: This question is helpful in planning future activities, to assist the learners to overcome these barriers.
Please suggest other topics and/or speakers that you would like to see included in future AACR CME activities.
Responses
A session on practicing clinical trial design might be helpful.
Add case studies of recent expedited drug development; realistic forecast of clinical trial design for new effective drugs.
Excellent!
Great course for someone who is trying to come up to speed.
Great workshop.
Great workshop and especially the case studies.
Great workshop!
I believe the content of the sessions was very useful, particularly the case studies highlighting specific challenges and
how they were overcome.
I feel this type of a workshop should be offered on a continual basis, to capture the rapidly changing "landscape" of
cancer research and the need for much faster practical advances in development to make new promising anti-cancer
agents available to the suffering patients as soon as possible, without sacrificing the safety issues.
I found this workshop very useful. Especially the case presentations were great. I would appreciate perhaps more
hands on training on how to really conduct biomarker development within a clinical trial/development program.
I would appreciate if more information about presentations would be available online after the meeting (posters,
presentations, recordings).
I would like to thank the organizers for this excellent meeting. It was really a pleasure joining the sessions where FDA
and the Industry presented the cases.
I'm wondering about adding learning opportunities regarding contracting for clinical trials and how to improve that
process. Perhaps this is not the forum. Also, discussion of the challenges in community versus academic settings in
bringing novel agents to patients.
My knowledge was enhanced by participating in this workshop. I can't state that it is extensive as I am not an expert
but it added tremendously to my above average knowledge.
Sessions on immunotherapy were excellent. As this field is widely expanding, we will need updates at the conference
next year. Also, it was very helpful to have some sessions tracked for those with only basic knowledge vs. more
experience -- keep this up. I would recommend a bit more rigor for the case studies and a more organized approach so
that key take home messages from each phase of development are very very clear. It was hard to capture these points.
for example, for each presentation, for each phase of development, the presentation should be structured: a) "goal"
[what company wanted to accomplish, e.g. new formulation; identify a relevant biomarker, faster enrollment etc.]; b)
"specific hurdles" that needed to be overcome; c) how these challenges were met; and d) a section "if we knew then
what we know now, we would have developed the drug this way."
Superb meeting. Sessions were well gestated and expertly presented. Intro was a bit long.
The workshop was very useful and instrumental for understanding the pitfalls and complexity of combination protocols
and importance of FDA involvement throughout the process.
This is an excellent program and perhaps one of the best learning experiences for those working to develop new
therapies.
This was my first attendance at this workshop and the combination of formal sessions and time for interaction outside
the sessions was a great mix.
Those of us who are considered minorities don't see folks who look like us as presenters and it is sad. A whole hour
should be dedicated to strategies for accrual and retention of minorities in clinical trials with presenters being the
advocates representing these groups.
Very useful and relevant workshop - an opportunity to interact with all relevant disciplines.
Would advocate for a session on locally advanced disease that includes radiation combination strategies.
8
9