REVIEW PROTOCOL TEMPLATE
Recommendation 1
The effectiveness of health professionals’ education and training
institutions designing and implementing continuous development
programmes for faculty and teaching staff relevant to the evolving healthcare needs of their communities.
A systematic review
Review protocol
Organization, City, Country:
Prepared by:
Date:
Senior supervisor
Project lead
Research team members
University of Cambridge, Cambridge, United Kingdom
Alison Andrew
July 2011
Francesca Celletti
Rebecca Bailey
Rachel Marie Amiya, Dolea Carmen Mihaela, Annette
Mwansa Nkowane, Erica Lynette Wheeler, Rodrigo
Rodriguez Fernandez.
1/12
Table of Contents
................................................................................................................................................................ 1
TO INVESTIGATE THE EFFECTIVENESS OF CONTINUOUS DEVELOPMENT PROGRAMMES FOR FACULTY
AND TEACHING STAFF, INCLUDING CURRENT HEALTH WORKERS, WHICH UPDATE AND DEVELOP
TEACHING SKILLS (E.G. CURRICULUM DEVELOPMENT AND INSTRUCTIONAL DESIGN), IN BOTH
UNDERGRADUATE AND POSTGRADUATE PROGRAMS, ON THE QUANTITY, QUALITY AND RELEVANCE OF
HEALTH PROFESSIONALS.......................................................................................................................... 1
A SYSTEMATIC REVIEW ............................................................................................................................ 1
1.0Background .................................................................................................................4
2.0Objective .....................................................................................................................4
3.0Review Question .........................................................................................................4
4.0Evidence gathering and study selection ......................................................................5
5.0Assessment of risk of bias and data extraction ...........................................................7
6.0Data synthesis ............................................................................................................8
7.0Dissemination .............................................................................................................8
8.0Resource implications .................................................................................................8
9.0References ...................................................................................................................8
Appendix A: database search strategy ...........................................................................10
Appendix B: data extraction forms (see attached excel workbook) ..............................12
Appendix C: Example of databases, websites and journals which could be searched ..12
2/12
1.0 Background
Include definitions, answers to background questions, description of the intervention and how it might
work to help better understand the question.
This question refers to the effectiveness of continuous development programmes for
faculty and teaching staff which are focused on which updating and developing teaching
skills. Such interventions could be in the form of one off workshops, seminars or short
courses or continuing programmes such as fellowship schemes; all aimed at improving
teaching ability. There have been systematic literature reviews that have analysed the
effectiveness of such interventions (most notably Steinert et al. 2006 reviewed the
effectiveness of such interventions in medical education) yet the primary issue lies in the
outcomes measured. Few studies have measured the effect of such programmes past
the level of participants’ reaction to the programme. Studies that measured further
outcomes (learning in the form of a change in attitudes, knowledge or skills, a change in
behavior or results in the form of a change in the system/ organizational practice or a
changes in learning or performance of health students) were those included in Steinert's
systematic review. However, since the outcomes of interest in this systematic review
are the quantity, quality and relevance of health graduates and health practitioners the
only outcomes relevant to this literature review are those measured are changes in the
quantity, quality and relevance of health graduates or professionals that can be
specifically attributed to such continuous development programmes for faculty and
teaching staff. Other process outcomes will be recorded but are not relevant for the
purpose of evaluation of evidence.
2.0 Objective
To investigate the effectiveness of Continuous Development Programmes for faculty and
teaching staff, including current health workers, which update and develop teaching
skills (e.g. curriculum development and instructional design), in both undergraduate and
postgraduate programs, on the quantity, quality and relevance of health professionals.
3.0 Review Question
For the purposes of this literature review, the population, intervention, comparators
and outcomes (PICO) framework to inform the review objectives are presented below.
Population
Intervention
Comparison
Outcome
Faculty and
Continuous
No Continuous
Quantity, quality
teaching staff
Development
Development
and relevance of
programs
Programmes for
Programmes for
health professionals
faculty and teaching faculty and teaching
staff which update
staff which update
and develop
and develop
teaching skills
teaching skills
3/12
4.0 Evidence gathering and study selection
See Appendix C for examples of databases, websites and journals which may be
searched.
4.1 Evidence gathering
The evidence gathering approach will have four components:
4.1.1 Searching databases
The databases in the table below will be searched with a pre-determined strategy as
detailed in Appendix A. In cases where the search results are small in number, search
terms will be reduced to maximize the search sensitivity.
Topic/Field
Ex: Education
Health
Database
Ex: ERIC, Campbell, Education Research Complete…
PUBMED
4.1.2 Hand searching
The following journals and websites will be hand-searched for relevant articles:
Resources that will be searched by hand
• Teaching and Teacher Education
• Academic Medicine
• Medical Teacher
• International Journal for Academic Development
• Biomed (http://www.biomedcentral.com/)
 WHO's Library Database (WHOLIS)
 Website of the Foundation of Advancement of International Medical Education
and Research (FAIMER)
 International Council of Nurses

Council on Graduate Medical Education

American Academy of Family Physicians

American College of Physicians
4/12
4.1.3 Expert network consultations
A network of health professional education experts will be consulted by email to identify
additional grey literature or research that has not been found through the above
processes. The following experts and networks will be contacted.
Experts and networks will be contacted regarding additional articles
Consultation with relevant members of the Core Guidelines Development Group
4.1.4 Reference searches
Bibliographies of those papers that match the eligibility criteria below will be searched
by hand to identify any further, relevant references, which will be subject to the same
screening and selection process.
4.2 Eligibility criteria
After gathering the evidence, the following eligibility criteria will be applied to the
results and all identified references screened independently by two reviewers (Aaron
Stoertz and Francesca Celletti) using a three-stage approach to reviewing the title,
abstract and full text.
4.2.1 Types of studies:
All types of evaluative study designs are eligible for inclusion, including grey literature.
Studies will not be selected on methodological quality.
4.2.2 Types of participants:
This is a succinct description of the types of participants that you will include. For example, you might
copy the health professionals definition here.
The participants this literature review will include are faculty and teaching staff
including current health workers and/or residents or interns who also teach students
from any of the health professions listed below.
To facilitate the development of the WHO guidelines for transforming and scaling-up
health professional education and training, the literature review will include the
following lists of health professionals, as defined by the International Standard
Classification of Occupations (ISCO) (WHO, 2010a; ILO, 2008):
Core List


Medical Doctors - both Generalist and Specialist Practitioners, including Public
Health Doctors (ISCO-08 minor group 221)
Nursing Professionals, including Public Health Nurses (ISCO-08 minor group 222, unit
2221)
5/12



Midwifery Professionals, including Public Health Midwives (ISCO-08 minor group
222, unit 2222)
Dentists (ISCO-08 minor group 226, unit 2261)
Pharmacists (ISCO-08 minor group 226, unit 2262)
4.2.3 Types of intervention:
This is a succinct description of the types of intervention you will include.
Continuous Development Programmes for faculty and teaching staff which update and
develop teaching skills. There is a wide variety of programmes that qualify as such e.g.
workshops, seminars, short courses, fellowship schemes.
Articles that focus on programmes that develop and enhance teaching skills and
educational methods (including PBL) not programmes that develop teaching content.
4.2.4 Types of outcome measures:
The primary outcomes of interest are the quantity, quality and relevance of practicing
health professionals. These are defined by a number of measurable outcomes found in
the Outcomes Framework document. Other important outcomes include values and
preferences, resource use/costs, benefits and harms, as well as all other unintended
effects of the intervention. Studies that include other outcomes should not be excluded
at this stage in the evidence retrieval. Other secondary outcomes can also be defined, as needed.
4.3 Exclusion criteria
Editorials, newspaper articles and other forms of popular media will be excluded.
Failure to meet any one of the above eligibility criteria (section 3.2) will result in
exclusion from the review and any apparent discrepancies during the selection process
will be resolved by a third, independent reviewer. The number of excluded studies
(including reasons for exclusion for those excluded following review of the full text) will
be recorded at each stage.
5.0 Assessment of risk of bias and data extraction
Following the initial selection of literature, the reviewers will consult with the WHO
Secretariat regarding the process to be followed for summarizing the studies. Data will
be extracted from relevant papers using predefined evidence summary templates
attached in Appendix B. Data will be collected regarding the reasons for exclusion,
6/12
characteristics of included studies, participants, interventions (including comparators)
and outcomes. The final decision for inclusion or exclusion will be made by a team
consisting of the WHO Secretariat, methodologist and researchers conducting the
review. Any potential disagreement will be recorded and resolved by further discussion.
Risk of bias across studies will be assessed using the approach outlined by the Grading
of Recommendations Assessment Development and Evaluation (GRADE) working group.
Any disagreements will be recorded and resolved by involvement of an additional
reviewer.
6.0 Data synthesis
The availability of appropriate data and resources to conduct a meta-analysis will be
considered, where feasible.
7.0 Dissemination
A final set of tables including a GRADE Evidence Table and Descriptive Evidence Table
will be produced and submitted to the WHO Secretariat as stipulated in the Procedures
for the Retrieval of Evidence and Summary of Evidence. In addition, a manuscript will be
prepared for submission to a peer-reviewed journal (a more specific dissemination proposal can
go here as well).
8.0 Resource implications
The project lead will work closely with the WHO Secretariat to define the scope and
methods of the review and facilitate access to unpublished literature, supporting
translation of foreign language literature where necessary. Proposed milestones and
timescales are outlined below:
MILESTONES
Identification and retrieval of the evidence
Conduct literature review, hand searching, and contacts with experts and networks
Summarizing evidence in Descriptive
Evidence Tables (Table 1. Summary of systematic literature reviews, Table 2. Summary of
relevant literature)
Development of GRADE Evidence Tables in collaboration with WHO, Geneva, and the
GRADE network methodologist
7/12
9.0 References
When citing articles, please use the WHO standard citation format, called the Harvard system. The
Harvard citation system shows the author and date in the body of the text. This may be done in one of two
ways:
Ballance, Ewart & Fitzsimmons (2001) have reported ...
It has been reported (Ballance, Ewart & Fitzsimmons, 2001) that ...
Some examples of formatting reference lists:
Article in a Journal
Burt BA, Pai S (2001). Sugar consumption and caries risk: a systematic review. Journal of Dental Education,
65:1017–1023.
Chapter in a book
Melton L J III (1995). Epidemiology of fractures. In: Riggs BL, Melton L J III, eds. Osteoporosis: etiology,
diagnosis, and management, 2nd ed. Philadelphia, PA, Lippincott-Raven, 225–247.
Corporate author
Heart Protection Study Collaborative Group (2002). MRC/BHF Heart Protection Study of antioxidant
vitamin supplementation in 20 536 high-risk individuals: a randomised placebo-controlled trial. Lancet,
360:23–33.
WHO Publication with no named author
World Health Organization (2003). The international pharmacopoeia, 3rd ed. Vol. 5. Tests and general
requirements for dosage forms; quality specifications for pharmaceutical substances and tablets. Geneva,
World Health Organization.
More on the WHO-style can be found here: http://www.nlm.nih.gov/bsd/uniform_requirements.html
8/12
Appendix A: database search strategy
PUBMED Search strategy
Population
(Faculty[Mesh] OR Preceptorship[Mesh] OR Mentors[Mesh] OR Faculty[tw] OR
Faculties[tw] OR "Academic physician"[tw] OR "Academic Physicians"[tw] OR
teachers[tw] OR teacher[tw] OR mentors[tw] OR mentor[tw] OR preceptor[tw] OR
preceptors[tw] OR tutor[tw] OR tutors[tw] OR educator[tw] OR educators[tw] OR
"teaching staff"[tw] OR lecturer[tw] OR lecturers[tw] OR instructor[tw] OR
instructors[tw] OR trainer[tw] OR trainers[tw]
Intervention
"Staff development"[Mesh] OR "Faculty development"[tw] OR "Facultydevelopment"[tw] OR "Staff-development"[tw] OR "Staff development"[tw] OR "Faculty
training"[tw] OR "Faculty-training"[tw] OR "teacher training"[tw] OR "teachertraining"[tw] OR "Continuing development"[tw] OR "Continuous development"[tw] OR
"Professional development"[tw] OR "professional continuing education"[tw] OR
"Professional development"[tw] OR "Human resources development"[tw] OR
"Professional training"[tw] OR "teacher improvement"[tw] OR "teaching
improvement"[tw] OR "teaching skills improvement"[tw] OR "teachingimprovement"[tw] OR "teacher development"[tw] OR "teaching development"[tw] OR
"teaching to teach"[tw] OR "teach the teacher"[tw]
Search
Most Recent Queries
Time
Result
#3 Search (#1) AND #2 Limits: English, Publication Date from 1990 to 2011
08:38:26
3505
#2 Search "Staff development"[Mesh] OR "Faculty development"[tw] OR
08:38:07
11026
"Faculty-development"[tw] OR "Staff-development"[tw] OR "Staff
development"[tw] OR "Faculty training"[tw] OR "Faculty-training"[tw]
OR "teacher training"[tw] OR "teacher-training"[tw] OR "Continuing
development"[tw] OR "Continuous development"[tw] OR
"Professional development"[tw] OR "professional continuing
education"[tw] OR "Professional development"[tw] OR "Human
resources development"[tw] OR "Professional training"[tw] OR
9/12
"teacher improvement"[tw] OR "teaching improvement"[tw] OR
"teaching skills improvement"[tw] OR "teaching-improvement"[tw]
OR "teacher development"[tw] OR "teaching development"[tw] OR
"teaching to teach"[tw] OR "teach the teacher"[tw] Limits: English,
Publication Date from 1990 to 2011
#1 Search Faculty[Mesh] OR Preceptorship[Mesh] OR Mentors[Mesh] OR
08:37:55
62612
Faculty[tw] OR Faculties[tw] OR "Academic physician"[tw] OR
"Academic Physicians"[tw] OR teachers[tw] OR teacher[tw] OR
mentors[tw] OR mentor[tw] OR preceptor[tw] OR preceptors[tw] OR
tutor[tw] OR tutors[tw] OR educator[tw] OR educators[tw] OR
"teaching staff"[tw] OR lecturer[tw] OR lecturers[tw] OR
instructor[tw] OR instructors[tw] OR trainer[tw] OR trainers[tw] Limits:
English, Publication Date from 1990 to 2011
ERIC Search Strategy
((((Thesaurus Descriptors:"Faculty" or Thesaurus Descriptors:"Teachers" or Thesaurus
Descriptors:"Practicum Supervision") and (Thesaurus Descriptors:"medical
education" or Thesaurus Descriptors:"medical schools" or Thesaurus
Descriptors:"medicine" or Thesaurus Descriptors:"dentistry"or Thesaurus
Descriptors:"dental schools" or Thesaurus Descriptors:"nursing" or Thesaurus
Descriptors:"nursing education" or Thesaurus Descriptors:"pharmacy" or Thesaurus
Descriptors:"pharmaceutical education")) or (Thesaurus Descriptors:"medical school
faculty" or Thesaurus Descriptors:"dental school faculty")) and (Thesaurus
Descriptors:"Faculty Development" or Thesaurus Descriptors:"faculty
improvement" or Thesaurus Descriptors:"Teacher Improvement" or Thesaurus
Descriptors:"Staff Development" or Thesaurus Descriptors:"Inservice Teacher
Education" or Thesaurus Descriptors:"Teacher workshops" or Thesaurus
Descriptors:"Teacher Education Programs"))Publication Date:1990-2011
Appendix B: data extraction forms (see attached excel workbook)
1) GRADE Evidence Table Template.xls
2) Descriptive Evidence Table.xls
10/12
Appendix C: Example of databases, websites and journals which could be searched
Databases
Databases for biomedicine and health sciences:
 Pubmed/MEDLINE
 CINAHL
 Cochrane Library (CDSR)
 Health Systems Evidence (PPD/CCNC)
 UK Pubmed Central
 Global Health Library (Global Index Medicus)
 UK Pubmed Central
Databases for social sciences:
 ISI Web of knowledge
 Campbell Library
 Social Science Research Network
Databases for education:
 ERIC
 BEME database
Regional databases for biomedicine and health sciences:
 African Index Medicus (www.indexmedicus.afro.who.int)
 Australasian Medical Index (www.nla.gov.au/ami)
 Index Medicus for Eastern Mediterranean Region (www.emro.who.int/his/vhsl)
 Chinese Biomedical Literature Database (CBM)
(www.imicams.ac.cn/cbm/index.asp)
 IndMED (www.indmed.nic.in)
 KoreaMed (www.koreamed.org/searchbasic.php)
 LILACS (www.bases.bireme.br/cgibin/wxislind.exe/iah/online/?IsisScript=iah/iah.xis&base=LILACS&lang=i)
 Index Medicus for the South-East Asia Region (IMSEAR)
(www.library.searo.who.int/modules.php?op=modload&name=websis&file=imse
ar)
 Panteleimon for Russia and Ukraine (www.panteleimon.org/maine.php3)
 Western Pacific Region Index Medicus (WPRIM) (wprim.wpro.who.int/SearchBasic.php)
 PASCAL (www.international.inist.fr/article21.html)
General databases:
 Google Scholar
11/12
4.1.2 Hand searching
The following journals and websites will be hand-searched for relevant articles:
Resources that will be searched by hand, for peer-reviewed articles and grey literature
 Assessment & Evaluation in Higher Education
 American Journal of Health Studies
 Teaching and Teacher Education
 Academic Medicine
 Journal of Interprofessional Care
 Academic Medicine
 Medical Teacher
 International Journal for Academic Development
 WHO's Library Database (WHOLIS)
 SAMMS
 HRH Global Resource Center
 World Federation of Medical Education
 International Council of Nurses
 Association of American Medical Colleges
 Council on Graduate Medical Education
 American Academy of Family Physicians
 American College of Physicians
 World Bank
 HMIC database
 Grey Literature Network Service http://www.greynet.org
 OPENSIGLE System for Information on Grey Literature in Europe
http://opensigle.inist.fr/
 NewYork Academy of Medicine Grey Literature site
http://www.nyam.org/library/online-resources/grey-literature-report/
 FADE Library http://www.fade.nhs.uk/
 HMIC: Health Management Information Consortium
http://library.nhs.uk/help/resource/hmic
 Biomed http://www.biomedcentral.com/
 INTUTE http://www.intute.ac.uk/
12/12