Implementing Quality Systems in
Core Facilities.
A Good Strategy to Improve Data
Reproducibility in Research?
SQA | 5 April 2016 | USS Session I-2
Carmen Navarro, PhD, Quality Research Service,
University of Barcelona, SPAIN
Rebecca Davies, PhD, Quality Central; College of
Veterinary Medicine, University of Minnesota, St- Paul,
Minnesota, USA
1
Schedule
• Core facilities: general features and advantages
for institutions and researchers
• Why QA implementation in core facilities?
• The case of the University of Barcelona: the
Scientific and Technological Centers
• The case of University of Minnesota: Quality
Central
• Closing remarks
2
Core facilities
• Core Facilities are centralized scientific research
infrastructures that allow research scientists shared access
to sophisticated and expensive technologies that would be
hard or impossible for the Universities and Research
Institutions to provide to each research group
independently.
• The concentration of first-rate equipment, with back up by
supporting experts, allows the covering of a broad
spectrum of services accessible in one place.
3
Main features of Core Facilities
• Equipment is of high performance and the most
current technology
• Facilities are readily accessible to scientists
• Staff with a high level of expertise in their area
• Central Management System
4
Advantages of Core Facilities
For the
researchers
-Access to high
performance equipment
-Support provided by
specialized staff
For the institution
- Optimization of
investments in equipment
- Centralized management
5
What do the scientists expect
from Core Facilities?
• Equipment suitable for the experiments they are doing
• Equipment in good condition and properly maintained, verified and
calibrated
• Technical staff expert in their area
• Reliable results (design, performance, evaluation)
Good Research
Practices
6
QA systems in Research groups
Most research groups have not established
research QA Systems
• Mentor based
• Lack of experience using Quality Management Systems
7
Contribution of Core Facilities to
Data Reproducibility in Research
•
•
•
•
•
Staff with high level of specialization
Equipment well controlled
Defined Organization Chart
Management System
Great impact on researchers: one single system is useful
for many scientists
Good place to implement Quality
Assurance best practices
8
Records
Equipment
QA
Best
Practices
Reagents
Specimens
Procedures
9
The University of Barcelona
case: Scientific and
Technological Centers
10
Methodological development · Tecnological Advice · Technological trainings
MISSION
SECTORS
Supporting Research
Tecnological Advice
Methodological development
Training
Multidisciplinarity
36 Technological
Units
USERS
440 UB researchers
69 public institutions
288 private institutions
170 Technicians
60 Doctors
Biosciences
ACCREDITATIONS
THE CCiTUB
11
Methodological development · Tecnological Advice · Technological trainings
12.000m2 facilities
Campus Diagonal
Campus Mundet
Campus Ciències de la Salut
de Bellvitge
Campus de Medicina –
Hospital Clínic
LOCATIONS
12
Methodological development · Tecnological Advice · Technological trainings
CCiTUB
Management
Quality Committee
Innovation and
technological
transfer Manager
Operational
Coordination
Quality Responsible
Administrative Unit
- Purchases and billing
area (supplier)
- Incomes and invoices
area (customer)
- Staff area
Image and IT
Logistics
Technological Units
Information Desk
Technological Coordination
Technologies
- Chemical
- Materials
- Bioscience
ORGANIZATION CHART
13
Methodological development · Tecnological Advice · Technological trainings
TECHNOLOGIES
MATERIALS
• Surface Analysis (ESCA/Auger)
• X-Ray Diffraction
• Secondary Ion Mass Spectrometry (SIMS)
• Granulometry
• Transmission Electron Microscopy
• SPM NanoTechniques (AFM,STM)
• Interferometry-confocal
• Scanning Electron Microscopy
• Electron Microprobe
• Paleomagnetism
• BET Specific Surface and Porosimetry
• Mechanical, Electronic and Vacuum
CHEMISTRY
• Metal analysis (ICP-MS, ICP-OES, FRX, AA)
• Gas Chromatography – Applied Mass Spectrometry
• Radiocarbon Dating
• Molecular Characterization Mass Spectrometry
• Isotopic Ratio Mass Spectrometry
• Molecular Spectroscopy
• Raman Spectroscopy
• Magnetic Measurements
• Polymorphism and Calorimetry
• Nuclear magnetic resonance
• Environmental Techniques
• Separation Techniques (HPLC, LC-MS, LCHRMS, EC and AEO).
BIOSCIENCES
• Image Analysis
• Molecular Interaction Analysis
• Bioinformatics
• Cytometry
• Electron Cryo-Microscopy
• Animal facilities
• Genomics
• Electron Microscopy (TEM/SEM)
• Advanced Optical Microscopy
• Proteomics
• SPM NanoBio techniques (AFM, STM)
• Radiological Protection Technical Unit
14
Methodological development · Tecnological Advice · Technological trainings
APPLICATIONS
MATERIALS
STRUCTURE
COMPOSITION
SURFACE CHARACTERIZATION
● Microstructural and morphological analysis
● Identification of crystalline phases
● Chemical analysis (qualitative / quantitative)
● Contaminants and impurities
● Metal oxidation states
● Compositional mapping
● Multilayer analysis and perfilometry
● Identification of oxide surfaces and corrosion products
● 2D and 3D topography (roughness analysis)
CHEMISTRY
IDENTIFICATION
ANALYSIS
CARACTERIZATION
● Chemical compounds and impurities
● Contaminants
● Waters, soils, sediments and wastes
● Metals in organic and inorganic samples
● Isotopic ratio studies (18O, 13C, 15N, 34S)
● Thermal analysis and calorimetry
● Crystalline engineering
● Development and validation of analytical methods
● Quality control
BIOSCIENCES
IDENTIFICATION
ANALYSIS AND CHARACTERIZATION
STUDIES
● Identification of microorganisms through biological
molecular techniques
● Detection and quantification of organisms and genes
● Analysis of proteins, DNA, RNA, amino acids
● Metabolic processes
● Pathogen detection
● Biomaterials and implants
● Radiological protection
15
TRAINING
Methodological development · Tecnological Advice · Technological trainings
AUTHORISED COURSES
COURSES ON CCiTUB TECHNOLOGIES
SELF-USER’S TRAINING
TECHNOLOGICAL ADVICE
CONFERENCES, SEMINARS, WORKSHOPS
COLLABORATIONS (MASTERS, DOCTORATES)
16
QA system at CCiTUB
• According to ISO 9001 Standard, certified
since 2005
• Focused on reliability and reproducibility of
results provided by equipment
• Implementation and maintenance under the
responsibility of the Quality Research Service
at the UB
17
Quality Research Service
• The QRS was created by the
University of Barcelona to provide
support in all the processes of
implementation, formal
acknowledgement and maintenance
of quality systems implemented in
research groups and Core Facilities
at UB.
18
Science and Technology Centres
Quality Policy
The Science and Technology Centres of the University of Barcelona (CCiTUB) are a
combination of specialized scientific-technical structures.
Mission
To give support to research and innovation in the fields of Chemistry, Materials Science and
Biosciences, as well as to mediate in university-business relations and promote innovation
and technology transfer through research and development agreements and projects with
industry.
Vision
To be a leading centre of scientific-technical activities and actively contribute to scientific
progress and the betterment of society on both a national and an international scale.
Values
 Results with a guarantee of quality, reliability, integrity and traceability
 Highly qualified and technically specialized human team
 Leadership based on people
 Wide array of technologically advanced scientific instrumentation
 Constant and specialized methodological and technological development
 Multidisciplinary and flexible services that are adaptable to user needs
 Confidentiality of information related to user services
 Mutually beneficial relationships with collaborators and suppliers
 Responsible management of the human resources and materials necessary to meet
the quality objectives
 Ethical, socially responsible and transparent code of conduct
 Practices that are safe and environmentally friendly
 Promotion of staff improvement activities, strengthening lifelong learning to enhance
the quality of services
The CCiTUB have an ISO 9001-certified quality management system that allows for the
continuous improvement of processes and raises user satisfaction, thanks to the support of
the University and the participation of the CCiTUB staff.
Dr José Ramón Seoane Trigo
Director of the CCiTUB
Barcelona, 23 February 2016
19
Equipment:
Use
Maintenance
Verification
Calibration
Staff records:
Education
Training
Competence
Evaluation:
Internal and
external audits
Management
revision
QA System
at CCiTUB
CAPA system:
Corrective actions
Preventive actions
Methods:
Validation
Quality control
Proficiency testing
Documentation:
Quality Manual
SOPs
Records
20
21
What is driving change?
 Access to QA tools (software) and expertise
 Research Reproducibility & Threat to Scientific
Progress
 Traditional Gatekeeper Limitations/Public Distrust
 High Cost of Research Wastage/ Questionable
Research Practices
 Potential to provide Credible Evidence for data quality
 Trainee and Stakeholder needs
 Competitive Advantage
22
22
What is limiting change?




Access to QA tools (software) and expertise
Simple, Sustainable and Science Centered Models
Lack of an acceptable Financial Model
Lack of QA experience within academia- among
administration, faculty and
 Lack of Funder/Publisher Pressure to incorporate QA
best practices in non-regulated research.
 Scientist and administration fear of Q overreach
23
23
What would ensure change?
24
24
Robust research: Institutions must do
their part for reproducibility
C. Glenn Begley, Alastair M. Buchan & Ulrich Dirnagl
Nature | Comment 01 Sep 2015
GIP
A grassroots idea to share QA capabilities
Quality Central: A cooperative approach to
implementing QA in regulated and nonregulated biomedical research at the
University of Minnesota.
Scientists helping scientists integrate QMS or
QA best practices into research or service
teams to support data reliability.
26
Centralized Support Model for Service
Units
Add Value
S
S
Educate
S
GXP
Prove Concept
S
R
Establish an
Infrastructure
S
GRP
Advocate for Quality
27
Veterinary
Diagnostic
Lab, AAVLD
Infectious
Agent
Repository,
GRP
Pre Clinical
Investigation
Center, GLP
$
GXP
2014
Swine
Medicine
PCRC,
GLP, GRP
Research
Community
Research $ ?
Diabetes
Institute
GRP
$
$
$
$ QA Budgets
Model 1: Individual Faculty Research
Individual
Faculty
SOPS and
RECORDS:
GXP
Research
Projects
Equipment
management
Reagent management
Research Grant Budget
QA Justification
Focus on Data Rigor
Notebook review
Method validation
Training
Project Based, 2 projects
funded
Audit
QA Integration “ACTIVATION ENERGY”
People,
Training,
Experience
Equipment
Facilities
Documents
Materials
Central
QMS
Methods
Records
QC
Data
Review
Processes
+ Monitoring for OFI
Model 1: Two Research Projects (each 6 months)
Funded by the National Pork Board
to improve immunodiagnostic
testing for Porcine Epidemic
Diarrhea Virus and Porcine
deltacoronavirus.
Both projects included funding for
external (centralized) QA support to
help researchers meet a risk-based
Quality Commitment.
Note: These are not full QMSystems!
The Research Quality Commitment
Description of Quality Assurance/Quality Control Plan – The
Quality Central personnel will work with researchers to ensure that:
1. Data integrity is preserved:
• data reconstruction is possible
• data are: Accurate, Legible, Contemporaneous, Original and
Attributable
• Laboratory work will be documented in bound notebooks or in
electronic files that are backed up to University servers
2. Methods are ‘fit for purpose;
• Test method data are: accurate (reflect the absolute true
value), reproducible (precise; consistent), and controlled (QC
strategies are designed to show that the test is working as
expected
3. Reagent and Equipment care and use records are maintained
32
The Quality Research Practices (GRP)
Checklist
 Section 1: Personnel Management/Training
 Section 2: Equipment Management
 Section 3: Methods Validation Management
 Section 4: Standard Operating Procedures
 Section 5: Data and Document Management
(paper/electronic)
33
GR
P
“I want the research results we generate to reflect
the biology of the pig, and not the effect of
equipment, reagents, inconsistent
methodology
GRP
or the inappropriate collection of samples or
University of Minnesota
Murtaugh
Laboratories
Audit Date:
Auditor:
Auditee(s):
Title: Good Research Practices Audit Checklist
SECTION 1. Personnel Management & Compliant
Training
Are resumes or CV’s available for project personnel?
Is a signature & initials identification log available
Are training records maintained?
SECTION 2. Equipment
Management
Are SOP’s available for use of equipment,
maintenance of equipment and calibration?
Partially
Compliant
Not
Compliant
Audit Report Form
Site: M.Murtaugh Lab; Audit Date: 8/19/2014 Audit Number: GRP-1
Lead Auditor: Carrie Wees, VDL Quality Manager
Auditee(s): Project teams (Supervisors, technicians and undergraduate
GR
employees)
P
Audit Purpose: To assess whether the Murtaugh Lab has developed a
research QA plan (GRP Research Checklist), and whether there is evidence
that the plan is being implemented.
Findings:
Compliant = Laboratory meets all GRP requirements
Partially Compliant = There is a plan in place to meet each
requirement
Not Compliant = There is noGRP
evidence or plan to meet a
requirement
Improving Data Integrity and Reliability
in Animal Health Research
Brendan Davies, Audrey Tseng, Matilda Wagner, Kyra
Martin, Suzanne Stone, Cheryl Dvorak, Katrina Laube,
Rebecca Davies, Michael Murtaugh
Department of Veterinary and Biomedical Sciences, College of Veterinary
Medicine, University of Minnesota, St.Paul, MN
Model 2 Research Consortium
GRP
Multiple PIs
QA funding pool
QA cost included in grant
Focus on data reliability
and reconstruction
Challenges
It is not easy to succeed
Scope, time requirement
Staff training (lack of experience/understanding of QA)
Resentment for ‘intrusion’ into scientists domain
Language (Best Practices vs Regualory jargon)
Buy in: management and staff
Perception vs reality; quick fix vs culture change
Perception of cost and budgetary decisions
More Models: Quality Collaborations
Collegiate Seminars
Training courses/Certificate Programs
Industry/Academia Partnerships
Non-Regulated/Regulated Research Team Partnerships
Web Based Resources
Multi-Institution Work Groups to develop training tools
40
What the scientists get from
core facilities’ QA system?
• Good experimental
design
• Reliable results provided
by instruments
• Integrity and traceability
of records
QA Systems are
good for data
reproducibility
in research
41
Closing remarks
• Core facilities have multiple advantages for scientists
and institutions to optimize access to sophisticated
technologies and QA expertise.
• Scientists’ expectations from core facilities are directly
related to the risk mitigation that QA elements provide.
• QA implemented in core facilities promotes reliability,
integrity and traceability of research results.
• This strategy is an effective way to show the scientists
how QA can contribute to improve research data.
42
Implementing Quality Systems in Core
Facilities.
A Good Strategy to Improve Data
Reproducibility in Research?
SQA 5 April 2016; USS Session I-2
Carmen Navarro, PhD, University of Barcelona
[email protected]
Rebecca Davies, PhD, University of Minnesota
[email protected]
43