NC3Rs resources to improve the design of
animal experiments
Dr Nathalie Percie du Sert
BBSRC STARS Course
Tuesday 11 April 2017
The NC3Rs (UK National Centre for the 3Rs)
 Lead on the discovery and application of new technologies and approaches
to replace, reduce and refine the use of animals for scientific purposes (the
3Rs)
Visit our website:
www.nc3rs.org.uk
@NC3Rs
National Centre
for the 3Rs
 Work with research funders, journals, academia, industry and regulators
 Activities divided between:
-
Research funding
Centre-led science programmes
NC3Rs resources
The ARRIVE guidelines and the Experimental Design Assistant
Background
Quality of published animal research
Experimental design
Statistical analysis
Reporting of studies
Only 12% of publications
report randomisation and
14% report blinding
Only 70% of publications fully
described the statistical
methods and presented the
results with a measure of
variability
Animal characteristics –
missing in 25%
Sample size justification –
missing in 100%
Only 59% stated the study
hypothesis, number and
characteristics of animals
used
Survey reviewed 271 publications and identified key areas for
improvement
Kilkenny C, Parsons N, Kadyszewski E, Festing MF, Cuthill IC, Fry D, et al. (2009). Survey of the quality of
experimental design, statistical analysis and reporting of research using animals. PLoS One 4(11): e7824.
Experimental design – internal validity
Consider threats which might compromise the validity of the experiment, any
opportunities for the investigator to influence:

animal selection

conduct of the experiment

assessment of outcome

which outcomes are reported
Measures used to reduce validity threats include:
 Random allocation to treatment groups
 Allocation concealment
 Blinding during outcome assessment
 Inclusion/exclusion criteria
Experimental design – randomisation
Method is important – haphazard is not random
Use a validated procedure (e.g. computer
generated, throw a dice, flip a coin)
Randomisation is crucial for two reasons:
1. Minimise selection bias
e.g. haphazard selection may results in slowest mice allocated to the
same group
2. Key assumption of the statistical analysis
Different groups should be drawn from the same background population
using random sampling
Experimental design – randomisation
Pick a number between 1 and 20
347 responses
Is 17 the “most random”
number?
http://scienceblogs.com/cognitivedaily/2007/02/05/is-17-the-most-random-number/
Experimental design – blinding
 12 students
 Maze-bright and maze-dull rats
 Elevated T-maze, dark arm reinforced
bright
rats
3
Number of 2
correct
responses 1
dull
rats
0
1
2
3
Days
4
5
 Rats had been labelled
bright or dull randomly
 Only difference was in the
minds of the investigators!
Rosenthal R, Fode KL (1963). The effect of experimenter bias on the performance of the albino rat. Behavioral Science 8(3): 183-189.
Improving the reporting of in vivo research
The ARRIVE guidelines
The ARRIVE guidelines were developed to improve the reporting of
biomedical research using animals.
 Checklist of 20 items, containing
key information necessary to
describe a study comprehensively
and transparently.
The guidelines include:
 Information which relates to
internal validity
 Information which would allow a
study to be repeated
 Information about the context and
scientific relevance of the study
https://www.nc3rs.org.uk/arrive-guidelines
The EDA
Experimental Design Assistant
The EDA was developed to improve the design of animal experiments
 Web-based tool
 Aimed at in vivo researchers
 Developed as a collaboration
between:
 In vivo researchers
 Statisticians
 Academia and industry
 Software designers specialised
in artificial intelligence
 Road tested by researchers and
statisticians
https://eda.nc3rs.org.uk
The EDA offers:
 The ability to build a stepwise visual representation of an
experiment – the EDA diagram
Effect
of drug A
on plasma
glucose
levels
Animals
characteristics:
diabetic mice
Experimental
unit:: mouse
Group 1
Independent
variable of
interest :
Drug A
Pharmacological
intervention 1
Vehicle
Drug
Vehicle
Pool of
animals
Measurement:
Plasma glucose
Allocation:
Complete
randomisation
Group 2
Pharmacological
intervention 2
Drug
Outcome
Measure:
Glucose levels
Analysis
The EDA offers:
 The ability to build a stepwise visual representation of an
experiment – the EDA diagram
Experiment
Effect
of drug A
on plasma
glucose
levels
Animals
characteristics:
diabetic mice
Experimental
unit:: mouse
Group 1
Independent
variable of
interest :
Drug A
Pharmacological
intervention 1
Vehicle
Drug
Vehicle
Pool of
animals
Measurement:
Plasma glucose
Allocation:
Complete
randomisation
Group 2
Pharmacological
intervention 2
Drug
Practical steps
Outcome
Measure:
Glucose levels
Analysis
Analysis
The EDA diagram
 Examples
 Templates
 Feedback from the critique
The EDA offers:
 The ability to build a stepwise visual representation of an
experiment – the EDA diagram
 Feedback and advice on your experimental plan
 Dedicated support for randomisation, blinding and sample size
calculation
 Practical information to improve knowledge of experimental
design
 Improved transparency of the experimental plan, allowing more
efficient communication
Feedback and advice from the EDA
 Dataset of rules triggers prompts based on the EDA diagram
 Feedback provided:
 Diagram structure
 Ask to provide more information
 Point out inconsistencies
 Prompt you to consider things that are not on the diagram
 Highlight implications of some of the choices made
 Provide recommendation for analysis
 Rule set to be expanded over time
 Provide more feedback
 Enable the system to detect more subtle issues
 Increasingly specific feedback
The EDA offers:
 The ability to build a stepwise visual representation of an
experiment – the EDA diagram
 Feedback and advice on your experimental plan
 Dedicated support for randomisation, blinding and sample size
calculation
 Practical information to improve knowledge of experimental
design
 Improves transparency of the experimental plan and helps
communication
The EDA offers:
 The ability to build a stepwise visual representation of an
experiment – the EDA diagram
 Feedback and advice on your experimental plan
 Dedicated support for randomisation, blinding and sample size
calculation
 Practical information to improve knowledge of experimental
design
 Improved transparency of the experimental plan, allowing more
efficient communication
The EDA offers:
 The ability to build a stepwise visual representation of an
experiment – the EDA diagram
 Feedback and advice on your experimental plan
 Dedicated support for randomisation, blinding and sample size
calculation
 Practical information to improve knowledge of experimental
design
 Improved transparency of the experimental plan, allowing more
efficient communication
The EDA workflow
Objectives
 Improve the reliability of published results
 Promote better understanding of experimental design, raise
awareness about issues
 Facilitate peer review/assessment of the experimental plans with an
explicit description
– Transparency
– Pre-registration
 Promote more careful consideration of the experimental plans
– Spend time planning
– Diagram facilitate discussion
EDA uptake
~2500 accounts on the system with 30 diagrams created per week
Recommended by UK funders
Acknowledgments
NC3RS working group
Certus Technology
Prof Clare Stanford (Chair), UCL
Dr Brian Lings
Dr Simon Bate, GlaxoSmithKline
Mr Ian Bamsey
Dr Manuel Berdoy, University of Oxford
Dr Robin Clark, Huntingdon Life Sciences
Alpha testers
Prof Innes Cuthill, University of Bristol
Beta testers
Dr Derek Fry, University of Manchester
Dr Natasha Karp, Wellcome Trust Sanger Institute
Prof Malcolm Macleod, University of Edinburgh
Dr Lawrence Moon, King’s College London
Dr Richard Preziosi, University of Manchester
https://eda.nc3rs.org.uk
www.nc3rs.org.uk/ARRIVE