Principles of environmental risk assessment: reducing
uncertainty in decision-making
Alan Raybould, Product Safety, Syngenta
Environmental risk assessment: it’s all very complex...
“The complexity of ecological systems presents considerable challenges
for experiments to assess the risks and benefits and inevitable
uncertainties of genetically engineered plants”
Wolfenbarger and Phifer (2000) The ecological risks and benefits of
genetically engineered plants. Science 290: 2088-2093
2
...so we need lots of data...
3
...to improve our decision-making
4
More data = better decision-making
● This analysis is wrong!
- Risk assessment doesn’t have to be complex
- Scientific uncertainties are often irrelevant
- Many data are superfluous or even detrimental to decision-making
● Technical reasons
- The scientific method
- Confusion between risk assessment and fundamental research
● Non-technical reasons
- Decision-making requires judgement
- Policy uncertainties are more important than scientific uncertainties
- Scientific analysis cannot replace judgement or policy
5
The bucket theory of science (empiricism)
Observations
6
Facts
Truth
The bucket theory of science (empiricism)
All swans
are white
7
The bucket theory of risk assessment
Observations
of the GMO
Facts about
the GMO
Safety of
the GMO
● Safety cannot be proven no matter how many data are collected
8
Science begins with problems not data
→ initial problem [P1] → tentative solution [TS] → error elimination [EE] →
new knowledge and a new problem [P2] →
Karl Popper, Objective Knowledge, an Evolutionary Approach
● Hypotheses are corroborated or falsified by testing them
- We try to show they are false
● Hypothesis are not proven by collecting data to support them
- We cannot show that they are true
● Knowledge accumulates by trial and error (elimination)
9
Risk assessment as hypothesis testing
Decide what constitute harmful effects of cultivating the GM crop [P1]
↓
Hypotheses that cultivation of the GM crop will not cause harm [TS1]
↓
Test the hypotheses [EE1]
↓
Increased knowledge of risk [P2]
↓
New hypotheses and decision-making [TS2]
↓
“Safety” is sufficiently well-corroborated hypotheses of no harm
10
The source of problems (P1)
● Risk assessment is scientific – follows the scientific method
● Nevertheless it differs significantly from fundamental research
● Science is generally regarded as objective
● This really applies to the error elimination (testing) phase
- Objectively compare observations with predictions
● Selection of scientific problems contains subjectivity
- Personal interests of the scientist
- Societal interests mediated through allocation of funds
- (Hypotheses are created not discovered)
11
The source of problems (P1)
● Mistaken view that problem selection is objective is probably not a
problem for fundamental research
● Highly detrimental to risk assessment
● Risk = harm x probability
- Harm implies loss of something we value
- Values are subjective
- Cannot be deduced scientifically
● Attempts to eliminate subjectivity prevent proper problem formulation
- Resort to collecting data instead of defining what is harmful
12
Source of problems
● Problem definition: change question from “what will happen if the GM
crop is cultivated?” to “what is the probability that something harmful will
happen if the GM crop is cultivated?”
Harmless
effects
All possible
effects
Harmful
effects
13
Formulating hypotheses (TS1)
● In fundamental research, good hypotheses provide explanatory power
- Better hypothesis → greater explanatory power
- Predictions are accurate and precise
● In risk assessment, good hypotheses help decision-making
- Better hypotheses → clearer distinction between choices
- Predictions are accurate and relevant
● H1: while I’m in Bern, the temperature at home will not fall below 0C
- Uninteresting scientifically, but relevant to decision-making
● H2: the minimum temperature at home will be 7.3467C at 4.07am today
- More interesting scientifically, but the extra precision is not relevant
● Uncertainty about the value of a variable (e.g., temperature or metabolic
profile), does not imply uncertainty about risk (e.g., the probability of
plants being damaged by frost or of a crop having a harmful phenotype)
14
Risk hypotheses
● Risk assessment should value accurate categorisation over precise
quantification
Harmless
effects
Prediction
All possible effects
Harmful
effects
15
Formulating useful hypotheses for ERA (risk hypotheses)
Scenario
Cultivation of GM crop
↓
Event A
↓
Event B
↓
Event C
↓
Event D (Harm)
16
Risk hypothesis
Event A will not occur
Event B will not occur
Event C will not occur
Event D will not occur
Formulation of risk hypotheses
Scenario
Cultivation of a Bt crop
↓
Bt crop produces pollen
↓
Bt protein is produced in pollen
↓
Pollen disperses outside the field
↓
Pollen is eaten by a valued species
↓
Bt protein is toxic to the valued species
↓
The species receives a harmful dose of protein
↓
The abundance of the valued species is reduced
17
Risk Hypothesis
Bt crop does not produce pollen
Bt protein is not produced in pollen
Pollen does not disperse outside the field
Pollen is not eaten by the species
Bt protein is not toxic to the species
Harmful dose of protein not received
The abundance of the species is not reduced
Categorisation verses quantification
Risk hypothesis
Research hypothesis
● There will be no crop A x wild
● Number of A x B hybrids = X c d
plant B hybrids in nature reserves
ef
● The weediness potential of
transgenic crop A is no greater
than that of its progenitor B
● The number of weeds of A = Y a
bcefghjklmnopqrstuv
w
● NOAEC > 10X highest exposure ● NOAEC = x µg/g diet
EEC = y µg/g diet
Risk hypotheses may concentrate on categorisation not quantification
Quantify only to the extent needed to make a decision
18
Testing hypotheses (EE1)
● Confidence in the risk assessment is derived from the rigour with which
risk hypotheses of no harm are tested
- How intensively have we searched for potentially harmful effects?
- Seek a black swan, do not count white swans
● It is the ability of the test to detect a potentially harmful effect that is
important, not the realism (complexity) of the test
- Lab studies to detect adverse effects of proteins
- Lab studies to test for hybridisation
- Glasshouse studies under worst-case conditions to detect increased
weediness potential
● Confidence in an ecological theory often does depend on realism
19
Testing hypotheses
Increase in realism
Reduction in generality
Ability to detect effects
Rigorous ERA
20
Ability to evaluate
relevance of effects
Rigorous ecological research
Use of existing data
● In ecological research, convincing corroboration of a new hypothesis
often requires new data
- Existing data may not be in a suitable form
- Existing data may have been used to formulate the hypothesis
● In risk assessment, convincing corroboration of new hypotheses with
existing data is likely and desirable
- NTO effects data on Cry1Ac and Cry2Ab collected for cotton are
relevant to pigeon pea (Journal of Applied Entomology 133: 571-583)
- Do we still need compositional analysis given that 20 years’
experience of breeding and selecting transgenic crops has revealed
no greater likelihood of harmful unintended effects than from
conventional breeding? (Trends in Biotechnology 27: 555-557)
21
Non-technical considerations – data may worsen decisions
● Needle in haystack
- Tests indicating potential harm may be
missed among a mass of irrelevant data
● Procedure to eliminate judgement
- Obsession with detail not substance
- e.g., discussion of statistics has superseded
discussion of the relevance of endpoints
- Transparency versus accountability
22
Non-technical considerations – data may worsen decisions
● D. Sarewitz, 2004, Environmental Science and Policy
- How science makes environmental controversies worse
23
Non-technical considerations – data may worsen decisions
● J. Tait, 2001 Journal of Risk Research 4, 175-189
- Not in my backyard (NIMBY) and not in anyone’s backyard (NIABY)
NIMBY
NIABY
Interest-based
Restricted to specific applications
Specific locations, locally organised
Can be resolved:
By giving information
By giving compensation
By negotiation
Concessions lead to mutual accommodation
Ideology-based
All applications
Organised nationally or internationally
Very difficult to resolve:
Information = propaganda
Compensation = bribery
Negotiation = betrayal
Concessions lead to escalation of demands
● If controversy is ideological, data will simply exacerbate the problem
24
Non-technical considerations – the role of policy
● “...in complex circumstances where there is a limited quantity of scientific
knowledge, the aim of the rational agent is not really to make the right
decision (there may be no such thing), it is to make the decision right.”
● “...making a decision is rarely the end of the affair, each decision has to
be followed by innumerably many more, correcting and refining the initial
one.”
- David Miller, Critical Rationalism
25
Risk management
Harmless
effects
Harmful
effects
Range of effects
predicted by the ERA
26
Risk
Management
Harmless
effects
Harmful
effects
Range of effects predicted
after risk management
Risk management
● Uncertainty is often greatest early in the development of a GM crop
● Need field trials to gain familiarity with crop
● Need reassurance that harm will not occur
Guard rows
27
Isolation
Risk management
● May be used when ERA
would become too complex
● Gene flow from Bt to feral
cotton in the USA
● Uncertainty about the
consequences
● The US EPA imposed risk
management by banning
cultivation of Bt cotton in
parts of the USA and its
territories where feral cotton
grows
28
Feral cotton
Long-term effects
● Assessing long-term ecological effects of a transgenic crop is probably
fruitless if its short-term effects are determined to pose negligible risk
- No harmful unintended effects
- Trait is not toxic, allergenic etc.
● Policy, other new technology and farmers’ creativity are likely to be far
more important than any subtle, immediately harmless differences
between transgenic and non-transgenic crops
- Agri-environment schemes
- Subsidies
- Changing rotations (e.g., from spring- to winter-sown cereals)
- Changing public preferences (e.g., increased consumption of meat)
● The ERA cannot and should not try to predict the different ecological
effects of these factors on transgenic and non-transgenic crops
● The ERA should not be asked to resolve policy questions
29
Conclusions: uncertainty does not equal risk
● Currently, ERA seems beset by scientism
- “scientism [is] the notion that science gives us certain knowledge and
might even be able one day to give us settled answers to all our
legitimate questions” – Brian Magee, Popper
● Understates our knowledge of risk; and overstates its importance
● Definitions of harm and decision-making criteria reduce complexity
- Test hypotheses that make categorical predictions
- Avoid excessive quantification
- Case-by-case is not starting from scratch for each new crop
● Policy and judgement may be difficult, but can’t be replaced by science
● Uncertainty: make the decision right versus make the right decision
- Risk management and policy incentives; not a perfect ERA
- There may be unexpected benefits of proposed actions
30