Advantage Business Group
‘The Barbican’, East Street, Farnham, Surrey GU9 7TB 01252 738500
www.Advantage-Business.co.uk
Combining Cognitive Maps with other OR
methods to support Decision-Makers
ISMOR 21, September 2004
Colin Drysdale
Presentation Outline
Introduction
• Home Office – Crime Reduction Model
• MOD, Dstl – C3I system Benefits Model
– (Command Control Communications & Intelligence)
Final Discussion
Advantage Technical Consulting 2004
Slide 3
Introduction
Advantage Technical Consulting 2004
Slide 4
Decision making – common themes
• Governmental decision makers
• Account rationally for decisions
• Possibility of some form of ‘public’ scrutiny
Advantage Technical Consulting 2004
Slide 5
Case Study 1
Home Office – Crime Reduction Model
Advantage Technical Consulting 2004
Slide 6
Home Office – Crime Reduction Model
• Responsibilities include reducing the level
of crime in England and Wales
• Culture change
• ‘evidence based policy making’
• Justification
Advantage Technical Consulting 2004
Slide 7
Spending Review 2004
• Knowledge existed across a number of
groups
• Some of this was locked in experts’ minds
• Seemed like a job for SODA cognitive
mapping
– (Strategic Options Development and Analysis )
Advantage Technical Consulting 2004
Slide 8
Cognitive Mapping - Justification
Advantage Technical Consulting 2004
Slide 9
Cognitive Map - Concepts
Goal
Intermediate step
Example
Intervention
Advantage Technical Consulting 2004
List
Slide 10
Cognitive Map - Links
Causal
Negative Causal
Logical AND
&
Weak Link
Possible Link
&
Connotative
Correlation
Advantage Technical Consulting 2004
Slide 11
Cognitive Map - Constructs
Goal
Intervention
Intervention
Advantage Technical Consulting 2004
Slide 12
Constructs
Goal
Argument
Intervention
Intervention
Advantage Technical Consulting 2004
Slide 13
Constructs
Goal
Argument
Intervention
Intervention
Advantage Technical Consulting 2004
Evidence
Memo
Evidence summary and
URL link to supporting
Documentation in a pop-up box
Slide 14
Constructs
Goal
Argument + Evidence
Intervention
Intervention
Advantage Technical Consulting 2004
= Case
Memo
Evidence summary and
URL link to supporting
Documentation in a pop-up box
Slide 15
Approach
• ~30 Interviews
– Generated initial maps
• interviewees invited to discuss:
–
–
–
–
–
causes of crime
policy goals
interventions
…
evidence sources / quality
• Synthesis
– To a single map (~2400 - 1800 concepts)
Advantage Technical Consulting 2004
Slide 16
Initial benefits
• Knowledge model
• Single resource
• Linked to exiting documents
• Shows where more research & analysis is
required
Advantage Technical Consulting 2004
Slide 17
Next issue
• Interventions that should be funded ?
• Defines the “Problem”
• Time stepped simulation
Advantage Technical Consulting 2004
Slide 18
Further insights
• Exploit causal pathways
• Interventions to goals
• ~1800 concepts
• MS Excel with VBA tool developed
• Enabled analysis of
– Contradicting interventions
– Loops in the causal logic
– Completely dependent interventions
Advantage Technical Consulting 2004
Slide 19
Dependent interventions
Advantage Technical Consulting 2004
Slide 20
Findings
• Many logical anomalies
• different interpretations of “intervention”
• Showed that
– more clarity needed
– for quantitative modelling
Advantage Technical Consulting 2004
Slide 21
Case Study 1 – Further benefits
• Cognitive modelling helped define ‘the
problem’ and requirements for more
quantitative modelling
• In this role SODA cognitive mapping should
not be viewed ‘soft’ OR
– it enables a ‘soft-to-harder’ transition
Advantage Technical Consulting 2004
Slide 22
‘soft-to-harder’ methodologies
objective
subjective
Hard
Soft
qualitative
Advantage Technical Consulting 2004
quantitative
Slide 23
‘soft-to-harder’ methodologies
objective
subjective
Hard
Soft
qualitative
Advantage Technical Consulting 2004
quantitative
Slide 24
‘soft-to-harder’ methodologies
objective
subjective
Hard
Soft
qualitative
Advantage Technical Consulting 2004
quantitative
Slide 25
‘soft-to-harder’ methodologies
objective
subjective
Hard
Soft
qualitative
Advantage Technical Consulting 2004
quantitative
Slide 26
‘soft-to-harder’ methodologies
objective
subjective
Hard
tacit
Soft
explicit
qualitative
Advantage Technical Consulting 2004
quantitative
Slide 27
Case Study 2
MOD - C3I system Benefits Modelling
Advantage Technical Consulting 2004
Slide 28
Benefits Modelling
• Dstl name
• Causal mapping & MCDA
• C3I systems
Advantage Technical Consulting 2004
Slide 29
The Task
• Aim - Develop a C3I system benefits model
• Purpose – support contractor down-select &
OR programme definition
• Implementation – constrained to perform
calculations in Excel with VBA for ease of
maintenance in-house
Advantage Technical Consulting 2004
Slide 30
The concept
• SODA COGNITIVE maps contain CAUSAL logic
– and additional information
• MCDA networks for modelling C3I can get
complicated ….
Advantage Technical Consulting 2004
Slide 31
MCDA nets for C3I get complicated
Investments
(SRD / ADD
Level)
Investments
(URD Level)
Advantage Technical Consulting 2004
System
Attributes
Organisational
Attributes
Military
capabilities
Value System
Slide 32
The concept
• SODA cognitive maps contain causal logic
– and additional information
• MCDA networks for modelling C3I can get
complicated
• Can’t use a Commercial Off The Shelf (COTS)
MCDA tool
Advantage Technical Consulting 2004
Slide 33
The concept
• SODA cognitive maps contain causal logic
– and additional information
• MCDA networks for modelling C3I can get
complicated
• Can’t use a Commercial Off The Shelf (COTS)
MCDA tool
• Why not use the cognitive map as the front end to
an Excel with VBA MCDA model
Advantage Technical Consulting 2004
Slide 34
What happened
• Workshop - “false start” !
• Unavoidable part of the problem exploration
process
• How to recover
– Stay calm
– Needs skill and experience
Advantage Technical Consulting 2004
Slide 35
Recovery
• Method was unfamiliar but ….
– Resembled familiar information flows
• Solution was to exploit familiar information
flows
• Converted these “off-line”
• High level structure
Advantage Technical Consulting 2004
Slide 36
Benefits Modelling
MCDA
objective
Hard
causal
mapping
subjective
Soft
MCDA
qualitative
Advantage Technical Consulting 2004
quantitative
Slide 37
Final Discussion
Advantage Technical Consulting 2004
Slide 38
Final Discussion
• Take-up of Soft OR techniques has not
been universal
• Combining cognitive mapping with other
techniques
– quantified logical case
– pass independent scrutiny
• Both studies illustrate that cognitive
mapping supports problem formulation
• The second study illustrates that cognitive
mapping requires skill & experience
Advantage Technical Consulting 2004
Slide 39
References / Reading
• Description of the SODA technique see: “Rational Analysis for a
Problematic World”, Ed. J Rosenhead, Chapter 2, John Wiley & Sons,
1989.
• “Multiple Criteria Decision Analysis – Practically the only way to
choose”, V Belton, OR 32, Sep 1990, Published in “Operational
Research Tutorial Papers: 1990”
• “Benefits Analysis – A Robust Assessment Approach”, G Mathieson,
Dstl, 2002, Unclassified.
Advantage Technical Consulting 2004
Slide 40