Applying the Systems Approach
Learning to Act Effectively in a Digital World1
Geoff McDonnell, Adaptive Care Systems and UNSW.
April 2012 Contact: [email protected]
Contents
Synopsis .............................................................................................................................. 1
Background: Real World Decision Making ........................................................................ 2
The Systems Approach ....................................................................................................... 2
Systemic Frameworks ..................................................................................................... 3
A Systemic Perspective............................................................................................... 3
Sociotechnical Systems ............................................................................................... 4
Socio-Ecological Systems ........................................................................................... 4
Structure Agency Framework ..................................................................................... 5
Critical Systems Thinking............................................................................................... 5
Systems Methods in Practice .............................................................................................. 6
Qualitative inquiry and reflection using a Systems Approach .................................... 8
Model Formulation and Simulation .......................................................................... 11
Simulation Testing and Evaluation ........................................................................... 12
Virtual Intervention Experiments ............................................................................. 14
Conclusion ........................................................................................................................ 18
References ......................................................................................................................... 21
Synopsis
Systems modelling is the application of a systemic perspective to conceptual mapping
and computer simulation to produce effective action. The most important component is
the initial qualitative reflection to select the scope, framing and relevant context to satisfy
the concerns of those involved and those affected This reflection produces a concept map
representation of these concerns and a clear statement of the variables of interest in the
language and mental models of the participants. Using both theory and data, this static
map can then be converted into a testable computational model of behavior of time.
Computer simulation is now a mature and powerful tool for modeling systems to test how
different factors may improve efficiency, effectiveness and equity in situations where it is
not possible to conduct real-world experiments. We can test whether a policy will
actually result in improvement in the right place and at the right time. Faced with future
uncertainties we need to perform in silico experiments to design and test policies that
cover a range of possible futures. These tools ensure policies are robust enough to
1
See http://insightmaker.com/insight/1121 for its application to eHealth
1
Real World Decision Making
Information
Individual & Collective
withstand
these future key uncertainties and
help shape the future towards those options
Decisions
Processes
that Task
deliverinteraction
more desirable and sustainableActions
ends for&citizens of the world.
Outcomes
Communication
Background:
CoordinationReal World Decision Making
Evidence
Theory
Decisions are the method by which managers turn information
into action. These actions
Data
produce outcomes, which change the information, resulting in further decisions. In joint
Experience
activities of informing, deciding and acting, these collective
processes require interaction,
Literature
communication and coordination. Information can exist in many forms, including data,
Shared
experience, empirical evidence, theory and relevant literature.
Again, in joint activities
people need to share or communicate information.
Decision makers use simplified mental models to make sense of the world. They match
perceived situations with these models and play out consequences of planned actions in
their heads before acting (Klein). Mental models include selective use of key information,
filtering of noise, and simplified heuristics or rules of thumb. For joint decisions, we need
to share views of the world, especially consensus about objectives and the consequences
of actions.
All decisions occur in a context, which includes physical and social features. This context
is the structure of the system, including the institutions and organisations, policies, social
norms, values and goals. This context provides physical resource and social constraints
on individual and group behaviours.
The Systems Approach
We adopt Systemic Frameworks and apply Critical Systems Thinking to real world
projects and programs using System Methods. The systems approach seeks to
understand interactions among peoples’ viewpoints, values, perceptions and beliefs, in
order to learn to take effective collective action.
It is an approach to problem solving, by viewing "problems" as parts of an overall
system, rather than reacting to specific parts, outcomes or events and potentially
contributing to further development of unintended consequences.
“The ultimate meaning of the systems approach lies in the creation of a theory of
deception and in a fuller understanding of the ways in which the human being can be
deceived about his world and in an interaction between these different (biased and
deceived) viewpoints. The systems approach begins when first you see the world through
the eyes of another
2
1. The systems approach goes on to discovering that every world view is terribly
restricted
2. There are no experts in the systems approach
3. The systems approach is not a bad idea.”
-C West Churchman The Systems Approach (p231)
Systemic Frameworks: A Systemic Perspective
A Systemic Perspective represents an iterative unfolding of understanding intended to
provide the basis for developing a strategy which, when implemented, is highly likely to
address the situation of interest as intended while minimizing the likelihood of
unintended consequences.
Key features of this way of thinking are:
Situation. A situation of interest considered to warrant attention, along with an
assessment of the implications of not acting, and a definition of the preferred
alternative situation, forms the basis for developing understanding.
Behavior. The patterns of behavior represent an unfolding of some aspects of a
network of interactions. As such we endeavor to understand the network of
interactions responsible for creating the patterns of behavior.
Interactions. The network of interactions is the result of some set of actions by
one or more stakeholders. As such we endeavor to understand the mental models
and motivations of the stakeholders responsible for the situation.
3
Stakeholders. We seek to understand the motivations and the mental models of
the stakeholders and the motivations and mental models of those stakeholders
who are influenced by the network of interactions.
Boundary. Based on an understanding of the network of interactions and
stakeholders boundaries are established to keep track of which stakeholders are
responsible for which aspects of the network of interactions and which set of
interactions are considered to be part of the addressable network of interaction.
Challenge Assumptions. It is essential that we challenge those assumptions
because decisions made on invalid assumptions are unlikely to support the
intended results.
Leverage Points. It is essential to identify those leverage points which are likely
to transform the current situation into the desired alternative situation.
Strategy. The strategy is developed with the intention of migrating the situation
of interest in the direction of the desired alternative situation
Unintended Consequences. Unintended consequences are typically the result of
actions taken without appropriate due systemic consideration.
-Gene Bellinger2
Sociotechnical Systems
This is an approach to complex organizational work design that recognizes the interaction
between people and technology in workplaces. The term also refers to the interaction
between society's complex infrastructures and human behavior.3
This broader approach, based on the work of Emery and Trist at the Tavistock Institute,
moves from an Input Process Output model of a single system to internal and external
interactions among a system of systems. Sociotechnical systems focus on the interactions
among people processes and technology and the surrounding organization and external
environment. It focusses on managing ongoing technology transitions due to successive
adoption of new, sometimes disruptive technologies. Formal models are used particularly
in designing and changing processes.4
Socio-Ecological Systems
A socio-ecological system consists of 'a bio-geo-physical' unit and its associated social
actors and institutions. Socio-ecological systems are complex and adaptive and delimited
by spatial or functional boundaries surrounding particular ecosystems and their problem
2
Systemswiki http://bit.ly/zfxzfx
3
http://en.wikipedia.org/wiki/Sociotechnical_systems
4
http://insightmaker.com/insight/1168
4
context. Change in these systems are driven by interacting nested fast and slow adaptive
cycles Adaptive management techniques acknowledge doubt and limits of control. This
learning while doing approach is referred to as reflexive governance.5
Structure Agency Framework
Agency refers to the capacity of individuals to act independently and to make their own
free choices. Structure, in contrast, refers to the recurrent patterned arrangements which
influence or limit the choices and opportunities available.
Social structures enable and constrain human agents, who produce and reproduce social
structure.6
Critical Systems Thinking
This system of systems (or creative holism) approach adopts a Critical Realist stance,
using the scientific method to test assumptions.
1. Critical Realism highlights a mind dependent aspect of the world, which reaches
to understand (and comes to understanding of) the mind independent world. A
systemic, dynamic and realist evaluation perspective in the pragmatic tradition.7
2. A science is a method of fixing belief, a persistent disinterested pursuit of truth.
3. It is a co-operative social venture, not an individual affair.
4. Its data must be obtained by some form of observation.
5. Its method of dealing with these data is that of rational or logical thought.
6. This includes deductive and synthetic inferences. Synthetic inferences are
induction (which relies on relevant representative sampling and self correction)
and abduction (the process of arriving at a scientific hypothesis. A surprising fact
is observed. But if a hypothesis were true, the fact would be a matter of course.
Hence, there is reason to suspect that the hypothesis is true).
7. Its conclusions must be verifiable by observation, experiment or both.
8. Its conclusions are intrinsically provisional and susceptible to further refinement
or correction as inquiry is continued.8
5
(see http://insightmaker.com/insight/1169 Panarchy Book http://bit.ly/zqWBwR and the
http://www.resalliance.org/ Website)
6
See http://insightmaker.com/insight/1163 and
http://en.wikipedia.org/wiki/Structure_and_agency
7
http://bit.ly/wDzynG
8
See http://insightmaker.com/insight/463
5
The key critical systems thinking competencies are Selecting Context, Clarifying Values
and Checking Reality.
This way of thinking admits that stakeholders, those involved and those affected, may
have different ways of making sense of the world, different values and therefore different
views of what facts are relevant, important and useful. It provides a formal, pluralistic
approach to deciding the scope and level of detail, which includes a broad range of
influences. It critically assesses the scope of inquiry (boundary critique) through
surfacing and reviewing boundary judgements, observations and concerns from multiple
perspectives.
It offers useful ways to help persuade stakeholders to start and continue to take collective
action.9
This attitude is also referred to as Creative holism (Jackson)
Critical systems thinking steps that are useful in argument include problem framing,
situation analysis, ideal mapping, choice of methods and evaluation, reflection on
methods and challenging of claims using valid argument structures (described by
Toulmin,10
Systems Methods in Practice
Applied systems methods combine qualitative and quantitative methods for implementing
a systems approach, including virtual intervention experiments using computer
simulation models.
In silico experiments can replace or extend studies that are impossible, too long, too late
or too expensive in the real world. They focus on improving real world decisions,
extending usual study methods and combining inductive, deductive and exploratory
theory development approaches (Axelrod).
9
See http://insightmaker.com/insight/1204
10
see http://insightmaker.com/insight/1175 )
6
Real World
Decision Making
Simulation
Experiments
in silico
Experiments
and Studies
in vivo and in vitro
Real world problems are perceived, studied and experimented on, and improvements are
designed and external world, the mental world and the social world.11 Virtual
experiments inform these real world approaches, especially where real world experiments
are impossible to implemented. In this way we learn to make sense of the world together.
The world consists of the perform due to cost, time, ethical or other reasons. In this
virtual world we make our mental models explicit, shareable and improvable by using
concept maps and computational model representations.12
Systems Simulation Method for Virtual Experiments
Based on commercial teaching and research experience with health systems simulation
projects over the past two decades, we have evolved a joint iterative development
approach using hybrid modeling, particularly System Dynamics and Agent based
modeling, available in a single commercial software package, AnyLogic or separate
specialized toolsets.
The method consists of 4 main stages of joint interactive development:
1. Qualitative inquiry and reflection
2. Model Formulation and Simulation
3. Simulation Testing & Evaluation
11
(Historical thesaurus http://bit.ly/yb4q4a )
12
See http://insightmaker.com/insight/806
7
4. Simulation Policy and Interaction Experiments
Systems Simulation Method
Simulation
Policy &
Interaction
Experiments
Simulation
Testing &
Evaluation
Model
Formulation
& Simulation
Qualitative
Reflection
Qualitative inquiry and reflection
This stage of the systems approach consists of identifying and defining the problem as a
series of key questions and conceptualising the subsequent model needed to answer these
key questions. In complex problems the key questions may be refined progressively as
the project proceeds and a deeper understanding emerges during subsequent analysis. The
aim is to share individual mental models through conversation, storytelling and argument
within communities of research and practice (Kim 1993). These ideas are expressed in
visual maps, images, recordings and narratives. Useful representations include system
maps, rich pictures, cognitive maps and concept maps with embedded pointers, images
and video clips. Much of this annotated discussion is now available online through
websites, blogs, discussion forums and wikis.
The steps within the conceptual modelling stage are:
1. Identify and define the opportunity for improvement
2. Conceptualise the Problem Situation
3. Elicit Mental Models
The output is expressed as annotated visual maps of linked concepts.
8
Identify and define the opportunity for improvement
This step makes the case for an improvement program, taking into account the level of
certainty, control and agreement among those involved and those affected. Here we use
the Critical Systems Heuristics approach.13
This is a System of Systems approach. It acknowledges that stakeholders, those involved
and those affected, have different ways of making sense of the world, different values and
therefore different views of what facts are relevant, important and useful. It provides a
formal pluralistic approach to deciding the scope and level of detail, which includes a
broad range of influences on how people are persuaded to take collective action.
Mapping the Boundaries of concern (from Boundary critique in CSH 12x2 questions)
About needs:
 What is the purpose?
 Who will benefit?
 How will improvement be measured?
About resourcing decisions
 What resources will be required to make the improvement?
 Who will decide to allocate and use the resources?
 Where and when will the decisions be made?
About knowledge sources
 What expertise will be required?
 Who will provide the required expertise?
 Who will verify the case?
About legitimacy (approval and formal agreement by those affected)
 How will the case be made legitimate to those involved and affected?
 How will those affected be involved (voice and power)?
 Who will verify the proposal for those affected?
 What viewpoints (worldviews) will be considered?
Conceptualise the Problem Situation
This step uses Ackoff’s Interactive Planning Steps adapted for the context and audience.
 Formulate the mess: What are the causes of performance?
 Ends planning: What are the goals of improvement?
 Means planning: What are the improvement interventions?
 Resource planning: What resource decisions are required?
 Implementation planning: How and when will the interventions proceed?
13
see http://insightmaker.com/insight/1218
9
Each phase includes the governing controls of the processes of agreeing, choosing,
cooperating, valuing the common good and ensuring competence.14
Visual metaphors, like Photovoice, are used to collect images that capture the meaning of
the situation for the participants An example is available at
http://insightmaker.com/insight/782
Typical policy questions about the future are:
What are current and future challenges?
What are the key policy levers?
What are the key measures of performance?
What are the probable alternative futures?
What are the important key uncertainties?
What are the likely scenarios?
In the classic system dynamics method, the entry point is a “puzzling dynamic” e.g. why
is cross city travel time worse despite billions spent on roads? Why do real estate prices
boom and bust (Sterman 2000)? In other problems we may start with describing the
current structure to explore and develop theories of behaviour; or to find ways to identify
and avoid unintended consequences of proposed policy changes.
Elicit Mental Models
This step aims to make the participants mental models explicit, shareable and improvable.
It answers the following questions.




What are the differences in mental models/beliefs that are contributing to the lack
of effective action or inability to learn?
What do the target audiences already know?
What do they need to know and when?
How will they learn it?
For risk messages the technique is well described by Morgan (2002).
1. Create an Expert model
Create a balanced legitimate focused and relevant expert model about the risk and
its management, summarized as a concept map showing factors that influence risk
management actions
2. Conduct mental model interviews(up to 1 hr each)
14
See http://insightmaker.com/insight/1263
10
3. Conduct 20-30 open ended interviews progressively shaped to cover the topics of
the expert model to elicit current beliefs. Include sorting pictures a mix of relevant
and irrelevant images/clips
4. Sort into misconceptions, peripheral (irrelevant) beliefs, over-general beliefs,
basic background beliefs not in expert models
5. Conduct Confirmatory surveys
Design, test, conduct and analyze
6. Estimate the relevant population prevalence of the beliefs among stakeholders
7. Design and test learning/communication strategy to correct/align the beliefs15
Output of qualitative reflection as Concept map:
The interim deliverable from the qualitative reflection phase is a concept map
which is used as the input to the next phase of computational model building.16
Model Formulation and Simulation
In this phase we build the computational dynamic model from the qualitative static
concept map. To do this we use a software package with appropriate visual
representations to explicitly model the time dimension. System Dynamics representations
include stocks (states), flows(transitions) and feedback loops. Agent based and analytical
models often use statecharts or mathematical formulae.17
Initially we adopt an aggregated top down view using SD methods, then bottom up agent
based if required for more detailed conceptualisation or if specific detailed data sets are
available. This combination of multilevel, multimethod approaches is possible due to the
object architecture of the simulation software (Borshchev 2004). Discrete event
simulation may be an appropriate starting point if most participants view the world as
“constrained randomness” or there are critical timing or scheduling events (Morecroft
2005).
The SD top down methods involve mapping the context using stocks and flows of key
items of interest in a system using a process view, together with the relevant connecting
information feedbacks and delays. This produces a graphical, logical structure (wiring)
diagram. The mathematical relationships among each of the components in the logical
structure are added or estimated from available data or best opinion. The behaviour of the
15
See MG Morgan 2002 ch 2-5 and p 77 for sample size and number of concepts
16
See http://bit.ly/zqJsNQ See general article http://bit.ly/zbaIqT Novak and dynamic feedback
Cyclic concept maps at http://bit.ly/zBL8Dq
17
See http://insightmaker.com/insight/1248
11
system over time is then displayed graphically by the computer simulation engine solving
a set of difference equations, using integration approximation and other numerical
analysis techniques.
The agent based simulation method uses object based analysis to define and describe the
detail of classes of interest and related functions, UML statecharts to describe their
dynamic behaviour and timers and messaging to specific coordination and interaction.
Our current practice is to translate our top down SD model into interacting sector objects
and, where required, replace some sector objects with more detailed agent based
representation. An example of detailed simulation tasks and interim deliverables for a
consulting project are in the Appendix (p 21)
Simulation Testing and Evaluation
George Box remarked, “All models are wrong, some are useful.” Therefore the
simulation model needs to be evaluated by assessing whether it is fit for its intended
purpose. During this phase the model is progressively refined over multiple iterations
with domain experts and differences between the predicted and observed historical
behaviours are detected and reconciled. Sterman introduces Validation and Model
Testing as Truth and Beauty (Sterman 2000 Ch21). He describes modelling as…” a
process of communication and persuasion among modellers, clients and other affected
parties. The real test is whether the model helps make better decisions. Therefore we
must assess the overall suitability of the model for its purpose, its conformance to
fundamental formulation principles, the sensitivity of results to uncertainty in
assumptions, and the integrity of the modelling process.” He goes on to list as series of
questions that need to be asked of any simulation model (Sterman 2000, table 21-1 p 852)
These address the following
 conceptual validity (can the model answer the questions asked?),
 structural and behavioural verification,
 simulation verification and
 pragmatics and politics of model use.
Conceptual validity questions include:
What is the purpose of the model?
What is the boundary of the model? Are the issues important to the
purpose treated endogenously? What important variables and issues are
exogenous or excluded? Are important variables excluded because there
are no numerical data to quantify them? What is the time horizon relevant
to the problem? Does the model include the factors that may change
12
significantly over the relevant time horizon as endogenous elements? Is
the level of aggregation consistent with the purpose?
Structural and behavioural verification questions are:
Does the model conform to basic physical laws e.g. conservation of matter Are the
equations dimensionally consistent without fudge factors. Is the stock and flow structure
explicit and consistent with the model purpose?
Does the model represent disequilibrium dynamics or does it assume the
system is in or near equilibrium all the time?
Are appropriate time delays constraints and possible bottlenecks taken into
account?
Are people assumed to act rationally and to optimise their performance?
Does the model account for cognitive limitations, organisational realities
non-economic motives and political factors
Are the simulated decisions based on information the real decisionmakers
actually have? Does the model account for delays distortions and noise in
information flows?
Simulation verification questions:
Is the model robust in the face of extreme variations in input conditions or
policies?
Are the policy recommendations sensitive to plausible variations in
assumptions including assumptions about parameters aggregation and
model boundary?
Questions on the Pragmatics and Politics of Model Use
Is the model documented? Is it publically available? Can you run your
model on your own computer?
What types of data were used to develop and test the model (e.g. aggregate
statistics collected by 3rd parties, primary data sources, observational and
field- based qualitative data, archival materials, interviews)?
How do the modelers describe the process they used to test and build
confidence in the model? (See Oliva, 2003). Did critics and independent
third parties review the model?
Are the results of the model reproducible? Are the results “add-factored”
or otherwise fudged by the modeler?
How much does it cost to run the model? Does the budget permit adequate
sensitivity testing?
How long does it take to revise and update the model?
Is the model being operated by its designers or third parties?
13
What are the biases, ideologies and political agendas of the modelers and
clients? How might these biases affect the results both deliberately and
inadvertently?
John Morecroft summarises these model tests as
Tests of Behaviour
Tests of Structure
Tests of Learning
Virtual Intervention Experiments
In this phase we design and perform virtual experiments to test the effects of our planned
interventions. These experiments may range from formal sensitivity analysis using Monte
Carlo methods or Optimizers, Laboratory or Web-based Learning Environments, or
computer and traditional board games.
Policy design ranges from changing parameter values to creating entirely new strategies,
structures and decision rules. These include changing time delays and the flow and
quality of information available at key decision points or fundamentally re-inventing the
14
decision processes of the actors in the system (Sterman). Lyneis(1988) describes
common mistakes in corporate policy as
.
.
.
policy isolated by functional area. This is like having three pilots
simultaneously trying to fly a single plane using three sets of controls. When
these problems are fixed in systemic structures, three different cockpits are
built into a single plane!
policy isolated from establishing goals (akin to no agreed flight destination, so
pilots are continually re-setting the flight destination)
policy design isolated from context (like ignoring the weather outside when
planning the flight).
Other problems with policy can be related to complexity, due to interactions among parts
and the environment, interactions being often more important than components and
because actions taken to correct an immediate problem may make matters worse in the
future.
The policy design approach in service industries involves the following:
 Represent the common service demand pattern;
 Describe how information about demand and capacity are used, how resourcing
goals and staffing levels are set and how discrepancies between goals and actual
are corrected.
 Identify problem behaviour
 Construct a computer model of the relationships believed to cause the behaviour
 Develop an understanding between structure and behaviour
 Design policies that improve behaviour
 Test the policies under a wide range of market demand patterns and varying other
key assumptions and scenarios
Testing policy can also use standard engineering variations in the system input pattern of
demand, such as: Step, Noise, Seasonal, Cyclical, Growth, Decline.
Scenario Planning and Robust Policy Design
In any simulation of the future there will always be important key uncertainties.
Therefore, we need to perform in silico experiments to design and test policies that cover
a range of possible futures. These tests ensure policies are robust enough to withstand
these future key uncertainties and help shape the future towards those options that deliver
more desirable and sustainable health care futures for the widest range of citizens.
15
Policy testing involves systematically varying interactions especially among structures,
policies, system constraints and individual behaviours (Rouwette). Behaviour sensitivity
is the policy outcome sensitivity to unknown parameter values. Policy sensitivity
analysis tests the assumptions of the method, the influence of the level of aggregation of
the model, and the assumptions of non-linear economic relationships. Stochastic
optimisation of policy space is performed if an agreed objective function exists
(Moxnes). If not, we must represent the various tradeoffs of policies among key
performance dimensions, such as cost, efficiency, equity and effectiveness.
Simulation as a Collaboration Artefact to share and improve mental model
Group model building and policy design and testing can help build consensus about
goals, relevant context, and policy impacts (Zagonel 2003). These group interactions can
take place as computer games or board games based on the structural system constraints
and policy rules of the game. Significant improvements in user interfaces and dynamic
graphic animations allow a wider range of non-technical participants in these group
activities, including patients families, interested and citizens and political interests.
Policy Flight Simulator Experiments to Evaluate the Impact of Systems Simulation
on Decision makers
Widely accepted game and policy simulators have been used to provide a standard test
environment for more controlled experiments about decision-making influences on
participants. The most common of these games is the classical supply chain Beer Game,
which explores ordering behaviour associated with delivery delays (Croson et al).
16
Planning the Policy Experiments using the Simulator
This follows the approach to more classical behavioral experiments in controlled
environments. These experiments are being focussed on the key research questions in
simulator use: How can we improve the mental models of decision-makers and will this
result in better policies?
Study designs include baseline and repeat measures pre-simulator and early and late postsimulator flight training, with controlled variations in the influences under test, to test
specific hypotheses (Gary).
Decision-making influence areas for policy flight simulator experiments
There is an opportunity for many observational and controlled experiments with
hypothesised influence variables. Some design questions are:
 What are the observed roles and decision patterns?
 Who are the agents playing the model?
 What are the key decision patterns about overall flows? Who makes them? How?
Why?
 What are the local social and cultural influences, performance metrics,
information feedbacks and ideologies in use?
 What are the effects of service availability and policies, resource constraints and
incentives especially financial?
 What are the relevant political dimensions (ownership, knowledge, collective
action resources (access to formal power, informal influence, identity, cohesion
and trust, social networks)?
Performing Policy Experiments/Serious Play (mixed in vitro and in silico tests)
Some future research questions about the interaction between individual decision-making
and policies in complex systems:
 What are the key decisions about individuals (which patient bed services etc)?
 Who makes them? How? Why?
 What are the heuristics and objective functions used?
 What are the hypotheses about agents' roles in playing the simulation/game?
 What are the behavioural influences such as lack of knowledge and lack of trust?
 What are the differences between novices and experts?
 How do Computer simulation and board game interactions differ?
 What are the effects of differences in individual learning styles?
 How and when are mental models malleable ?
 What are the quantitative impacts of known confounders such as cognitive ability
and complexity of task?
 How effective are group simulations in conflict resolution and building
consensus?
17
Progressively expanding and testing the scope of the model to include individual
behaviours
Agent based representation make it possible to incorporate the subject behaviours
observed in simulation and game experiments into successive versions to provide more
flexible and complete decision-making and learning environments. This concept is
similar to having the simulator play the roles of individual decision-making participants
(like a chess game) or having the audience “step into” a play.
Conclusion
Systems simulation, using multilevel multimethod approaches, is being applied to
improve the understanding of complex systems in science, biology, engineering, business
and health (Kitano). In silico policy experiments extend the scientific method used for in
vitro and in vivo testing, particularly when rigorous experiments are used in designing
and testing policy flight simulators. These simulated learning environments can be
progressively refined to provide more realistic decision-making challenges. We need to
test how these broader simulations can improve mental models in various contexts and
improve both systems design and individual contexts.
Benefits of these system simulation tools include:
·Helping to define the relevant system and its boundaries;
·Combining high-level system structures with detailed individual behaviours;
·Demonstrating dynamic behaviour of a system by playing out the logic,
relationships and feedback loops that are built into a system;
·Playing out long-time spans quickly and predicting the performance of the system
over time;
·Promoting cross-functional understanding, team building & organisational
learning;
·Helping identify potential unintended consequences and developing strategies so
that they can be avoided or their impact mitigated;
·Enabling search for high leverage points in a risk free environment; and
·Guiding additional data collection by showing where estimates need to be most
precise.
We are beginning to use these tools for sustainable health system design by solving
relevant, focussed, health policy problems. In the future we hope to collaborate with
teams that can integrate several system and discipline areas in more compelling and
realistic versions. Current application areas include the acute aged care interface, the
health workforce, technology diffusion and the use of information and communications
18
technology. Ongoing research and development work with health policy makers and
clinical decision-makers should turn health systems simulation into a useful, practical
toolkit for health systems improvement.
Philosophical Note (Olaya 2009)
This explicit representation of our mental models in SD has been described as presentationalism,
idealism, a search for positive knowledge within an idealist epistemology, instrumentalism "not
claims about the world but instruments for systematizing observations and for boosting learning
processes using experimentation via simulation", and relativism rather than positivism.
To summarise
Systems Methods and Models focus on better policies and explanations of problems
explain causal structure of a problem or situation
explain how the problem is created by this structure
explain why one policy has high impact while others do not
explain how established and defended policies are the underlying cause of the problem
behaviour
argue in favour of better policies
Formal computer models are constructed following the scientific method
reference mode of problem behaviour,
dynamic hypothesis,
formal model,
testing of the hypothesis against data,
extensive analysis, and
policy design.
A full page table of activities, representations at each stage and information sources
follows. Diagram examples are taken from http://insightmaker.com/insight/1400
19
20
Appendix
Example of Detailed Simulation Tasks and Interim Deliverables for a Consulting Project
Qualitative Reflection Phase
Confirm Scope, Context and Level of Detail
What questions does the model need to answer?
Deliverable: Issue Diagram
What Data Sources are available?
Deliverable: Documented Data Sources
What are the key System Performance measures and techniques?
Deliverable: The Case for Improvement including Key Performance Indicators
(KPIs) and the Boundary of concerns
Deliverable: Concept map
Develop Initial SD System Simulation using Ithink software and method (Richmond)
Convert to Object Sectors in AnyLogic
Add agent-based AB detail to selected sectors
Define Object Classes and Variables using class diagrams
Define Dynamic Behaviour of Objects using UMLstatecharts
Develop AB Simulation Logic using logic functions, agents and actions
Develop Initial Animation Display using animation objects
Calibrate Model against Existing System Performance
Deliverable: Calibrated simulation and Documented Data Gaps
(Calibration is from Existing Data Sources, Delphi best estimates, and special surveys if
indicated by sensitivity analysis)
Refine Model using Joint Interactive Group Modelling.
Develop and Simulate Future Case Scenarios
Deliverable: Model with what-if Simulations
Select Scenarios (workshop)
Extend model to handle scenarios
Refine Animation and Interactive User Controls
Refine Calibration
Data Sources for Parameter Estimates
There is a wide range of sources for estimating parameters in simulation models;
these estimates can be progressively refined, where possible, over the life of the
model. Sources we have used in previous projects include:
Published literature in journals and conference papers;
Publically available data from ABS and AIHW;
Internal data from organisations;
Focussed studies conducted by special interest groups;
Commonwealth and State Health Data Sets; and
Estimates from other relevant modelling and simulation studies performed by
Government departments, academics and modelling organisations such as
NATSEM, Access Economics, IPART, Productivity Commission, CSIRO,
HILDA and the like.
21
References
Axelrod, Robert. (2003) Advancing the Art of Simulation in the Social Sciences
UMICH Aug 2003
http://www-personal.umich.edu/~axe/research/AdvancingArtSim2003.pdf
Axelrod Robert and Tesfatsion Leigh. A GUIDE FOR NEWCOMERS TO AGENTBASED MODELING IN THE SOCIAL SCIENCES. To appear in the Handbook
of Computational Economics, Vol. 2: Agent-Based Computational Economics,
edited by Leigh Tesfatsion and Kenneth L. Judd, Handbooks in Economics Series,
North-Holland, Amsterdam, the Netherlands. available at
http://www.econ.iastate.edu/tesfatsi/abmread.htm
Borshchev, Andrei & Filippov, Alexei. From System Dynamics and Discrete Event to
Practical Agent Based Modeling: Reasons, Techniques, Tools
www.xjtek.com/download/papers
Churchman CW (1979) The Systems Approach. Delacorte Press, New York. Paperback
edition Dell Publishing, New York, 1969, second, revised ed.
Croson, Rachael, Donohue, Karen, Katok, Elena and Sterman, John. Order Stability in
Supply Chains: Coordination Risk and the Role of Coordination Stock
http://web.mit.edu/jsterman/www/Order_stability.html
Eddy, David. Archimedes Model http://www.archimedesmodel.com/archimedes.htm
Forrester, Jay W. (1980). Information Sources for Modeling the National Economy.
Journal of the American Statistical Association 75(371): 555-574
Forrester, Jay W. (1974) Understanding Social and Economic Change in the US.
Luncheon Address Summer Simulation Conference 1974 MIT D-memo 2103
Gary, Shayne and Wood, Robert.(2005) Mental Models, Decision Making, and
Performance in Complex Tasks, Abstract Paper ISDC Boston July 2005
Goude, Thomas A (1950) The Thought of C S Peirce University of Toronto Press,
Toronto
Gundersen, Lance and Holling C S (2001) Panarchy: Understanding Transformations in
Systems of Humans and Nature. Island Press
Homer, Jack B and Hirsch Gary B. (2006) System Dynamics Modeling for Public Health:
Background and Opportunities. Am J Public Health. March; 96(3): 452–458.
Jackson MC (2003) Systems Thinking: Creative holism for managers. Wiley
Kim, Daniel H (1993) The Link Between Individual and Organizational Learning MIT
Sloan Management Review; Fall 1993; 35, 1 p37-50
Kitano, Hiroaki. (2002) Systems Biology: A Brief Overview. Science 295, 5560, 1662-4
Ison, R.L., (2008). Systems thinking and practice for action research, in Reason, P.,
Bradbury, H. (Eds.), The Sage Handbook of Action Research Participative
Inquiry and Practice (2nd ed.), London, Sage Publications, 139-158.
Klein, Gary (1999). Sources of Power: How people make decisions MIT Press, ISBN
0262611465 www.decisionmaking.com
Lane, David C. (1999) Social theory and system dynamics practice. European Journal of
Operational Research. 113 501-527.
22
Lyneis, James M (1988) Corporate Planning and Policy Design A SD Approach PughRoberts Assoc Cambridge MA ISBN 0-262-12083-6
Meadows, Donella. The Unavoidable a Priori. D-4880 MIT D-memo
Meadows, Donella H (2008) Thinking in Systems a primer. Chelsea Green Publishing,
Vermont
Morecroft, John with Robinson, Stewart. (2005) Explaining Puzzling Dynamics:
Comparing the Use of System Dynamics and Discrete-Event Simulation. ISDC
Boston July 2005
Morecroft, John (2007) Strategic Modelling and Business Dynamics. Wiley
Morgan, M Granger, Fischoff B Bostrom A and Atman CJ (2002) Risk communication: a
mental models approach Cambridge University Press Cambridge UK
Moxnes, Erling (2005) Policy sensitivity analysis: simple versus complex fishery models
Syst. Dyn. Rev. 21, 123-145,
Novak JD and Gowin DB (1984). Learning How to Learn. Cambridge University Press
Cambridge
Olaya, Camilo (2009) System Dynamics: Philosophical Background and Underpinnings
Encyclopedia of Complexity and Systems Science Springer Volume 9 p9057-9078
Oliver, Adam and Mossialos, Elias. European Health Systems Reforms: Looking (2005)
Backward to See Forward? Journal of Health Politics, Policy and Law, Vol. 30,
Nos. 1–2, February–April. p7-28
Oliva R. (2003). Model calibration as a testing strategy for system dynamics models.
European Journal of Operational Research 151(3):552-568.
http://iops.tamu.edu/faculty/roliva/research/sd/calibration.html
Reid, Proctor P. Compton, W. Dale Jerome H. Grossman, and Gary Fanjiang, Editors,
(2005) Committee on Engineering and the Health Care System, National
Academy of Engineering, Institute of Medicine Building a Better Delivery
System: A New Engineering/Health Care Partnership. National Academies Press
http://www.nap.edu/catalog/11378.html
Richmond, Barry. ithink User’s Manual. High Performance Systems, Hanover NH
www.hps-inc.com
Rittberger, Berthold. (2003) Endogenizing institutional change: Moving beyond the
institutionalist ‘holy trinity’ Paper prepared for panel 12-9 ‘Beyond
Institutionalism’ at the 2nd General Conference of the European Consortium for
Political Research, 19-21 September 2003, Marburg.
Rouwette E, Grossler M and Vennix J. (2004) Exploring Influencing Factors on
Rationality: A Literature Review of Dynamic Decision-Making Studies in System
Dynamics. Systems Research and Behavioral Science 21, 351-370
Sterman, John D (2001) System Dynamics Modelling: Tools for Learning in a Complex
World California Mgt Review 2001 43(4) 8-25 and 1994
Sterman, John D Business Dynamics: Systems Thinking and Modeling for a Complex
World. McGraw-Hill 2000 ISBN 0-07-231135-5 www.mhhe.com
Toulmin, S. (1958) The Uses of Argument Cambridge University Press, 2nd edition 2003
23
Towill, Denis R. (1996) "Industrial dynamics modelling of supply chains", Logistics
Information Management, Vol. 9 Iss: 4, pp.43 - 56
Ulrich W (2012) Operational research and critical systems thinking—an integrated
perspective Part 1: OR as applied systems thinking Journal of the Operational
Research Society advance online publication, 14 December 2011
doi:10.1057/jors.2011.141
Ulrich W (2012) Operational research and critical systems thinking—an integrated
perspective Part 2: OR as argumentative practice Journal of the Operational
Research Society advance online publication, 14 December 2011
doi:10.1057/jors.2011.145
Zagonel, Aldo. (2002) Conceptualization in Group Model Building…Tension between
Representing Reality and Negotiating a Social Order. ISDC Palermo July 2002
http://www.systemdynamics.org/conf2002/proceedings/308Zagonel.pdf
24