Some simple ways to better understand
economic modelling
In our last article on modelling we defended the art of
economic modelling. We identified why purchasers,
users and readers have an important role in ensuring
that the outcomes from modelling are reasonable,
transparent and understandable - and how this may
even lead to better decision making and better public
policy outcomes.
The most obvious of these was the suggestion that the
estimated 7,200 fewer homes that would be built
would reduce GDP by $19 billion. A simple calculation
shows that this implies that every additional house or
apartment constructed in Australia would increase the
size of the economy by $2.6 million, much more than
could reasonably be expected. 3
To follow up on this theme, we’ve put together five
simple tips that can help you to become a more
informed reader of economic modelling.
There can be very good reasons why modelling results
are much larger (or smaller) than expected. However,
unless the reasons for this are clearly spelt out, the
reader would do best to treat the results with
suspicion.
While there is no substitute for expert advice, there are
some simple things to keep in mind that will go a long
way to helping you make an informed assessment of
an economic modelling report.
Tip 1
Brisbane
Level 8, 10 Felix Street
Brisbane Qld 4000
GPO Box 2605
Brisbane Qld 4001
P 61 7 3227 9500
F 61 7 3221 0404
Melbourne
Level 16, 330 Collins Street
Melbourne Vic 3000
P 61 3 9670 3792
F 61 7 3221 0404
Perth
Suite 7, 272 Hay Street
Subiaco, WA 6008
If it sounds too good (or bad) to be
true, it probably is (the sniff test).
It is unusual for economic impacts to be multiples of
the direct impacts. These generally only occur where
extreme economic conditions prevail (such as where
there are absolute rigidities in markets or very high
levels of unemployment). These extreme economic
conditions have not been observed in Australia for
decades. For this reason it should be unusual for
models to produce net economic impacts that are
multiples of the direct impacts.1 Where they do, it is
often worth questioning if the assumptions
underpinning the results are valid.
Often, the modelling results can be tested using
simple calculations.
Although we don’t want to enter into the debate about
the pros and cons of changes to negative gearing, the
BIS shrapnel report released on this issue provides a
very good case in point. As was pointed out by John
Daly from the Grattan Institute2, the modelling in this
report had a number of glaring inconsistencies that
should have rung alarm bells for readers.
P 61 407 348 612
F 61 7 3221 0404
Sydney
Level 26,
Market
Street
Level
16,44
330
Collins
Street
Sydney NSWVIC
2000
Melbourne
3000
P 61 2 9089 8666
PF 61
9670 8989
3792
61 3
2 9089
F 61 7 3221 0404
Tip 2
Most discussion around economics is concerned with
jobs and growth (as measured by gross domestic
product or GDP). However, for many policy options
these are the wrong measures, and can provide a
misleading picture of the veracity of the benefits or
costs of policy change.
This is because employment and GDP are measures
of production (how hard we work) rather than a
measure of the benefits we receive from this work
(income). See our article on income and GDP for
more on this.
Take emissions trading policy. Under most modelled
scenarios with global trading, Australia would become
a net purchasers of emissions permits from overseas
(since we have relatively high emissions intensities
due our reliance on abundant reserves of coal and
gas). These foreign permit purchases deduct from the
incomes earned in Australia, but are not counted in
GDP. That means that emissions trading policies
might have only a very small impact on GDP, but have
a much bigger impact on incomes.
Better measures of welfare are real Gross National
3
1
Although there is no hard and fast rule, be suspicious if the
net impact is more than 1.5-2 times the direct impact.
2
https://grattan.edu.au/news/why-it-is-good-policy-not-badpolitics-to-ignore-bad-modelling-on-negative-gearing/
Ensure you understand which unit of
measure is important for assessing
the policy or event being modelled.
The contribution to GDP from constructing a house should be
less than the cost of construction since any imported
components (including those along the supply chain) should
be netted out.
Product (GNP) as opposed to gross domestic product
(the most frequently quoted parameter), which
accounts for income flows to foreigners and real Gross
National Income (GNI), which accounts for both
foreign incomes flows and differences between the
prices of goods consumed and goods produced
(important when policies impact on prices4).
Tip 3
Read the fine print.
Often the key drivers of change will be hidden deep in
a modelling report. This means that most often, it is
not possible to understand the results simply by
reading the executive summary of a report or its’
associated press release.
Often a reading of the modelling detail will throw up an
obvious flaw or identify a key assumption that has
been used to drive results. Most credible modelling will
discuss uncertainties and key assumptions upfront
and may even decompose the results into the key
factors driving outcomes. Where this detail isn’t
available, treat the results with caution or seek out
answers.
Tip 4
Brisbane
Level 8, 10 Felix Street
Brisbane Qld 4000
GPO Box 2605
Brisbane Qld 4001
P 61 7 3227 9500
F 61 7 3221 0404
Melbourne
Level 16, 330 Collins Street
Melbourne Vic 3000
P 61 3 9670 3792
F 61 7 3221 0404
Perth
Suite 7, 272 Hay Street
Subiaco, WA 6008
P 61 407 348 612
F 61 7 3221 0404
Sydney
Recognise that economic modelling
is a simplification of the real world.
All modelling is a simplification – after all the real world
is a complex place, and even very detailed models are
not capable of replicating all of this complexity.
This means that different models may give different
answers to the same question. While this may seem a
great weakness – it is actually a strength. Rather than
showing that the modellers don’t know what they are
talking about, different results may reflect that we
simply don’t know precisely what the impacts of a
policy change will be, or there is significant uncertainty
around the key parameters that influence results.
There is often value in understanding these
differences as they can illuminate the contentious
issues under consideration.
Credible modelling makes these uncertainties
transparent and run sensitivity tests to demonstrate
how varying these parameters might affect the results.
Where this analysis is missing or the uncertainties
faced by the modeller are not clear, question whether
you have the full picture.
Tip 5
Recognise that there are some
basic realities that all reasonable
economic modelling should stick to.
These include:
•
The economy operates within constraints. At the
very least economic modelling should consider the
limited supplies of skilled workers and equipment,
but might also consider how markets are
constrained by transport networks or how
migration is constrained by preferences.
•
The economy is dynamic. Firms and workers
respond to incentives (such as prices and wages)
and the market usually finds solutions to complex
problems.
•
Government expenditures have to be paid for.
Modelling should fully account for this and avoid
the modelling trick of assuming that governments
are able to borrow with no cost to taxpayers.
These realities mean that it is not possible to generate
long lasting employment gains by simply spending
more money. It also means that the net economic
impacts of Government spending will often less than
the direct expenditures (sometimes significantly so) –
so that a million dollars of expenditure may very well
generate less than one million dollars of economic
activity (implying there should be a non-economic
rationale for the expenditure). A good example of this
would be the Rudd Government’s insulation program.
Finally, remember that it is an economist’s job to
explain the results of their complex models in a way
that people can understand. As Einstein once said if
you can’t explain it to a six year old, you clearly don’t
understand it yourself. While many economic issues
are complex, there is never a reason why they can’t be
clearly explained to an informed adult.
Euan Morton
Principal
Matt Clark
Manager
p: +61 7 3227 9556
e: [email protected]
p: +61 7 3227 9571
e: [email protected]
Level 26,
Market
Street
Level
16,44
330
Collins
Street
Sydney NSWVIC
2000
Melbourne
3000
P 61 2 9089 8666
PF 61
9670 8989
3792
61 3
2 9089
F 61 7 3221 0404
4
This is important when considering trade policies, such as
subsidies to exports. These subsidies might increase
production, but are unlikely to result in an increase in real
income since they cause export prices to fall at the expense
of production for domestic use.