What Makes A Good Model
in Economics and Finance
Ioanid Rosu
University of Chicago
ASE, December 21, 2006
Introduction

What is a good theoretical model in economics?
 Why
is it that most famous models in economics are
relatively simple?

We will not discuss here what makes a model
important (i.e. useful). Typically, such a model
 Explains
an important fact (e.g. the Phillips curve:
inflation vs. unemployment)
 Gives a framework to think about important issues (e.g.
Arrow’s social choice theory)
 Provides an important methodology or economic intuition
(e.g. Akerlof’s lemons model – asymmetric information)
Good = True = Credible

Instead, we want to know what a true model is



In general, a model:


Assumptions ) Conclusions (via causal chains)
Naively, one may propose two criteria for a credible
economic model:



In economics true is usually replaced by credible or appropriate
We do not test an economic model, but rather we validate it, i.e.
we decide whether it is credible or not
Internal consistency (mathematically true)
Verifiable conclusions (empirically true)
This works for physics


Assumptions and causal chains are very reliable (physical laws
and mathematical derivations)
Conclusions can be verified by controlled experiments
Economics vs. Physics

Can economics imitate physics? Some say no:


“Economics suffers from physics envy. […] We would love to
have 3 laws that explain 99 percent of economic behavior;
instead, we have about 99 laws that explain maybe 3 percent of
economic behavior!” (A.W. Lo)
Nevertheless, are economic assumptions reliable?


For example in many models agents are assumed to be rational,
even though there is ample evidence to the contrary
Milton Friedman (Nobel prize 1976): it doesn’t matter as long as
the conclusions are verified


E.g. agents are assumed to behave as if they were rational, even if
individually they can be irrational
And how about the structure of the model? Can we have
as many assumptions and causal chains as we want?

Mathematical economists seem to believe so
Why Economics Is Hard

In fact things are not so simple with economic models

Assumptions and causal chains are noisy, and controlled
experiments are rare (i.e. one cannot eliminate the outside noise)



Moreover, economic models themselves may affect human
behavior, which creates two sides of economics:




Each causal chain amplifies (compounds) the noise, so one must focus
only on a few causal chains
As a consequence, a typical economic model is not a complete
system, but only a window through which we study reality
Positive (describe reality as it is)
Normative (describe reality as it “should” be)
The economic truth is a partial, moving target
Often when a model fails empirically, economists still use
the model as a guide

They rely on the normative aspect: if only the economics agents
were rational and listened to us…
How to Think of Economic Models


Because of the noise, an economic model is not a
complete system (e.g. Newtonian mechanics), but a
window which connects a few economic variables
So we should think of a model as




A few causal chains between a small set of variables
A plausible explanation for a set of conclusions
Now, when we think of a good model, we want to know
whether the window through which we observe reality does
a credible job
Let us call an economic model credible (good) if



The assumptions are plausible, although possibly imperfect (noisy)
The explanation itself is plausible, i.e. the results are strong enough
to be distinguished from noise
The conclusions are not too sensitive to the assumptions
The 3R of Economic Research &
the ABC of Model Verification

The 3R of economic research: a good (credible)
economic model must be
1. Realistic
2. Relevant
3. Robust

Moreover, a (positive) economic model should be
4. Verifiable
The ABC of model verification: a good model has
A. Alternative
B. Byproducts
C. Calibration
Example: CAPM
(Capital Asset Pricing Model)

Assumptions
 Agents only care about risk and return
 Their utility depends only on the mean of the portfolio ( E )
and its volatility (  )
 Agents have the same beliefs about asset returns
 E(R) and  (R) are the same for all agents
 The market is in equilibrium

Conclusions
 Agents
invest the same relative percentages of their
wealth in the risky assets (stocks)

Everybody invests only in the riskless asset (= the “bond”)
and the collection of all stocks (= “the market”)
 Main
conclusion: the expected return of a stock
satisfies the CAPM formula
E(Ri) – Rf = i ( E(RM) – Rf )
The 3R of Economic Research


A model approximates reality, and tries to reliably explain an
economic phenomenon
Since we start with noisy assumptions and we draw noisy
inferences (causal chains), we need to make sure that
conclusions are not too noisy


In some sense, the noise of the conclusions equals the noise of
assumptions, compounded by the noise of the causal chain
The model therefore must be

Realistic: the assumptions are plausible, i.e. they are a good
approximation of reality


Relevant: The results are strong, i.e. first order


The causal chain noise outside the model is relatively small
Robust: The conclusions are not too sensitive to the assumptions


The assumption noise is small
The causal chain noise inside the model is small
Then the conclusion noise is also relatively small
1. Realistic


Assumptions must be realistic, i.e., they should represent a
reasonable approximation of reality
Contradiction with Friedman’s thesis?


Realistic assumptions are the foundation of good research: Suppose
you find a conclusion is false



No, because “plausible” is not the same as “true”. Assumptions need not
be true per se.
Then one of the assumptions is false
If the assumptions are not realistic (plausible), it is hard to learn from the
failure of the conclusion
Example: When testing CAPM, we find that agents are far from
holding the market portfolio (e.g. home bias)

Then one of the assumptions is false, but which one?
 Maybe that E(R) and  (R) are the same for all agents
 Since the assumption is plausible, it is clear how to modify it: e.g. agents
have different information about asset returns and volatilities
 The Black–Litterman model modifies exactly this assumption and
suggests how agents with different beliefs should invest
2. Relevant

The results must be relevant, i.e. the assumptions provide
a plausible explanation of the conclusions



Relevance is the key feature of a good model. Typically a
relevant model satisfies the 3S rule:





Since a model is only a window to reality, one has to make sure the
causal chains are strong enough compared with the outside noise
For example, many models can predict that in the summer it’s hot,
but do they provide a plausible explanation?
Has a story, i.e. a clear intuition behind it
Is simple: too many assumptions weaken the value of a conclusion
(each assumption comes with an inherent error)
The sizes implied by the model are plausible (e.g. from a
calibration)
Example: CAPM is relevant for explaining expected returns
by betas (Fama & MacBeth 1973)
Even if the CAPM formula today is empirically less true
(APT), the model remains relevant: the normative aspect!
3. Robust

A model is robust when small modifications of the
assumptions do not change conclusions in an essential way


It is very rare that assumptions are perfectly plausible, so we must
make sure that the inherent error in choosing them does not affect
the results significantly
Example: In one version of CAPM it is assumed that agents
have quadratic utility



This is a strong assumption: it implies a saturation point
Do the conclusions of CAPM depend essentially on this assumption?
No, the conclusions also hold if stock returns are normally distributed


In reality, the distribution of stock returns is not exactly normal, but has
“fat tails” (it is leptokurtic)
If we modify this assumption, it turns out that CAPM is still approximately
true
4. Verifiable


Karl Popper’s criterion: a (positive) scientific model must be
verifiable, i.e. falsifiable
Verification of a single conclusion can be done by

Intuition or anecdotal evidence: Is the conclusion reasonable? Do
we have arguments or examples that support it?


Statistics: Test the conclusion by bringing it to the data


E.g. in corporate finance (moral hazard, risk shifting)
E.g. to test CAPM, one can check whether a = 0 in the linear regression
E(Ri) – Rf = a + b £ i + ei
One can also check whether b = E(RM) – Rf
It is not enough to verify just one conclusion of the model.
One also needs to verify the model itself, since in
economics conclusions are model dependent

The ABC of model verification: a good model has
A. Alternative
B. Byproducts
C. Calibration
A. Alternative


One must have an alternative (H1) to the main
conclusion (H0) of a model
In many cases, the alternative is obtained by
negating H0
 Example:

for CAPM, H0: a = 0 , H1: a  0
Ideally, one would like to have an alternative
model, to provide an alternative explanation for
the main conclusion
 Example:
One can compare CAPM to the alternative
model, APT (Arbitrage Pricing Theory) in its Fama–
French version:
E(Ri) – Rf = i ( E(RM) – Rf ) + hi E(RHML) + si E(RSMB)
where HML is a factor that represents “value – growth”
firms, and SMB represents “small – big” firms
B. Byproducts

Besides an economic model’s main conclusion,
there are also secondary conclusions (byproducts)
 Since
other alternative models may have the same main
prediction, one can check that the given model provides
the correct explanation by also testing the byproducts


Example: The main conclusion of CAPM is that
expected stock returns only depend on betas
It also has the byproduct that agents invest in the
same way in stocks – the market portfolio
 This
may explain the popularity of index funds (low cost
copies of the market portfolio)
 But there are empirical problems (e.g. home bias)

Again, one can refer to the normative side of the model
C. Calibration

One should be able to calibrate the model (both
the assumptions and the conclusions)
 Many
models can predict that in the summer it is hot
 The question is: exactly how hot is it in the summer?
 Otherwise, we cannot tell if our explanation is correct
 Example: CAPM not only predicts that the higher the
risk – the higher the expected return, but also gives a
precise estimate of the expected return ( E(R) )

The parameters of the calibration should be
plausible
 For
this, it is important that the model be realistic
(otherwise, what does “plausible parameter” mean?)
In Conclusion

A good economic model should be (the 3R)
 Realistic
 Plausible assumptions
 Relevant
 Results should be of first order (plausible explanations)
 Robust
 Small changes in assumptions do not change conclusions in an
essential way
 Verifiable (the
 Alternative


Compare the given explanation with an alternative one
Byproducts


ABC of verifying a model)
The other conclusions of the model should also hold
Calibration

One should calibrate the model (with plausible parameters) and
compare it with the data