Agent-Based Financial Modelling: A Promising Alternative to the Standard Representative-Agent Approach / T. Ramanauskas
WORKING PAPER SERIES
No 3 / 2009
AGENT-BASED FINANCIAL MODELLING:
A PROMISING ALTERNATIVE
TO THE STANDARD
REPRESENTATIVE-AGENT APPROACH
By Tomas Ramanauskas
1
WORKING PAPER SERIES
No 3 / 2009
AGENT-BASED FINANCIAL MODELLING:
A PROMISING ALTERNATIVE TO THE STANDARD
REPRESENTATIVE-AGENT APPROACH
By Tomas Ramanauskas*
* Bank of Lithuania, Vilnius Gediminas Technical University
For correspondence: Economics Department, Bank of Lithuania,
Totorių g. 4, LT-01121 Vilnius, Lithuania; e-mail: [email protected]
© Lietuvos bankas, 2009
Reproduction for educational and non-commercial purposes is
permitted provided that the source is acknowledged.
Address
Totorių g. 4
LT-01121 Vilnius
Lithuania
Telephone +370 5 268 0132
Fax +370 5 212 4423
Internet
http://www.lb.lt
Working Papers describe research in progress by the author(s) and
are published to stimulate discussion and critical comments.
The Series is managed by Economic Research Division of Economics
Department
All Working Papers of the Series are refereed by internal and
external experts
The views expressed are those of the author(s) and do not
necessarily represent those of the Bank of Lithuania.
ISSN 2029-0446 (ONLINE)
Abstract / Santrauka....................................................................................................4
1. Introduction.............................................................................................................5
2.Agent-based financial models versus the standard
representative-agent paradigm..............................................................................6
2.1.General discussion of homogeneity, perfect rationality and
market efficiency................................................................................................6
2.2.The new paradigm – markets as complex agent-based systems.........................8
3. Brief ASM literature review and important ASM design issues.......................11
3.1.ASM building principles and problems............................................................11
3.2.Brief ASM literature review.............................................................................13
3.2.1.ASM models based on random, heuristic and
hard-wired behavioural rules..................................................................13
3.2.2.ASM models based on intelligent adaptation .........................................14
4. Reinforcement learning in the context of ASM modelling................................16
5. Concluding remarks..............................................................................................19
References...................................................................................................................21
List of Bank of Lithuania Working Papers..............................................................25
Agent-Based Financial Modelling: A Promising Alternative to the Standard Representative-Agent Approach / T. Ramanauskas
Contents
3
Bank of Lithuania Working Paper Series No 3 / 2009
Abstract
In this paper we provide a brief introduction to the literature on agent-based financial
modelling and, more specifically, artificial stock market modelling. In the selective
literature review two broad categories of artificial stock market models are discussed:
models based on hard-wired rules and models with learning and systemic adaptation.
The paper discusses pros and cons of agent-based financial modelling as opposed to the
standard representative-agent approach. We advocate the need for the proper account
of market complexity, agent heterogeneity, bounded rationality and adaptive (though
not simplistic) expectations in financial modelling. We also argue that intelligent
adaptation in highly uncertain environment is key to understanding actual financial
market behaviour and we resort to a specific area of artificial intelligence theory,
namely reinforcement learning, as one plausible and economically appealing algorithm
of adaptation and learning.
Keywords: Agent-based financial modelling, artificial stock market, complex dynamical
system, market efficiency agent heterogeneity, reinforcement learning.
JEL classification: G10, G11, G14, Y20.
Santrauka
Šiame straipsnyje trumpai pristatoma agentų elgsena pagrįsto finansinio modeliavimo
literatūra, didžiausią dėmesį skiriant dirbtinės finansų rinkos modeliams. Atrankioje
apžvalgoje pateikiami modeliai suskirstyti į dvi bendras kategorijas: tai – modeliai,
kuriuose agentų elsena pagrįsta aiškiai nustatytomis taisyklėmis, bei modeliai,
pasižymintys agentų mokymusi ir sistemos lygmens adaptacija. Straipsnyje išreiškiama
nuostata, kad kuriant finansinius modelius reikia tinkamai atsižvelgti į rinkos
kompleksiškumą, agentų heterogeniškumą, ribotą racionalumą ir adaptyvius (tačiau ne
pernelyg primityvius) lūkesčius. Čia taip pat argumentuojama, kad protinga adaptacija
dideliu neabrėžtumu pasižyminčioje aplinkoje yra esminis modeliavimo aspektas, kuris
padėtų suvokti svarbiausius finansų rinkų funkcionavimo principus. Tuo tikslu šiame
pradiniame susijusių tyrimų etape trumpai idėjiniu lygmeniu nagrinėjama konkreti
dirbtinės intelekto teorijos sritis – paskatinamasis mokymasis, kuris iš principo galėtų
pasiūlyti ekonomiškai priimtinus mokymosi ir sisteminės adaptacijos algoritmus.
4
Introduction
In 2008 the world economy encountered a global financial crisis, caused by a mix of a
global asset price bubble, overwhelming irrational exuberance and systemic mistakes
of economic agents and financial market participants. Against this background,
the standard financial theory, based on the efficient market hypothesis and rational
representative agent paradigm, seems to be losing touch with reality. Unfortunately,
there are no satisfactory alternatives yet, but with growing computing power,
modelling possibilities expand and new promising frontiers of research emerge. One
of them could be agent-based finance. Agent-based financial models take account of
fundamental features undoubtedly inherent to the real world financial markets, such
as agent heterogeneity, bounded rationality and complex interaction of agents. More
generally, these computer models give researchers great flexibility to model interesting
features of real world phenomena. This will eventually allow very realistic modelling
of financial markets, goods markets or the economy as a whole. Before this vision
materialises, however, a major breakthrough in realistic modelling of intelligent, but
boundedly rational human behaviour is needed. It can only be achieved by blending and
expanding advancements of the economic theory, cognitive psychology and artificial
intelligence theory.
Mainstream capital market theories widely regarded financial markets as extremely
efficient price determination mechanisms. Moreover, trading processes themselves were
effectively excluded from the analysis by applying strong market clearing assumptions.
Now one can observe the gradual paradigm shift in the financial literature, as financial
markets are being increasingly viewed as complex dynamical systems, consisting of
interacting atomistic agents whose complex interaction and private learning result in
some systemic adaptation but not necessarily high market-level efficiency. The ongoing
paradigm shift is by no means a merely academic debate. It is of great importance
for market participants and policy makers. It is pretty obvious that in recent years
economists’ blind belief in nearly perfect markets’ self-regulation abilities outshined
the premonition of the looming global financial catastrophe and arguably even paved
the way for it. Hence, the current crisis offers economic researchers a good reason to
devote more effort for enhancing generative understanding of market processes rather
than concentrate on equilibrium relations.
The current paper is aimed at providing a non-specialist reader with a very brief and
selective introduction to the emerging area of agent-based financial modelling and,
more specifically, the artificial stock market (ASM) modelling. To our knowledge,
such analysis has not been conducted in the Lithuanian economic literature. An
intuitive discussion of problems of the standard financial modelling as well as specific
modelling alternatives will hopefully stimulate academic discussion among Lithuanian
economists and be policy-relevant.
The paper is organised as follows. In Section 2 we give a general discussion about
agent-based financial models as an alternative and a complement to standard financial
theories. A brief ASM literature review and presentation of some specific ASM design
issues are provided in Section 3. Section 4 is devoted for intuitive presentation of basic
principles of the standard reinforcement learning and its relevance in the context of
financial modelling. Section 5 contains concluding remarks.
Agent-Based Financial Modelling: A Promising Alternative to the Standard Representative-Agent Approach / T. Ramanauskas
1.
5
Bank of Lithuania Working Paper Series No 3 / 2009
2.Agent-based financial models versus the standard
representative-agent paradigm
Much of the mainstream financial theory builds on the efficient market hypothesis
(EMH) and the rational representative agent paradigm. These presumptions have clearly
played a crucial role in shaping the widely accepted understanding of risk, determinants
of asset prices, portfolio management principles, etc. Yet there is a growing need to
recognise that the gap between this idealisation and reality may be too substantial for
the theory to grasp correctly the essence of functioning of financial markets, as the
standard theory arguably abstracts from the salient features of examined phenomena.
The central question is whether the financial market can be seen as a perfect (or nearperfect) price determination mechanism. There are a lot of theoretical caveats and
empirical anomalies associated with standard financial theories, strong assumptions of
perfect rationality and the efficient market hypothesis.
2.1.General discussion of homogeneity, perfect rationality and market
efficiency
Theoretically, the homogeneous agent assumption is far from innocuous, as at the heart
of finance lies the collective discovery of securities prices in the process of trading,
i.e. as a result of the interaction of heterogeneous agents. It is a plain fact, which hardly
requires any scientific inquiry, that investors have a bewildering variety of views,
expectations and preferences. They possess different amounts of information, and their
financial decisions vary greatly in the level of sophistication. In reality, the financial
market awakes to life and trading takes place exactly owing to this heterogeneity of
market participants. An assumption that each individual, or the aggregate market
behaviour, can be approximated by some average or a fictitious representative agent
inevitably leads to the loss of a large degree of freedom. By assuming this, one clearly
risks attributing effects of changes in individual agents’ perceptions and strategies to
something like the representative agent’s consumption smoothing preferences.
Even stronger is the perfect rationality assumption. It is obvious that, as already noted
by Simon (1957) half a century ago, individuals act in a highly uncertain environment
and their natural computing abilities are limited, while information search and analysis
are costly and time consuming. All of this implies that even if perfect rationality was
feasible in the information collection, processing and decision making sense, it would
simply be too costly economically. Paradoxically, it is hardly rational to attempt being
perfectly rational. Moreover, a large thread of literature of psychology and behavioural
finance instigated by laboratory experiments of Kahneman and Tversky (1973) and
Tversky and Kahneman (1974) suggest that economic behaviour is often better explained
by simple heuristic rules and irrational biases rather than by dynamic optimisation.
The conflict between the perfect rationality idea and common sense deepens
further if we consider specifically financial markets. This is related to an inherently
large impact of expectations on the aggregate financial market behaviour. For example,
consider the standard line of thinking that individuals invest in risky assets so as to
optimise their consumption patterns. Owing to their different preferences, under the
heterogeneity assumption they all have potentially different valuations of expected
6
Agent-Based Financial Modelling: A Promising Alternative to the Standard Representative-Agent Approach / T. Ramanauskas
fundamental payoffs (e.g. an expected stream of stock dividends). As investors want
to optimally adjust their investment positions, aggregate supply and demand shift, and
market prices of risky assets change as a result. The magnitude of this change is largely
unpredictable because in reality there is no way of knowing idiosyncratic factors
affecting each investor’s asset supply and demand curves. In the short run, stock prices
are arguably more affected by these idiosyncratic shocks than by relatively infrequent
news on the structural changes of the processes materially affecting fundamentals –
this idea can be traced back to Keynes (1936); see also Cutler et al. (1989).
Probably even more importantly, every market participant has some marginal
impact on these price fluctuations and their expectations about likely price changes
may become self-fulfilling. Any signal, such as good news related to a specific stock,
may lead to investors’ coincident actions. This often triggers changes in the stock
price in a predictable direction, which implies that immediately following the news it
can become optimal for short-term speculators to buy the stock irrespective of actual
fundamentals. There is nothing to prevent under- or over-reactions to the news, hence
it is highly unrealistic to assume that the actual market price always coincides with
some fundamental value. Partially self-fulfilling expectations may lead to multiple
sunspot equilibria, which, of course, are not consistent with rational expectations by
definition. In other words, Muth’s (1961) rational expectations hypothesis can simply
be seen as an elegant way to exclude “ad hoc” forecasting rules and market psychology
from economic modelling (Hommes, 2006) but due to the self-referential nature of
predictions they may be deductively indeterminate. In reality market participants are
more likely to form expectations inductively (Arthur, 1995) – subjective expectations
are formed, tested and changed dynamically, as market conditions change and market
participants gain experience or interpret (possibly erroneously) plentiful information
signals.
Every reasonable person knows from their own experience that there is no conceivable
mechanism ensuring that their economic or social behaviour is perfectly rational. Yet
proponents of the EMH hypothesis argue that perfect rationality can be an emergent
feature of the financial market1. There are claims, for instance, that the existence of
arbitrage traders, evolutionary competition and generally offsetting each other noise
traders’ bets may ensure that securities prices always reflect fundamentals correctly.
The idea (which is also known in the literature as the Friedman hypothesis) that poor
performance drives non-rational investors out of the market is indeed appealing.
However, investment in stocks and some other securities is not a zero-sum game.
Stock prices generate positive returns in the long run in the overwhelming majority
of cases. Hence, it is not clear why non-rational investors, especially passive investors,
should “die out” – they may well enjoy decent returns to their less-than-rational (e.g.
passive) investment strategies. Moreover, their army is constantly replenished with
new inexperienced and hence non-rational investors. The argument of a negligible
impact of emotional and non-rational traders can also be challenged. It is exactly them,
rather than “fundamentalist” traders, who are more likely to react to non-fundamental
headline news and push market prices in the predictable direction, acting as a powerful
Emergent features are systemic features that cannot be deduced by simply scaling individual behaviour – see
Chen and Yeh (2002) for discussion.
1
7
Bank of Lithuania Working Paper Series No 3 / 2009
market-moving force and thereby imposing “rules of the game”. Moreover, it is well
known that sophisticated traders, instead of acting as a stabilising force, may try to
exploit the resultant predictable market movements. For instance, Frankel and Froot
(1987) conclude from their survey that investors often recognise a considerable price
deviation from their perceived fundamentals but nevertheless they find it logical to
follow the trend until it reaches some turning point.
There are also a number of empirical problems with traditional financial models
based on perfect rationality and EMH assumptions. Financial literature discusses quite
a few empirical anomalies, i.e. empirical regularities that are not explained by the
theory. Probably the most famous one is the equity premium puzzle raised by Mehra
and Prescott (1985). Stock returns (or equity risk premia) seem to be too high to be
explained by investors’ consumption optimisation behaviour, implying implausibly high
levels of their risk aversion. Shiller (1981) and others have noted that stock prices exhibit
excessive volatility, as compared to changes in fundamentals. There are also some
indications that markets may be more predictable than the EMH hypothesis suggests
(Campbell and Shiller, 1988, Lo and MacKinlay, 1988). Empirical facts, such as large
trading volumes, fat tails of returns distribution or persistent stock price volatility,
are not well understood either (LeBaron, 2006). And, of course, booms, busts and
financial crises – which are the absolutely salient features of today’s economic reality
and should be placed really high on economists’ research agenda – are in a discord
with the standard rational representative agent paradigm and the EMH hypothesis.
2.2. The new paradigm – markets as complex agent-based systems
Making strong assumptions in standard financial models may have been about the
only way to make theoretical generalisations about the market behaviour. But now that
the growing computing power and advancing computational methods have enabled
researchers to relax some of those assumptions, economics and finance are witnessing
an important paradigm shift towards a behavioural, agent-based approach. According
to this approach, markets are seen as complex dynamical systems consisting of
heterogeneous learning, boundedly rational heterogeneous agents (see Hommes, 2006,
LeBaron, 2006).
Computational study of these dynamical systems of interacting agents is what agentbased computational finance is all about. Let us very briefly discuss the principal
attributes of the research object of agent-based financial models. Naturally, at the centrestage are agents. Agents, in this context, are given quite a broad meaning. According to
Tesfatsion (2006), they comprise bundled data and behavioural methods representing
an entity in a computationally constructed environment. They can range from active,
learning and data-gathering decision-makers (e.g. investors, consumers, workers), their
social groupings (e.g. firms, banks, families) and institutions (e.g. markets, regulatory
systems) to passive world features such as infrastructure. From the operational point of
view, they are similar to objects and object groups in the object-oriented programming,
whereas agent-based models technically are collections of algorithms embodied in
those entities termed “agents”. The possibility to develop composite and hierarchical
structures of computational agents implies that they can become arbitrarily complex
8
Agent-Based Financial Modelling: A Promising Alternative to the Standard Representative-Agent Approach / T. Ramanauskas
and may greatly surpass their analytical counterparts of standard models in respect of
reflecting salient features of the real world entities.
The interdisciplinary nature of the notion of an agent also leads one to the realm
of the computer science. Here, an autonomous agent is understood as a system
situated in, and part of, an environment, which senses that environment, and acts on
it, over time, in pursuit of its own agenda (Franklin and Graesser, 1997). If agents
are capable of learning to achieve their goals more efficiently or their population as
a whole continuously adapts to be better suited to survive, the artificial intelligence
theory comes into play. Learning and adaptation are crucially important in agent-based
modelling since the ultimate goal of any economic analysis is to model the actual
human intelligent behaviour and its consequences at the individual or aggregate level.
Agents form complex adaptive systems. A system is said to be complex if it is
constituted of interacting elements (agents) and exhibits emergent properties, i.e.
properties inherent to the system but not necessarily to individual agents. Depending on
the complexity of studied phenomena, complex adaptive systems may include reactive
agents (capable of reacting in a systematic way to changing environmental conditions),
goal-directed agents (reactive and capable of directing some of their actions to achieving
their goals) and planning agents (goal-directed and capable of exerting some control
over environment). It is important that these systems are self-sufficient or dynamically
complete, i.e. they may evolve – without interventions from the modeller – in reaction to
exogenous environmental changes or even as a result of merely endogenous interaction
of agents. For a more thorough discussion of basic elements and principles of agentbased computational economics and finance see Tesfatsion (2006).
Once agents are put together in a complex system, systemic patterns resulting from
agents’ interaction may be observed and the system’s reaction to exogenous shocks can
be analysed. Hence, agent-based financial models basically are a simulation tool. This
determines the delicate position of agent-based financial modelling among standard
scientific inference methods: deduction-based theoretical models (i.e. theoretical
generalisation from certain assumptions) and induction-based empirical models
(recognition of systematic patterns in the empirical data). Simulation, and agent-based
modelling in particular, does not allow one to prove theoretical propositions, nor does
it directly measure real world phenomena, so there is always a risk of analysing an
artificial world too remote from reality. On the other hand, simulation, just like deductive
analysis, is based on explicit assumptions, which in many cases are much more realistic
than in analytical models. If those assumptions and parameters of exogenous processes
are calibrated to match empirical data, then simulation analysis does lend itself to
drawing valuable inductive inferences about the real world behaviour. Generally, as
Axelrod and Tesfatsion (2006) observe, simulation permits increased understanding
of systems through controlled computational experiments. Epstein (2006) also notes
the importance of agent-based modelling as a tool for generative explanation. While
most economic and financial theory deals with analysis of equilibria, he argues that
it is not enough to claim that a system – be it an economy, financial market or other
social grouping consisting of rational agents – if put in the Nash equilibrium, stays
there. For a fuller understanding of system’s behaviour for generativists it is important
to understand how the local autonomous interactions of atomistic, heterogeneous and
boundedly rational agents generate the observed macro-level regularities and how
9
Bank of Lithuania Working Paper Series No 3 / 2009
10
the system reaches, if reaches at all, the equilibrium. Moreover, plausibility of any
equilibrium patterns at the macro-level is required by generativists to be confirmed by
generating it from suitable microspecifications. As Epstein (1999) puts it, “If you didn’t
grow it, you didn’t explain it”. In general, agent-based economic and financial modelling
has several primary objectives – the abovementioned empirical understanding of
macro-level regularities, normative understanding of potential institutional and policy
improvements, and qualitative insight and theory generation through examination of
simulated behaviour (see Tesfatsion, 2006, for extended discussion).
Key features of agent-based financial models are well summarised by Epstein
(2006). The most important feature and actually the primary reason for departing
from standard analytical settings is heterogeneity of agents. Agents may differ in
their preferences, skills, decision rules, information sets, levels of wealth, etc., and
their characteristics may change over time independently of others. Agent behaviour
is generally characterised by bounded rationality, which arises both from limited
information and limited computational capacities of agents. Agent interactions are
autonomous, i.e. there is no central planner, Walrasian auctioneer or other central
controllers, though interaction rules, behavioural norms and institutional settings
may be arbitrarily sophisticated. Agent-based models also require an explicit agent
interaction network, which may be centralised or decentralised (in which case agents
interact locally), and their financial decisions may be influenced by information flows
through social networks. Finally, analysis of non-equilibrium dynamics of analysed
systems in agent-based modelling is of no lesser importance than studying equilibrium
properties.
Agent-based modelling clearly gives researchers a large degree of much desired
flexibility necessary to understand the real world financial market phenomena.
Unfortunately, this poses problems too. Much room for manoeuvre implies that agentbased models vary to such an extent that they lack some unifying fundament, which could
help to develop this interesting area of research into a solid theory with an established
methodology and conventional wisdom about basic building blocks. There are also
serious difficulties related to modelling micro-level behaviour. Having made the pretty
obvious proposition that the real world investors are less than fully rational, researchers
face difficult conceptual issues related to deciding what then governs agents’ behaviour
is and how to model it. Should modelled micro-behaviour match that of human subjects
in laboratory experiments? Should modellers deliberately include in their agent-based
models behavioural biases and heuristic rules confirmed by experimental data? Should
artificial agents apply decision rules that have some substantiation in the theoretical
representative agent models? How learning and expectation formation processes should
be modelled? Answers to these questions, of course, depend on the problem at hand
but, as stressed by Duffy (2006), in principle they are not yet systemically addressed
by researchers of this field. Dealing with these issues, Duffy suggests that external
validation of simulation results should not be limited to comparison of aggregate
outcomes of simulated and real world phenomena. He advocates careful selection of
model parameters based on experiments with human subjects and suggests seeking
stronger external validation by comparing simulation results to those obtained in the
experiments with humans, both at the micro- and macro-level.
Brief ASM literature review and important ASM design issues
The literature on the development of the artificial stock markets2 (ASM) is one specific
area of the agent-based modelling. In this section we discuss some important issues
arising when designing ASMs and review – though only briefly and selectively – the
literature on ASMs.
3.1. ASM building principles and problems
All ASM modellers have to deal with challenging design issues and face modelling
tradeoffs. These include, but are not limited to, the choice of agents’ preferences and
objectives, properties of securities, mechanisms of price determination, expectations
formation, evolution and learning algorithms, timing issues and benchmarks.
Since stock markets usually constitute extremely complex environment, one
immediate problem is that any degree of realism imposes huge computational costs and
results in the loss of analytical tractability. The problem is most severe in modelling
agents’ intelligent behaviour. It is further aggravated by the fact that the decision
making processes are unobservable and there is too little theoretical guidance on how
to model these processes realistically. For these reasons ASM models usually are highly
stylised, and in many cases model settings are kept close to certain benchmarks –
standard theoretical rational expectations models or tractable and well understood
special cases of agent-based models. This generally strengthens the credibility of ASM
models as tools of generative explanation of systemic equilibria derived under strong
assumptions and eases interpretation of simulation results.
Agents and their decision-making processes occupy the centre-stage in agent-based
models of stock markets and are the main source of diversity of ASMs. The agent
design might vary from budget constrained zero-intelligence agents to sophisticated
artificially intelligent decision-making entities. It is worth noting in passing that some
agent designs lack dynamic integrity and lasting identity inherent to human subjects,
which brings interpretation of these artificial agents closer to competing bundles of
strategies rather than to actual investors. Agents usually are given utility functions,
and utility levels associated with different strategies are important in driving agents’
behaviour or determining their “fitness” in the evolutionary selection process. Agents
may derive utility from different sources, e.g., consumption, wealth or returns. A serious
limitation but a very natural one, given the complexity of the model environment, is
that in most cases agents are myopic in that they care only about one-period utility and
do not attempt to carry out dynamic optimisation.
In simplest settings agents’ behaviour may be completely random (constrained only
by budget constraints to make it economically interesting, as in Gode and Sunder,
1993). Alternatively, they may follow strict decision rules or choose conditional
strategies from a (dynamically evolving) bundle of strategies. These rules may be
suggested by standard theories or may mimic popular actual investment strategies.
The central design question in the ASM models based on artificial intelligence is how
Agent-Based Financial Modelling: A Promising Alternative to the Standard Representative-Agent Approach / T. Ramanauskas
3.
ASMs are also referred in the literature to as simulated stock markets and agent-based models of stock
markets.
2
11
Bank of Lithuania Working Paper Series No 3 / 2009
agents choose investment strategies and how the pool of available strategies evolves.
It should be noted that in almost all models, agents – taken individually – have very
limited intelligence, whereas systemic adaptation and strategy improvement mostly
take place at the population level. Most ASM models employ Holland’s (1975) genetic
algorithm technique to drive the evolution of strategies. In such algorithms, inspired
by the theory of biological evolution, strategy pools evolve as a result of the rule
crossover, mutation and evolutionary survival of the fittest rules. Alternatively, agents
may choose their investment strategies or form their forecasts based on neural network
or simple econometric forecasting techniques. Another learning possibility is the Roth
and Erev (1995) type stimulus-response learning3 (discussed, e.g., in Brenner, 2006, or
Duffy, 2006), which leans on the simple idea that actions yielding larger payoffs tend
to be repeated more frequently. More technically rigorous and economically appealing
is the reinforcement learning mechanism established in the artificial intelligence
literature (see, e.g., Sutton and Barto, 1998). This approach has been hardly ever used
in the context of ASM modelling, partly due to some known problems of application
of such algorithms in multi-agent settings. But, in our view, the inability to ensure that
the learned strategies are asymptotically optimal should not preclude modellers from
taking advantage of these economically intuitive learning algorithms.
Specification of the market setting is another very important ASM design question.
ASM modellers usually simplify the portfolio allocation task, and in most models
there are only two types of securities traded – a risky dividend-paying stock and
a riskless bond. Moreover, pricing in the bond market is typically shut down by
assuming constant interest rates. Pricing of the stock is hence determined by both
fundamental factors and interaction of heterogeneous agents (and dynamics of their
expectations), though some features of these determinants may be greatly simplified
for analytical or computational purposes. For instance, the dividend process may not
be modelled specifically, or dividend can be assumed to be paid out every trading
period, which is a highly unrealistic but quite necessary assumption in the myopic
agent environment. Next critical issue in specifying the market setting, is the choice
of the price determination mechanism. According to LeBaron (2001, 2006), there are
four major classes of price determination mechanisms: (i) gradual price adjustment, in
which case individual sell and buy orders (for a given price) are aggregated and in the
next trading period the price is gradually shifted by the market-maker in response to
excessive supply or demand (the market is almost never in equilibrium), (ii) immediate
market clearing, whereby the market clearing price is computed from agents’ supply
and demand functions (the market is always in temporary equilibrium), (iii) randomly
matched trading, whereby trade takes place between randomly matched agent pairs,
and (iv) an order book, which most closely models the actual trading process on orderdriven automated stock exchanges. In this context it is useful to note the problem of trade
synchronicity – something, which is not an issue in standard analytical representative
agent models where there is simply no trade. Actually, the real world traders arrive in
the market and make their orders asynchronously, which may lead to strategic intraperiod interaction. There are some attempts to build event-driven ASMs instead of
It is also known in the agent-based modelling literature as the reinforcement learning but it should not be
confused with the formal reinforcement learning algorithm developed in the artificial intelligence literature.
3
12
3.2. Brief ASM literature review
Now let us turn to some specific models. The area of agent-based modelling of stock
markets has been active for about two decades. The need for this alternative modelling
of stock market behaviour arose from dissatisfaction with the abovementioned strong
assumptions of the standard financial theory, its neglect for simple investment behaviour
rules that are often used by finance practitioners and inability of standard models to
explain satisfactorily the real world stock price dynamics (e.g. the US stock market
crash on 17th October 1987 and, of course, the ongoing global financial meltdown).
We divide (somewhat arbitrarily) the models into two broad categories – (i) models
based on stochastic, heuristic and standard theory-implied behavioural rules and (ii)
models with learning agents or evolutionary systemic adaptation. The latter category
is arguably more promising and interesting.
3.2.1.ASM models based on random, heuristic and hard-wired
behavioural rules
The first group of ASM models generally investigate whether interaction of heterogeneous
agents, who base their decisions on simple deterministic rules or even act in a random
manner, might induce stock price movements qualitatively similar to those observed
in real stock markets. Agents in these models usually follow simple, “hard-wired”
investment rules. In order to generate interesting market dynamics without sacrificing
model parsimony and tractability, it is very common to allow just a few investment
strategies (these are the so-called few-type models; see LeBaron, 2006). For instance,
market participants may be broadly categorised as “fundamentalists”, “chartists” and
“noise traders”. Fundamentalists base their investment decisions on fundamental
information about stock dividend potential, chartists rely on technical analysis of time
series of stock prices, whereas noise traders may base their investment decisions on
erroneous signals about fundamentals, follow aggregate market behaviour or, say,
simply behave in a random manner. Though such strategies bear some resemblance to
the real world investment behaviour, the problem lies in determining the distribution of
different investor types in model population, as this distribution may play a key role in
shaping the aggregate market behaviour. Clearly, market developments are influenced
by relative popularity of different strategies. Under different circumstances some
strategies may become dominant and optimal to follow, hence in some models agents
are allowed to switch to different strategies. They can switch to alternative strategies on
their performance, or underperforming investors may simply exert smaller influence
on market developments due to their smaller financial wealth.
The origins of this strand of literature are linked to the few-type models of foreign
exchange markets proposed by Frankel and Froot (1988), Kirman (1991), De Grauwe
et al. (1993) and others. A prominent early example of the few-type model of a stock
market is Kim and Markowitz (1989). In their stylised model there are two types of
Agent-Based Financial Modelling: A Promising Alternative to the Standard Representative-Agent Approach / T. Ramanauskas
ASMs evolving in equal time increments. However, owing to technical and conceptual
difficulties, most ASM models assume that trading decisions are taken by all agents
simultaneously without having any strategic interaction of this type.
13
Bank of Lithuania Working Paper Series No 3 / 2009
agents that pursue either the portfolio rebalancing strategy or the portfolio insurance
strategy. Rebalancers aim at keeping a constant fraction of their assets in a risky stock,
while portfolio insurers try to ensure the minimum level of wealth by defensively selling
some of the stock holdings if the minimum threshold approaches. The rebalancing
strategy works as a market stabilising force (a stock price decline spurs stock buying),
whereas the portfolio insurance strategy amplifies market fluctuations (a stock price
decline prompts stock selling). With simple expectation formation and decision rules
and with the uncertainty induced by monetary shocks, the model shows that some
investment strategies, namely the abovementioned portfolio insurance strategy, may
have a sizeable destabilising effect on the market and can be partly responsible for
market crashes. Later models developed along this line of research are much more
detail-rich but they are still aimed at giving a plausible explanation of complicated
empirical market dynamics, which is not quite consistent with standard financial
models. In detailed models of Lux (1995), Lux and Marchesi (1999, 2000) much of
market dynamics is attributed to agents’ endogenous switching to different trading
strategies depending on the prevailing majority opinion. Another popular idea in
the ASMs is that some “smart money” traders possess superior information and are
able to form (boundedly) rational expectations, while others are noise traders (see,
for example, Shiller 1984, DeLong et al., 1990a, 1990b). Some models consider the
choice between costly optimisation and cheap imitation strategies (see e.g. Sethi and
Franke, 1995). In several other models artificial agents follow investment strategies
based on standard theoretical principles, such as standard mean-variance optimisation
(see Jacobs et al., 2004, Sharpe, 2007). Finally, it should be noted that contributions
from econophysicists in this area of research are very significant – see Samanidou et
al. (2007) for a review. A lot of agent-based financial models actually are a product
of economists’ close collaboration with physicists, drawing on the experience of the
latter in studying systemic behaviour resulting from complex interaction of atomistic
particles.
3.2.2.ASM models based on intelligent adaptation
Rational-expectations representative-agent models examine optimal investment
strategies and asset pricing in the equilibrium. ASM models based on hard-wired
investment strategies mostly deal with emergent properties of stock markets. All
of this is important in ASM models based on intelligent adaptation. In addition to
this, these ASM models also help to explain how investors may come up with good
strategies, examine their stability and whether such artificial markets can generate
equilibria derived from standard models with restrictive assumptions. Dynamically
evolving and improving strategies is a remarkable feature of these agent-based financial
market models. It greatly reduces reliance of modelled market behaviour on arbitrarily
chosen investment strategies and increases model realism. Participants of real world
financial markets act in a highly uncertain environment and most of them try to adapt
to changing environmental conditions and learn to improve their strategies. Learning
and adaptation mechanisms in ASM models may still be very far from anything that
realistically describes genuine human learning but, in any case, attempts to model this
crucial feature of investment behaviour constitute a qualitatively different approach to
14
Agent-Based Financial Modelling: A Promising Alternative to the Standard Representative-Agent Approach / T. Ramanauskas
financial modelling. In most of ASMs, artificial intelligence techniques, most notably
evolutionary algorithms and neural networks, are favoured over simplistic adaptive
rules. Since intelligent adaptation implies choosing among many different strategies, as
well as creating new ones, there is usually a large and evolving ecology of investment
strategies in these models, and they are therefore sometimes referred to as the “manytype” models.
A detailed review of the related ASM literature is provided by LeBaron (2006), and
here we only briefly mention some of the more popular models. A model proposed
by Lettau (1997) is one of the early attempts to examine whether a population of
heterogeneous agents are able to learn optimal investment strategies in a very simple
market setting, for which the analytical solution is known. In his model, agents are
endowed with myopic preferences and have to decide what fraction of their wealth
to invest in the risky asset with exogenously determined random return. As is very
common in ASM models, evolutionary systemic adaptation is ensured by the application
of the genetic algorithm. Even though individual agents actually have no intelligence,
fitter individuals (i.e. those whose strategies give higher levels of utility) have better
chances of survival, and this evolutionary selection leads to near-optimal strategies over
generations in this simple model. Routledge (1999, 2001) examines adaptive learning
in financial markets in a more complex setting, namely, a version of Grossman and
Stiglitz’s (1980) model of heterogeneous information about future dividends and signal
extraction. He presents analytic framework for adaptive learning via imitation of
better-informed agents and shows that the rational expectations equilibrium is broadly
supported by adaptation modelled with genetic algorithm.
Another interesting market setup is proposed by Beltratti and Margarita (1992). In
this model, agents apply artificial neural network techniques to forecast stock prices
from historical data, and trading may take place between randomly matched individuals
with differing expectations. A noteworthy feature is that agents may choose to apply
either a sophisticated neural network (with more hidden nodes, or explanatory variables)
at a higher cost or a cheaper naïve network, and this corresponds to the real world fact
that sophisticated investment is a costly endeavour. Interestingly, in some settings the
naïve investors gain ground, once markets settle down and additional benefits from
sophisticated forecasting do not cover its cost.
One of the most famous ASMs is the Santa Fe artificial stock market model developed
by Arthur et al. (1997), also described in LeBaron et al. (1999) and LeBaron (2006).
This model is aimed at exploring evolution and coexistence of a pool of strategies
that compete with each other in the genetic algorithm environment and drive the
market toward some informational efficiency. In this model there are two securities:
a risky dividend-paying stock and a riskless bond that offers the constant interest
rate. Heterogeneous agents have myopic4 constant absolute risk aversion preferences.
They try to forecast next period’s stock price by applying simple “condition-forecast”
rules and plug the forecast in their (induced) asset demand functions. The equilibrium
price is determined by the auctioneer by balancing aggregate demand for shares with
fixed supply. Forecast adaptation in this model is based on modified Holland’s (1975)
“condition-action” genetic classifier system. Each agent is given a set of rules mapping
Agents care just about one period and do not dynamically optimise.
4
15
Bank of Lithuania Working Paper Series No 3 / 2009
states of the world (such as the relative size of price/dividend ratio or a stock price
relative to its moving average) to forecasts (which are linear combination of the current
stock price and dividend). The rules endogenously evolve as a result of the cross-over,
mutation and selection. The authors of the Santa Fe ASM examine convergence of
the market price to the homogenous rational expectations equilibrium, and show that
this is the case for certain parameter settings corresponding to the “slow learning”
situation. The model is able to generate some statistical features of price dynamics
qualitatively similar to stylized facts about the real world financial markets, though no
attempt is made to quantitatively line them up with actual financial data or examine the
realism of assumptions about dividend processes (LeBaron 2006). The model served
as a platform for a number of further extensions (see e.g. Joshi et al., 2000, Tay and
Linn, 2001, Chen and Yeh, 2001).
In an interesting model, LeBaron (2000) examines how investors’ heterogeneous
time horizons (i.e. different information sets, upon which agents choose decision rules)
affect evolution of the market, its convergence to the known homogenous rational
expectations equilibrium and domination of different types of investors. The model is
in some respects similar to the Santa Fe model but has some notable original features.
The learning mechanism in this model is an interesting combination of the neural
network technique with the evolutionary search mechanism. In contrast to most
other models, agents learn portfolio allocation decisions rather than explicitly form
price expectations. Also, agents have access to a public – rather than private – pool
of investment strategies, which are based on simple feedforward neural networks.
Agents evaluate these strategies by feeding data series of heterogeneous length into
these networks, and this forms the basis of their heterogeneity. Furthermore, the neural
networks are evolved by applying mutation, crossover, weight reassignment and rule
removal operations, and some of the worst-performing individuals are also replaced by
new agents. One of the model’s main findings is that short-memory (and short investment
horizon) investors are not driven out of the market, and acting as some market volatility
generators they hinder attaining the rational expectations equilibrium.
4.
Reinforcement learning in the context of ASM modelling
Expectations regarding prospects of a particular stock or a stock index play a crucial
role in active portfolio management. If one is willing to accept the obvious empirical
fact of a large variety of market expectations, or if one is aimed at explaining how
this diversity comes about, then it is necessary to abandon the rational expectations
assumption5, which allowed to circumvent these issues in standard models. According
to Arthur (1995), rational expectations equilibrium is a special and in many cases not
robust state of reality, whereby individual expectations induce actions that aggregatively
create a world that validates them as predictions.
But how do investors behave in realistic, out-of-equilibrium situations? Needless
to say, realistic modelling of expectations formation poses great challenge and it is
essentially a grey area of the financial theory. A few observations about investor
The rational expectations equilibrium can only be warranted if the critical mass of agents basically apply the
same, true model of the reality (possibly with some non-systemic perception errors).
5
16
Agent-Based Financial Modelling: A Promising Alternative to the Standard Representative-Agent Approach / T. Ramanauskas
behaviour in a highly uncertain environment can be made, however. In such an
environment it seems perfectly sensible for investors to act adaptively and follow
inductive reasoning, or in other words, form simple forecasting models, test them and
update them depending on their performance. Hence, investors should constantly learn
from interaction with environment. Their learning is not a supervised process because
due to the model uncertainty generally there is no way of knowing the true intrinsic
value of a stock even in retrospective. For instance, if an investor observes a stock
price realisation, which is different from what he had expected, he generally cannot
know whether this deviation is attributable to his misperception of fundamentals,
unpredictable shocks to fundamentals, complex interaction of market participants
or other factors. However, even without knowing retrospectively what the “correct”
expectations and actions should have been, investors can judge about adequacy of their
beliefs and actions by reinforcement signals that they receive from interaction with
the environment. Possible reinforcement signals include their performance relative to
the market, long-term portfolio returns, utility from consumed earnings, etc. Another
important aspect is that in order to better adapt to the uncertain environment investors
have to both exploit their accumulated experience and explore seemingly suboptimal
actions.
If the above-described adaptive investment behaviour is deemed an adequate
description of how boundedly rational investors actually behave, reinforcement learning
methods developed in the artificial intelligence literature seem to be conceptually
suitable for modelling investor behaviour (though there are some problems with
technical implementation).
Inspired by the psychology literature, reinforcement learning is the sub-area of
the machine learning, and in its core lies agents’ interaction with environment in
pursuit of highest long-term rewards. It could be of a particular interest to economists
because standard theoretical economic agents’ behaviour is often guided by very
similar principles. In standard economics and finance, agents choose action plans that
ensure maximisation of life-time utility (long-term rewards), and that is exactly what
reinforcement learning agents seek – the main difference being that the latter do not
know the underlying model of the economy. The well-established link between the
basic reinforcement learning algorithm and dynamic programming, as well as the
proven ability of some reinforcement learning algorithms to achieve (under certain
conditions) convergence to optimal policies are especially attractive features of this
methodology, from the economists’ viewpoint. The reinforcement learning agents are
also well-positioned to solve the temporal credit assignment problem, i.e. determine
strategic actions that enable them to reach their ultimate goals even though those
actions may not give desirable results in the short-term. For economists, it is a great
advantage over simpler forms of adaptive learning.
Reinforcement learning addresses the question of how an autonomous agent that
senses and acts in its environment can learn to choose optimal actions to achieve
its goals (Mitchell 1997, p. 367). By taking actions in an environment and obtaining
immediate associated rewards, a reinforcement learning agent tries to find optimal
17
Bank of Lithuania Working Paper Series No 3 / 2009
policies, which maximise long-term rewards, and the process of improvement of agent
policies is the central target for reinforcement learning methods6.
Practical implementation of reinforcement learning techniques in the domain of
interesting economic and financial problems can be problematic, as policy convergence
requirements include stationary environment, fully observable states and the singleagent setting. In other words, the reinforcement learning agent is capable of learning
to effectively adapt in the well-defined stationary environment but, naturally, simple
adaptive learning cannot guarantee optimal behaviour once the multi-agent interaction
brings in strong non-stationarity and strategic interaction among agents.
Existing financial research provides little guidance on applying reinforcement
learning ideas in the ASM context. To our knowledge, no well-known multi-agent
stock market models based on reinforcement learning have been developed so far.
The reinforcement learning literature, however, provides some evidence that using the
single-agent reinforcement learning (more specifically, Q-learning) algorithm in the
multi-agent setting quite often leads to either exactly or approximately optimal policies
(Tesauro, 2002). For instance, Tesauro and Kephart (2002) show that in a stylised twoseller market price-setting policies derived by using the standard Q-learning algorithm
outperform some fixed and myopic policies and, in some settings, convergence to
optimal policies is achieved. It should also be noted that in order to improve performance
in multi-agent settings different extensions to the standard reinforcement learning
algorithms have been proposed. They are mainly applied in two-player games, and they
take into account the opponent’s estimated strategies (e.g. Littman’s (1994) Minimax-Q
algorithm, Tesauro’s (2004) Hyper-Q algorithm or the Nash-Q algorithm developed
by Hu and Wellman, 2003). Alternatively, agents may adapt learning rates according
to the current performance, as in WoLF (Win or Learn Fast) algorithm developed by
Bowling and Veloso (2001).
As the current discussion paper is an integral part of our ongoing research aimed at
combining ASM modelling with the reinforcement learning techniques, we provide a
cursory discussion how we deal with the abovementioned implementation problems.
In our research, strategic interaction among agents is limited as agents interact in a
competitive manner via the centralised exchange, where they participate in double
auctions and take decisions simultaneously. The problem is arguably alleviated by
the fact that the number of agents is quite large, and what matters for any specific
agent is the relatively stable distribution of all other agents’ actions (buy or sell orders)
rather than individual actions per se. Hence, the average market price and other trading
statistics can be seen as some summarising functions of multi-agent interaction and
this is taken into account when individual decisions are made. Additional stability of
learning processes can also be ensured by combining the Q-learning algorithm with
some evolutionary adaptation principles.
It should also be noted that reinforcement learning methods have been quite
successfully applied in various portfolio management problems. One of the early
applications is Neuneier’s (1996) model, which employs the Q-learning in combination
with the neural network as a value function approximator for optimal currency
A good introduction to the reinforcement learning techniques may be found in Sutton and Barto (1998),
Bertsekas and Tsitsiklis (1996) and Mitchell’s (1997) books, and some broad overview of reinforcement
learning models is given in Kaelbling, Littman and Moore (1996) survey.
6
18
5.
Concluding remarks
Global economic and financial problems caused by the disorderly unwinding of
imbalances that had been accumulating over decades cannot be easily reconciled with
the efficient market paradigm based on rational expectations and perfectly rational
representative agent assumptions. Some even see the collapse of global consumption
as a result of the disregard of intertemporal budget constraints, which is of course
at odds with basic principles of consumer (investor) optimising behaviour. In any
case, the mainstream financial and economic theory cannot explain current economic
developments, give adequate forecasts or policy prescriptions. In this situation there
is actually a growing need to look for a replacement for the work-horse financial and
macroeconomic models based on heroic assumptions. This paper advocates the view
that agent-based financial modelling, which models financial markets as complex
dynamical systems consisting of interacting heterogeneous agents, can potentially
become a viable alternative. However, significant further interdisciplinary progress
is needed before agent-based models can take the centre-stage of financial market
modelling.
In the paper we discuss, at the conceptual level, the need for the proper account of
market complexity, agent heterogeneity, bounded rationality and adaptive (though not
simplistic) expectations. These features are usually absent in standard financial models
but are at the heart of agent-based modelling. Agent-based models use procedural
(computer code) rather than functional (mathematical equations) model description
form, which technically enables researchers to easily inquire into many interesting
features of systemic behaviour. However, theoretical or empirical foundation of
individual agents’ behaviour and external validation of model results are two main
problem areas, which have not been systemically addressed to date and constitute a
serious obstacle to the further progress of agent-based financial modelling.
The paper focuses in particular on the artificial stock market modelling. In the
brief literature review two broad categories of models are presented: (i) models based
on stochastic, heuristic and standard theory-implied behavioural rules and (ii) models
with learning agents or evolutionary systemic adaptation. The first group of models
generally emphasise the role of agent heterogeneity in determining complex market
behaviour and emergent systemic properties, whereas the second group of models
concentrate on the generation of good investment strategies and the macro-level
implications of intelligently adapted investment strategies.
Intelligent adaptation in the highly uncertain environment can arguably be key
to understanding actual financial market behaviour. It is instructive to resort to the
artificial intelligence literature for specific algorithms of adaptation and learning. In
Agent-Based Financial Modelling: A Promising Alternative to the Standard Representative-Agent Approach / T. Ramanauskas
allocation in a simple two-currency, risk-neutral setting. Moody and Saffell (2001)
apply direct reinforcement learning to optimise risk-adjusted investment returns on
the intra-day currency and stock-index trading. Van Roy (1999) uses the temporal
difference learning algorithm for valuing financial options and optimising investment
portfolio. Reinforcement learning methods of portfolio management are gradually
gaining popularity among practitioners but theoretical literature remains relatively
scarce and further research is needed to unleash the potential of this approach.
19
Bank of Lithuania Working Paper Series No 3 / 2009
20
particular, we find reinforcement learning algorithms economically very appealing,
though there are certain problems with their practical implementation. Without
knowing the true model of reality, reinforcement learning agents learn from interaction
with environment and adjust their strategies so that they attain maximum long-term
reinforcement (utility) from the environment. Similarly, investors seek to reach their
long-term objectives in highly uncertain environment. We continue to explore these
ideas further in the ongoing artificial stock market research.
Arthur W. B. 1995: Complexity in Economic and Financial Markets. – Complexity, Vol. 1, No 1,
20‑25.
Arthur W. B., Holland J., LeBaron B., Tayler P. 1997: Asset Pricing under Endogenous Expectations
in an Artificial Stock Market. – In Arthur W. B., Durlauf S., Lane D. (Eds.) The Economy as an
Evolving Complex System II, Addison-Wesley, Reading, MA, 15-44.
Axelrod R., Tesfatsion L. 2006: A Guide for Newcomers to Agent-based Modelling in the Social
Sciences. – In Tesfatsion L., Judd K. L. (Eds.) Handbook of Computational Economics, Vol. 2,
1648-1659, Elsevier, Amsterdam.
Beltratti A., Margarita S. 1992: Evolution of Trading Strategies among Heterogeneous Artificial
Economic Agents. – In Meyer J. A., Roitblat H. L., Wilson S. W. (Eds.) From Animals to Animats 2,
MIT Press, Cambridge MA.
Bertsekas D. P., Tsitsiklis J. N. 1996: Neuro-Dynamic Programming. Athena Scientific.
Bowling M., Veloso M. 2001: Convergence of Gradient Dynamics with a Variable Learning Rate. In
Proceedings of the Eighteenth International Conference on Machine Learning, 27-34.
Brenner T. 2006: Agent Learning Representation: Advice on Modelling Economic Learning. – In
Tesfatsion L., Judd K. L. (Eds.) Handbook of Computational Economics, Vol. 2, 949-1011, Elsevier,
Amsterdam.
Campbell J. Y., Shiller R. J. 1988: The Dividend-Price Ratio and Expectations of Future Dividends and
Discount Factors. – Review of Financial Studies, 1, 195-227.
Chen S. H., Yeh C. H. 2002: On the Emergent Properties of Artificial Stock Markets: the Efficient
Market Hypothesis and the Rational Expectations Hypothesis. – Journal of Economic Behavior
and Organization, Vol. 49, 217-239.
Chen S.H., Yeh C. H. 2002: Evolving Traders and the Business School with Genetic Programming: A
New Architecture of the Agent-based Artificial Stock Market. – Journal of Economic Dynamics
and Control, 25, 363-394.
Cutler D. M., Poterba J. M., Summers L. H. 1989: What Moves Stock Prices? – Journal of Portfolio
Management, 15, 4-12.
De Grauwe P., Dewachter H., Embrechts M. 1993: Exchange Rate Theory: Chaotic Models of Foreign
Exchange Markets, Blackwell, Oxford.
DeLong J. B., Shleifer A., Summers L. H., Waldmann R. J. 1990a: Noise Trader Risk in Financial
Markets. – Journal of Political Economy, 98, 703-738.
DeLong J. B., Shleifer A., Summers L. H., Waldmann R. J. 1990b: Positive Feedback Investment
Strategies and Destabilizing Rational Speculation. – Journal of Finance, 45, 379-395.
Duffy J. 2006: Agent-based Models and Human Subject Experiments. – In L. Tesfatsion, K. L. Judd
(Eds.) Handbook of Computational Economics, Vol. 2, 949-1011, Elsevier, Amsterdam.
Epstein J. M. 1999: Agent-based Computational Models and Generative Social Science. – Complexity,
4, 51-57.
Epstein J. M. 2006: Remarks on the Foundations of Agent-based Generative Social Science. – In
L. Tesfatsion, Judd K. L. (Eds.) Handbook of Computational Economics, Vol. 2, 1585-1604,
Elsevier, Amsterdam.
Frankel J. A., Froot K. A 1987: Short-term and Long-term Expectations of the Yen/Dollar Exchange
Rate: Evidence from Survey Data. NBER working paper 0957.
Agent-Based Financial Modelling: A Promising Alternative to the Standard Representative-Agent Approach / T. Ramanauskas
References
21
Bank of Lithuania Working Paper Series No 3 / 2009
22
Frankel J. A., Froot K. A. 1988: Explaining the Demand for Dollars: International Rates of Return
and the Expectations of Chartists and Fundamentalists. – In R. Chambers, P. Paarlberg (Eds.)
Agriculture, Macroeconomics, and the Exchange Rate, Westview Press, Boulder.
Franklin S., Graesser A. 1997: Is it an Agent, or just a Program?: A Taxonomy for Autonomous
Agents. – In Proceedings of the Third International Workshop on Agent Theories, Architectures,
and Languages, Springer-Verlag, 1996.
Gode D. K. and Sunder S. 1993: Allocative Efficiency of Markets with Zero-Intelligence Traders:
Market as a Partial Substitute for Individual Rationality. – Journal of Political Economy, 101, 119137.
Grossman S. and Stiglitz J. 1980: On the Impossibility of Informationally Efficient Markets. – American
Economic Review, 70, 393-408.
Holland J. H. 1975: Adaptation in Natural and Artificial Systems. University of Michigan Press, Ann
Harbor.
Hommes C. H. 2006: Heterogeneous Agent Models in Economics and Finance. – In Tesfatsion L., Judd
K. L. (Eds.) Handbook of Computational Economics, Vol. 2, 1109-1186, Elsevier, Amsterdam.
Hu J., Wellman M. P. 2003: Nash Q-Learning for General-Sum Stochastic Games. – Journal of Machine
Learning Research, 4, 1039-1069.
Jacobs B. I., Levy K. N., Markowitz H. M., 2004: Financial Market Simulation in the 21st Century. –
the Journal of Portfolio Management, 30, 142-151.
Joshi S., Parker J., Bedau M. A. 2000: Technical Trading Creates a Prisoner’s Dilemma: Results from
an Agent-based Model. – Computational Finance, 99, 465-479.
Kaelbling L. P., Littman M. L., Moore A. W. 1996: Reinforcement Learning: A Survey. – A Journal of
Artificial Intelligence Research, 4, 237-285.
Kahneman D., Tversky A. 1973: On the Psychology of Prediction. – Psychological Review, 80,
237‑251.
Keynes J.M. 1936: The General Theory of Unemployment, Interest and Money. Harcourt, Brace and
World, New York.
Kim G., Markowitz H. 1989: Investment Rules, Margin, and Market Volatility. – Journal of Portfolio
Management, 16, 45-52.
Kirman A. P. 1991: Epidemics of Opinion and Speculative Bubbles in Financial Markets. In M. Taylor
(Ed.) Money and Financial Markets. Macmillan, London.
LeBaron B. 2001: A Builder’s Guide to Agent Based Financial Markets. – Quantitative Finance, Vol. 1,
No. 2, 254-261.
LeBaron B. 2000: Evolution and Time Horizons in an Agent Based Stock Market. Brandeis University
Working paper, http://papers.ssrn.com/sol3/papers.cfm?abstract_id=218309.
LeBaron B. 2006: Agent-based Computational Finance. – In L. Tesfatsion, K. L. Judd (Eds.) Handbook
of Computational Economics, Vol. 2, 1189-1233, Elsevier, Amsterdam.
LeBaron B., Arthur W. B., Palmer R. 1999: Time Series Properties of an Artificial Stock Market. –
Journal of Economic Dynamics and Control, 23,1487-1516.
Lettau M. 1997: Explaining the Facts with Adaptive Agents: The Case of Mutual Fund Flows. Journal
of Economic Dynamics and Control, 21, 1117-1148.
Littman M. 1994: Markov Games as a Framework for Multi-Agent Reinforcement Learning. – In
W. W. Cohen, H. Hirsh (Eds.) Proceedings of the Eleventh International Conference on Machine
Learning, 157-163.
Lo A. W., MacKinlay A. C. 1988: Stock Prices Do not Follow Random Walks: Evidence from a Simple
Specification Test. – Review of Financial Studies, 1, 41-66.
Agent-Based Financial Modelling: A Promising Alternative to the Standard Representative-Agent Approach / T. Ramanauskas
Lux T. 1995: Herd Behaviour, Bubbles and Crashes. – Economic Journal, 105, 881-896.
Lux T., Marchesi M. 1999: Scaling and Criticality in a Stochastic Multi-Agent Model of a Financial
Market. – Nature, 397, 498-500.
Lux T., Marchesi M. 2000: Volatility Clustering in Financial Markets: A Micro-Simulation of
Interacting Agents. Discussion Paper Serie B, University of Bonn.
Mehra R., Prescott E. 1985: The Equity Premium: A Puzzle. – Journal of Monetary Economics, 15,
145-161.
Mitchell T. 1997: Machine Learning. McGraw Hill.
Moody J., Saffell M. 2001: Learning to Trade via Direct Reinforcement. – IEEE Transactions on
Neural Networks, Vol. 12, No. 4, 875-889.
Muth J. F. 1961: Rational Expectations and the Theory of Price Movements. – Econometrica, 29,
315‑335.
Neuneier R. 1996: Optimal Asset Allocation using Adaptive Dynamic Programming. In K. Touretzky,
M. Mozer, M. Hasselmo (Eds.) Advances in Neural Information Processing Systems, MIT Press,
952-958.
Roth A. E., Erev I. 1995: Learning in Extensive-Form Games: Experimental Data and Simple Dynamic
Models in the Intermediate Term. – Games and Economic Behavior, 8, 164-212.
Routledge B. R. 1999: Adaptive Learning in Financial Markets. – Review of Financial Studies, 12,
1165-1202.
Routledge B. R. 2001: Genetic Algorithm Learning to Choose and Use Information. – Macroeconomic
Dynamics, 5, 303-325.
Samanidou E., Zschischang E., Stauffer D., Lux T. 2007: Agent-based Models of Financial Markets. –
Reports on Progress in Physics, 70, 409-450.
Sethi R., Franke R. 1995: Behavioural Heterogeneity under Evolutionary Pressure: Macroeconomic
Implications of Costly Optimization. – Economic Journal, 105, 583-600.
Sharpe W F. 2007: Investors and Markets: Portoflio Choices, Asset Prices, and Investment Advice,
Princeton.
Shiller R. J. 1981: Do Stock Prices Move Too Much to be Justified by Subsequent Changes in Dividends.
NBER Working Paper 0456.
Shiller R. J. 1984: Stock Prices and Social Dynamics. Brookings Papers in Economic Activity, 2,
457‑510.
Simon H. A. 1957: Models of Man. Wiley, New York.
Sutton R. S., Barto A. G. 1998: Reinforcement Learning: An Introduction. MIT Press.
Tay N. S. P., Linn S. C. 2001: Fuzzy Inductive Reasoning, Expectation Formation and the Behavior of
Security Prices. – Journal of Economic Dynamics and Control, 25, 321-362.
Tesauro G. 2002: Multi-Agent Learning Mini-Tutorial. Presentation in Workshop on Multi-Agent
Learning, Vancouver.
Tesauro G. 2004: Extending Q-Learning to General Adaptive Multi-Agent Systems. In Advances in
Neural Information Processing Systems, MIT Press.
Tesauro G., Kephart J. O. 2002: Pricing in Agent Economies Using Multi-Agent Q-Learning. –
Autonomous Agents and Multi-Agent Systems, 5, 289-304.
Tesfatsion L. 2006: Agent-based Computational Economics: A Constructive Approach to Economic
Theory – In Tesfatsion L. , Judd K. L. (Eds.) Handbook of Computational Economics, Vol. 2, 831893, Elsevier, Amsterdam.
Tversky A., Kahneman D. 1974: Judgement under Uncertainty: Heuristics and Biases. – Science, 185,
1124-1131.
23
Bank of Lithuania Working Paper Series No 3 / 2009
24
Van Roy B. 1999: Temporal-Difference Learning and Applications in Finance. In Abu-Mostafa Y.
S., LeBaron B., Lo A. W., Weigend A. S. (Eds.) Computational Finance (Proceeding of the Sixth
International Conference on Computational Finance), MIT Press, Cambridge, MA.
Watkins C. J. C. H. 1989: Learning from Delayed Rewards. Ph.D Thesis, King’s College.
No 3:“Agent-Based Financial Modelling: A Promising Alternative to the Standard RepresentativeAgent Approach” by Tomas Ramanauskas, 2009.
No 2:“Personal Income Tax Reform: Macroeconomic and Welfare Implications” by Sigitas
Karpavičius and Igor Vetlov, 2008.
No 1:“Short-Term Forecasting of GDP Using Large Monthly Datasets: A Presudo Real-Time Forecast
Evaluation Exercise” by G. Rünstler, K. Barhoumi, S. Benk, R. Cristadoro, A. Den Reijer, A.
Jakaitienė, P. Jelonek, A. Rua, K. Ruth, C. Van Nieuwenhuyze, 2008.
Agent-Based Financial Modelling: A Promising Alternative to the Standard Representative-Agent Approach / T. Ramanauskas
List of Bank of Lithuania Working Papers
25