23 April 2014
Entropy, Diversification and the Inefficient Frontier[1]
Gianni POLA, Quantitative Research – Paris
Ali ZERRAD, Quantitative Research – Paris
 Diversification measures can be divided in three
main groups: (i) metrics in portfolio weights; (ii)
approaches in risk contributions; (iii) measures
that are based on fundamental factors.
Introduction
In “The Battle for Investment Survival” [2], G. M. Loeb claimed that “diversification might be
necessary where no intelligent supervision is likely”. Hence diversification is not an issue for
the intelligent investor who is able to perfectly forecast market returns. Conversely portfolio
diversification is desirable if there is a lack of information or uncertainty in the financial
markets. Investors should only accept to lower portfolio diversification in presence of
strong market convictions.
The bear equity market over the last decade once again calls into question the predictability
of asset risk premia, and the effect of incorrect assumptions in the portfolio selection
process. In the eighties and nineties, many researchers ([3, 4, 5]) demonstrated that the
sub-optimality due to estimation risk can be dramatic.
Entropy is a concept that originates from Physics, but nowadays applied in many disciplines,
such as Computer Science, Sociology, Economics, Medicine, Mathematics and Finance. In
Information Theory, entropy is related to the degree of predictability of a dynamical
system: the higher the entropy, the less predictable the system; on the other hand
entropy decreases when additional information is available.
Entropy and diversification are closely related:
in this focus we investigate their relationship.
“
Investors should only
accept to lower
portfolio diversification in
presence of strong
market convictions.
Even though diversification is one of the most
popular ideas in finance, there is no agreement
on a common measure. This reflects the fact
that a portfolio can be seen from different
angles. A single measure is probably not
able
to
fully
describe
portfolio
diversification.
In [1] we will address the role of estimation risk in portfolio diversification showing
clearly that the main problems come from instability in correlation. Our study
demonstrates that in an out-of-sample exercise unstable correlation produces “bad”
portfolio turnover that responds only to noise in time-series; portfolio turnover is only
acceptable (“good” portfolio turnover) if it corresponds to significant changes in
asset risk-return characteristics.
 Diversification metrics in asset risk contribution
depends on asset volatility and correlation, the
latter being particularly relevant because
correlation instability is shown to produce
unstable asset allocation.
 Diversification and risk are not strictly related:
more diversification does not always imply less
risk.
 Mean-variance portfolios are poorly diversified:
in our case study the efficient portfolios exhibit
only half of the available diversification.
 A small penalty in terms of risk-return can allow
a
significant
improvement
in
portfolio
diversification.
 Leverage-aversion is very costly in terms of
portfolio diversification especially for high risk
profiles.
”
Document for the exclusive attention of professional clients, investment services providers and any other professional of the financial industry
23 April 2014
1
Panorama of Diversification Measures
Diversification approaches can be divided into three main groups. This section provides an
overview of the most popular approaches.
Portfolio diversification in portfolio weights. These approaches do not require any
specific information on asset risk properties. They depend only on portfolio weights, and
hence they do not suffer from estimation risk. They are adequate for quantifying
diversification of a portfolio composed of similar assets (e.g. stock portfolios, buy & hold
bond allocations).
Portfolio diversification in risk contributions. These metrics incorporate information on
asset returns, most of the time, through an historical evaluation. The main methods rely on
the covariance matrix, and hence they suffer from estimation risk. They describe well the
diversification in an investment universe characterized by a relatively large volatility
spectrum (e.g. diversified portfolios investing from short term bonds to high volatile stocks).
Portfolio diversification according to economic scenario methods. In these approaches
prior information might be particularly relevant in identifying fundamental factors; additional
information on asset volatilities and correlation are, sometimes, also taken into account.
These metrics tend to represent portfolio diversification according to asset exposure to
relevant macroeconomic and stress factors.
We refer to [1] for a more detailed review of portfolio diversification measures, while in this
study we focus on entropy-based diversification measures (see Annex for technicalities).
Entropy quantifies the degree of randomness of a physical system. Let us suppose that a
physical system can be described by N discrete states, and that {pi}i=1, …, N represents the
probability associated with each state. Entropy is defined as follows:
N
H   pi ln pi .
Exhibit 1:
Taxonomy of main diversification measures
Diversification in portfolio weights.
1)
the Herfindahl index
2)
the Lorentz curve
3)
the Gini coefficient
4)
the entropy-based diversification measure in
portfolio weights.
Diversification in risk contributions.
1)
the entropy-based diversification in asset
volatility
2)
the entropy-based diversification in asset risk
contribution
3)
the entropy-based diversification metric in
principal components
4)
the diversification ratio
5)
the Herfindahl index
6)
the Lorentz curve
7)
the Gini coefficient
Diversification according to economic scenario
methods
1)
Standard factors model (regression-based) on
macroeconomic variables
2)
DAMS (Diversification Across Macroeconomic
Scenarios (see Exhibit 2)
i 1
The number of relevant states is defined as η=exp(H). It can be proven that:
1) the entropy is maximal (η=N) in correspondence with a completely unpredictable
physical system according to which all states are equally likely {pi=1/N}i=1, …, N .
2) the entropy is minimal (η=1) in correspondence with a fully deterministic system
according to which only the probability of one state is different from zero and equal to
one.
In the following we present the applications in Finance. Moreover we quickly review the
diversification ratio. In order to simplify the mathematical notations, in the following we will
refer to η as an entropy measure.
Entropy-based diversification measure in portfolio weights (Hw). Hw depends on
portfolio weights, and it is maximal for the equally-weighted portfolio. ηw=exp(Hw) takes
values from 1 (maximal concentration) to N (maximal diversification), N being the number of
assets in the investment universe.
Exhibit 2:
Portfolio diversification according to DAMS
According to DAMS, asset returns dynamics can be
mostly explained in terms of growth, inflation and market
stress. Assets can be represented in a three dimensional
cube expressing polarizations to macro-factors (i.e. the
closer to the hedge the more sensitive assets are, the
closer to the centre the more insensitive they are). See
[6] for more details.
According to the DAMS paradigm a portfolio is well
Diversified Across Macroeconomic Scenarios if it is
placed in centre of the cube.
Entropy-based diversification measure in asset volatility (Hvol). Hvol is a function of
asset volatility, and it is maximal for the naïve risk parity portfolio (i.e. portfolio weights are
inverse proportional to the asset volatility). In this case ηvol represents the number of
relevant assets in the risk space assuming constant correlation among assets. ηvol ranges
from 1 to N, being equal to N for a naïve risk-parity portfolio.
Entropy-based diversification measure in asset risk contribution (Hwrisk). Hwrisk is a
function of asset risk contribution, and it is maximal for the risk parity portfolio. In this case
ηwrisk represents the number of relevant assets in the risk space. ηwrisk ranges from 1 to N,
being equal to N for a risk-parity portfolio.
portfolio is diversified across
macroeconomic scenarios
P
portfolio is poorly diversified
portfolio is concentrated on
specific macro scenarios
Source: Amundi Quantitative Research
Entropy-based diversification measure in risk contribution of principal components
(Hpcrisk). Hpcrisk depends on portfolio allocation in principal components, it is maximal in
correspondence to a risk parity portfolio in the principal component space (see [1] for more
details). ηpcrisk represents the number of relevant principal components represented in the
portfolio. ηpcrisk ranges from 1 to N; however for correlated assets just a few principal
components are usually able to explain most of the portfolio variance.
2
Document for the exclusive attention of professional clients, investment services providers and any other professional of the financial industry
23 April 2014
Diversification ratio (DR). It is given by the ratio of the weighted sum of asset volatilities
and the full portfolio volatility. DR is maximal for the maximum diversification or maximum
sharpe portfolio. The square of DR can be interpreted as the portfolio degree of freedom
(see [7]): in case of independent assets and portfolio weights inverse proportional to asset
volatilities (naïve risk-parity portfolio), DR reaches its maximum (N1/2).
In order to clarify the diversification measures, we provide in Exhibit 3 a practical application
to a balanced portfolio of an institutional client. The asset allocation is made up of 61%
nominal bonds and 39% equities. 37% of assets are in foreign currencies, hence we
considered an overlay exposure of 37% in fx-rates; Exhibit 3 (top panel) reports the asset
allocation. In terms of risk allocation the portfolio is exposed to 55% to equities, 41% to fxrates, and a residual 3% to nominal bonds.
In Exhibit 3 (bottom panel), we report the calculation of the entropy measures and the
(square of) the diversification ratio. The portfolio looks well diversified when viewed in terms
of the portfolio weights; however a more detailed analysis shows that the high degree of
diversification is only apparent. The main results from the analysis are:
A good degree of diversification in the portfolio weights (the entropy corresponds to
7.86; the maximum is 9 in this case, eight assets + fx-rates).
Exhibit 3:
Diversification in a balanced portfolio
The top panel reports the average allocation, the bottom
panel shows the corresponding figures of the
diversification measures.
1%
6%
Domestic Bond
9%
Global Bond
Emerging Bond
6%
47%
US Equity
17%
Asia/Pacific Equity
Emerging Equity
9%
Europe Equity
5%
Weak diversification in the allocation of risk (the entropy in asset volatility is 6.80, while
the entropy in asset risk contribution is 5.52; the maximum is 9 in this case).
entropy in portfolio weights
Portfolio exposed significantly to two (statistically) independent factors (the entropy
is 2.17; the square of the diversification ratio corresponds to 2.28).
entropy in asset risk
A poor degree of diversification across macroeconomic scenarios (see [6] for more
details): the portfolio polarizes towards rising growth and falling stress scenarios, while being
rather blended versus inflation scenarios; the polarization coefficients to inflation, growth and
market stress are (resp.) -51, 94%, -97%. The portfolio is indeed well positioned to benefit
from an increase in global growth, but it suffers in case of market stress (61% exposure in
nominal bonds is not enough to compensate for 39% on equities). The portfolio is
represented in Exhibit 2 by a green spot marked by “P”.
As it is clear from the above analysis, portfolio diversification should be addressed
from many angles, indeed a single measure is not able to describe it adequately.
2
Diversification measures and correlation
The diversification measures in risk contribution suffer from estimation risk. Indeed, as we
will show in [1], the main instabilities come from correlation estimates. Understanding the
relationship between diversification metrics and correlation is crucial.
We report some analytical results below, which are derived in detail in [1]:
entropy in volatility
entropy in principal…
square of the divers ratio
0 1 2 3 4 5 6 7 8 9
Source: Amundi Quant Research
Exhibit 4:
Illustration of the “duplication invariance”
In order to illustrate the “duplication invariance”, let us
consider an investment universe made up of three
assets, all of them with the same volatility (10%) and a
correlation matrix given by:
1
0.2
0.3
the entropy in asset risk contribution scales linearly with correlation matrix;
asset1
asset2
asset3
the diversification ratio depends quadratically on correlation matrix;
We conclude this section, briefly discussing the so-called “duplication invariance” that is
closely related to the attitude of the portfolio construction to deal with identical assets
(assets correlated to one). As detailed in [7], “consider a universe where an asset is
duplicated (for example, due to multiple listings of the same asset). An unbiased portfolio
construction process should produce the same portfolio, regardless of whether the asset
was duplicated”. Among all the diversification measures discussed in the previous
paragraph, only the diversification ratio verifies this property. Exhibit 4 offers an illustration.
However the cost of fulfilling the duplication invariance is that the consequent higher
sensitivity to correlation might lead to unstable allocations in certain circumstances.
In [1] we show that the entropy-based diversification measure in asset risk provides a
good compromise to quantifying diversification in risk contribution.
0.3
0.1
1
H w risk
32%
35%
33%
100%
H pc risk
0%
75%
25%
100%
DR
29%
37%
34%
100%
Source: Amundi Quantitative Research
the entropy in principal components depends non-linearly on correlation matrix.
This analytical result sheds light upon the dependency of diversification measures on
instabilities in correlation: higher sensitivity to correlation matrix might lead to higher
instability to estimation risk.
0.2
1
0.1
The optimal solutions obtained maximizing Hwrisk, Hpcrisk
and DR are given below:
the entropy in portfolio weights does not depend on correlation matrix;
the entropy in asset volatility does depend on asset volatilities, and hence being exposed
to estimation risk; but it does not depend on correlation matrix;
Domestic Equity
Let us consider now adding one asset that is identical to the
first one. Portfolio solutions change as
asset1
asset2
asset3
asset4
H w risk
21%
30%
28%
21%
100%
H pc risk
0%
52%
48%
0%
100%
DR
14%
37%
34%
14%
100%
Source: Amundi Quantitative Research
The DR is the only portfolio construction that does not
change the exposure to assets 2 and 3, hence it is the
only one to be invariant under duplication of asset 1.
3
Document for the exclusive attention of professional clients, investment services providers and any other professional of the financial industry
23 April 2014
3
Exhibit 6: Diversification VS volatility;
The inefficient frontier
Efficient portfolios are poorly diversified
In this section we considered a practical case study of an investment universe made up of
ten assets[8]. We computed volatilities and correlation according to a standard EWMA (last
three years half life; monthly observations; series from July 1983 to September 2013).
Expected returns have been set according to the constant Sharpe ratio hypothesis[9] (0.25;
see [10] for more details). The aim of the constant Sharpe ratio hypothesis is to provide
an unbiased framework to design strategic asset allocation.
Exhibit 5 (left panel) reports the composition of the portfolios in the efficient frontier: it is
evident that despite the neutrality of the hypothesis on expected returns, the efficient
portfolios look quite concentrated in few assets (only five risky assets have been
significantly allocated). The plot on the right in Exhibit 5 seems more coherent with a
neutral view on the markets: it reports optimally diversified portfolios that are obtained
maximizing the entropy in asset risk contribution.
Exhibit 5: Efficient portfolios vs. optimally diversified portfolios
Efficient frontier
Entropy in asset risk contribution
Source: Amundi Quantitative Research
Exhibit 7: Diversification VS volatility;
Optimally diversified portfolios
Source: AMUNDI Quant Research
In Exhibit 6 we computed the diversification curves for different diversification measures in
correspondence with the efficient portfolios[11]. The main messages are:
portfolios of the efficient frontier are poorly diversified: the curves peak at about half
of the available diversification within nine assets;
the diversification curves show a complex pattern between portfolio volatility and
diversification: higher diversification does not always imply lower risk;
the diversification curves converge to one for the highest risk portfolio; the only
exception is represented by the entropy in principal components according to which
even a 100% investment in only one asset can be exposed to more than one principal
component; in particular the highest risk portfolio is 100% composed of Crude Oil WTI that
is exposed (on average) to 2.24 principal components (i.e. the entropy is 2.24).
The above analysis clarifies that a standard mean-variance optimizer is not able to
deliver diversified allocation. In Exhibit 7 and in the right panel of Exhibit 5 we suggest a
simple remedy that is shown to produce more diversified allocations. Portfolios are obtained
by maximizing the diversification measures, and requiring an expected portfolio return that is
no more than 10 bps below one of the efficient portfolios. Exhibit 7 shows that a small
penalty in terms of portfolio return (10 bps) can produce a huge increase in portfolio
diversification according to all the metrics[12].
Portfolio allocations might be very different from each other according to the specific
diversification metric: in particular the diversification ratio and the entropy in principal
components produce quite concentrated allocation in terms of portfolio weights, thus being
more exposed to idiosyncratic asset risk (see [1] for more details), while the entropy in
portfolio weights, asset volatility and risk contribution look quite concentrated when
observed in terms of exposure to principal components. This empirical evidence suggests
us that probably a unique measure is not able to fully describe portfolio
diversification.
We finally move to discuss the cost of leverage-aversion in terms of portfolio diversification.
Exhibit 8 offers an example with respect to the entropy in asset risk contribution. We
considered the same optimization scheme as in Exhibit 7, however we allowed the cash to
be negative and maximum allowed leverage was set at 130%. The chart shows that: (i)
leverage-aversion significantly limits portfolio diversification, especially for high risk
profiles; (ii) small leverage can help to improve portfolio diversification.
Source: Amundi Quantitative Research
Exhibit 8: Diversification VS volatility;
Leverage-aversion limits diversification
Allowing leverage
enables portfolio
diversification to be
significantly increased
Source: Amundi Quantitative Research
4
Document for the exclusive attention of professional clients, investment services providers and any other professional of the financial industry
23 April 2014
Conclusion
Uncertainty provides a link between entropy and diversification: as entropy increases in case
of lack of information, diversification is a useful tool to navigate uncertain financial conditions.
On a more practical ground, entropy is shown to adequately represent diversification in
portfolio weights, risk contribution in assets and in principal components; among them
diversification measures in the risk space suffers from estimation risk.
Diversification for global diversified portfolio can be approached according to statistical and
more fundamental approaches (e.g. DAMS). In order to figure out portfolio diversification, we
advise to rely on the entropy-based diversification measure in asset risk contribution
and on DAMS: while the entropy permits us to incorporate asset idiosyncratic risk,
DAMS approach allows one to extract the biases towards changes in expectations of
macroeconomic and stress factors.
As a side consequence of our study, we advise complementing the standard meanvariance optimizer with entropy-based diversification measures in order to (i) enhance
portfolio diversification, and (ii) obtain a more coherent allocation with neutral views
as expressed e.g. by the constant sharpe ratio hypothesis for asset returns. A
moderate use of portfolio leverage can lead to a remarkable enhancement of portfolio
diversification.
References
[1] For a more complete and technical analysis, refer to “Is your portfolio effectively diversified? A critical assessment of diversification measures for portfolio construction.“ G. Pola – AMUNDI
Working paper in preparation 2014. We are particularly grateful to Sylvie de Laguiche (Head of Quant Research in Paris) for very useful comments that improved the quality of the analysis and of
the manuscript. Ali Zerrad was a trainee in Amundi Quantitative Research in 2013.
[2] G. M. Loeb. “The Battle for Investment Survival”. John Wiley & Sons, 2007.
[3] J. D. Jobson and B. Korkie. “Estimation for Markowitz efficient portfolios”. Journal of the American Statistical Association, 75:544–554, 1980.
[4] M. J. Best and R. R. Grauer. “On the sensitivity of mean-variance-efficient portfolios to changes in asset means: some analytical and computational results”. Review of Financial Studies, 4:315–
342, 1991.
[5] V. Chopra and W. T. Ziemba. “The effects of errors in means, variances, and covariances on optimal portfolio choice”. The Journal of Portfolio Management, pages 6–11, 1993.
[6] G. Pola. “Rethinking strategic asset allocation in terms of diversification across macroeconomic scenarios”. AMUNDI Cross Asset Special Focus, May 2013.
[7] Y. Choueifaty, T. Froidure, J. Reynier. “Properties of most diversified portfolios”. Journal of Investment Strategies, vol. 2/number 2, Spring 2013 (49-70).
[8] Nominal bonds: Barclays US Aggregate Treasury (B1), Barclays US Aggregate Corporate (B2), Barclays US Corporate High Yield (B3). Equities: S&P 500 (E1), Russell 2000 (E2), Msci World
(E3). Commodities: Gold (C1), Crude Oil WTI (C2), Industrial Metals (C3). Cash has been modelled according to a certain variable; cash rate has been fixed to 2%.
[9] We stress that this hypothesis makes maximum diversification portfolios by-construction mean-variance efficient (see [7]).
[10] “Unexpected Returns: Methodological Considerations on Expected Returns in Uncertainty“. S de Laguiche and G Pola – Amundi Working Paper WP 032-2012, November 2012.
[11] Diversification measures have been computed only on risky assets (portfolios are then rebased to sum up to 1). For the diversification ratio we prefer to also include exposure to risk-free
assets to be more coherent with its formulation in the literature.
[12] The only exception is given by the diversification ratio; however, as noted above, the poor improvement of diversification in this case is a by-product of our assumptions on expected returns;
other hypotheses on asset returns certainly would change the pattern.
Annex - Entropy-based diversification measures
N
Entropy is defined as
H   pi ln pi
, pi being the probability associated to a given physical state. It can be used in finance to describe diversification in portfolio
i 1
weights, risk contribution in asset space and principal components.
Entropy-based diversification measure in portfolio weights (Hw ). It is obtained by replacing pi with wi (wi representing portfolio weight in asset i) in the entropy
equation.
Entropy-based diversification measure in asset risk contribution (Hwrisk). It is obtained by replacing pi with a rescaled function θi of the asset risk contribution,
N
i 
σi and ρij being (resp.) the asset volatility and correlation.
w w   
j 1
N
N
i
j
i
j
ij
,
  wh w j h j hj
h 1 j 1
Entropy-based diversification measure in asset volatility (Hvol ). It is given by Hwrisk by assuming zero correlation among assets.
Entropy-based diversification measure in risk contribution of principal components (Hpcrisk). Principal Component Analysis allows us to rewrite the covariance matrix
CM as EΛE’, E and Λ corresponding (resp.) to the eigenvector matrix and to a diagonal matrix including the eigenvalues. The portfolio exposure w can be projected
into the principal components as u=E’w. The risk exposure to principal components is given by the following terms {hi=ui2 Λ(i,i)}i=1, …, M, where M indicates the number
of relevant principal components. The entropy measure is given replacing pi with
h
i
.
M
h
j 1
j
The formula includes only principal components that provide a good representation of most of the full variance of asset returns (e.g. 95%, 99%).
5
Document for the exclusive attention of professional clients, investment services providers and any other professional of the financial industry
23 April 2014
Editor
Philippe Ithurbide – Global Head of Research,
Strategy and Analysis
DISCLAIMER
Chief editor: Pascal Blanqué
Editor: Philippe Ithurbide
In the European Union, this document is only for the attention of “Professional” investors as defined in Directive 2004/39/EC dated 21 April 2004 on markets in financial
instruments (“MIFID”), to investment services providers and any other professional of the financial industry, and as the case may be in each local regulations and, as far as
the offering in Switzerland is concerned, a “Qualified Investor” within the meaning of the provisions of the Swiss Collective Investment Schemes Act of 23 June 2006 (CISA),
the Swiss Collective Investment Schemes Ordinance of 22 November 2006 (CISO) and the FINMA’s Circular 08/8 on Public Advertising under the Collective Investment
Schemes legislation of 20 November 2008. Under no circumstances, may this material be distributed in the European Union to non “Professional” investors as defined in the
MIFID or in each local regulation, or in Switzerland to investors who do not comply with the definition of “qualified investors” as defined in the applicable legislation and
regulation.
This document neither constitutes an offer to buy nor a solicitation to sell a product, and shall not be considered as an unlawful solicitation or an investment advice.
Amundi accepts no liability whatsoever, whether direct or indirect, that may arise from the use of information contained in this material. Amundi can in no way be held
responsible for any decision or investment made on the basis of information contained in this material. The information contained in this document is disclosed to you on a
confidential basis and shall not be copied, reproduced, modified, translated or distributed without the prior written approval of Amundi, to any third person or entity in any
country or jurisdiction which would subject Amundi or any of “the Funds”, to any registration requirements within these jurisdictions or where it might be considered as
unlawful. Accordingly, this material is for distribution solely in jurisdictions where permitted and to persons who may receive it without breaching applicable legal or regulatory
requirements.
The information contained in this document is deemed accurate as at the date of publication set out on the first page of this document. Data, opinions and estimates may be
changed without notice.
You have the right to receive information about the personal information we hold on you. You can obtain a copy of the information we hold on you by sending an email to
[email protected]. If you are concerned that any of the information we hold on you is incorrect, please contact us at [email protected]
Document issued by Amundi, a société anonyme with a share capital of €596,262,615 - Portfolio manager regulated by the AMF under number GP04000036 – Head office:
90 boulevard Pasteur – 75015 Paris – France – 437 574 452 RCS Paris www.amundi.com
Document for the exclusive attention of professional clients, investment services providers and any other professional of the financial industry