Consumers Price Index Advisory
Committee 2013 Discussion Paper
Crown copyright ©
This work is licensed under the Creative Commons Attribution 3.0 New Zealand licence.
You are free to copy, distribute, and adapt the work, as long as you attribute the work to
Statistics NZ and abide by the other licence terms. Please note you may not use any
departmental or governmental emblem, logo, or coat of arms in any way that infringes any
provision of the Flags, Emblems, and Names Protection Act 1981. Use the wording
'Statistics New Zealand' in your attribution, not the Statistics NZ logo.
Liability
While all care and diligence has been used in processing, analysing, and extracting data
and information in this publication, Statistics New Zealand gives no warranty it is error free
and will not be liable for any loss or damage suffered by the use directly, or indirectly, of the
information in this publication.
Citation
Statistics New Zealand (2012). Consumers price index advisory committee 2013
discussion paper. Available from www.stats.govt.nz.
ISBN 978-0-478-40823-2 (online)
Published in April 2013 by
Statistics New Zealand
Tatauranga Aotearoa
Wellington, New Zealand
Contact
Statistics New Zealand Information Centre: [email protected]
Phone toll-free 0508 525 525
Phone international +64 4 931 4610
www.stats.govt.nz
Contents
Introduction to the advisory committee ........................................................................... 6
Terms of reference .......................................................................................................... 6
Background ...................................................................................................................... 6
Purpose of this paper....................................................................................................... 7
1 Principal purpose of the CPI ........................................................................................ 8
1.1 Executive summary ................................................................................................... 8
1.2 Issues to consider ...................................................................................................... 8
1.3 Purpose of this chapter.............................................................................................. 9
1.4 Background to CPIs ................................................................................................. 10
1.5 Purpose of the CPI .................................................................................................. 11
1.6 Design of a CPI........................................................................................................ 13
1.7 International practice ............................................................................................... 21
1.8 A single CPI versus multiple indexes ...................................................................... 23
1.9 Conclusion ............................................................................................................... 25
2 Consumer price change for subpopulations............................................................ 26
2.1 Executive summary ................................................................................................. 26
2.2 Issues to consider .................................................................................................... 26
2.3 Context for subpopulation indexes .......................................................................... 29
2.4 Past practice of presenting price change for subpopulations ................................. 31
2.5 Conceptual frameworks for subpopulations ............................................................ 32
2.6 Dissemination of subpopulation indexes ................................................................. 33
2.7 Quality of indexes of price change for subpopulations ........................................... 33
2.8 International comparisons of measuring price change for subpopulations............. 35
2.9 Feasibility study for indexes of consumer price change for subpopulations........... 38
2.10 Further developments............................................................................................ 61
2.11 Options for indexes of consumer price change for subpopulations ...................... 65
References..................................................................................................................... 66
Appendix 2a: Discontinued series ................................................................................. 68
Appendix 2b: Coding of government transfer payments ............................................... 70
Appendix 2c: Australian expenditure patterns of selected household types ................ 71
3 Sampling framework ................................................................................................... 72
3.1 Executive summary ................................................................................................. 72
3.2 Issues to consider .................................................................................................... 72
3.3 Recommendations of the 2004 CPI Revision Advisory Committee ........................ 74
3.4 Sampling framework ................................................................................................ 75
3.5 Basket coverage ...................................................................................................... 77
3
Consumers price index advisory committee 2013 discussion paper
3.6 Regional pricing centres .......................................................................................... 77
3.7 Outlet coverage ....................................................................................................... 84
3.8 Product coverage..................................................................................................... 85
3.9 Temporal coverage .................................................................................................. 85
3.10 Sampling trade-offs ............................................................................................... 85
3.11 Regional weighting ................................................................................................ 86
3.12 Regional-spatial indexes ....................................................................................... 89
3.13 Modes of data collection ........................................................................................ 90
3.14 Options for sampling framework............................................................................ 91
Appendix 3a: Household Economic Survey – regional sample sizes ........................... 92
Appendix 3b: Population covered by territorial authorities with pricing centres............ 93
Appendix 3c: Retail sales coverage by territorial authority areas ................................. 94
4 Frequency of weight updates ..................................................................................... 95
4.1 Executive summary ................................................................................................. 95
4.2 Issues to consider .................................................................................................... 96
4.3 Purpose of this chapter............................................................................................ 96
4.4 Background – why weights are updated ................................................................. 97
4.5 Current practice ....................................................................................................... 99
4.6 Future possibilities ................................................................................................. 101
4.7 Issues with updating expenditure weights on a more frequent basis ................... 105
4.8 Options for more frequent weight updates, and implications ................................ 108
4.9 Conclusion ............................................................................................................. 109
5 Retail transaction data .............................................................................................. 111
5.1 Executive summary ............................................................................................... 111
5.2 Issues to consider .................................................................................................. 111
5.3 Introduction to using scanner data ........................................................................ 111
5.4. The advantages of scanner data .......................................................................... 112
5.5 The challenges of scanner data ............................................................................ 116
5.6 The solutions so far ............................................................................................... 118
5.7 Current international practice ................................................................................ 121
5.8 The current research focus.................................................................................... 122
5.9 Potential issues associated with putting scanner data into production................. 122
5.10 Conclusion ........................................................................................................... 124
References................................................................................................................... 125
Appendix 5a: SAS code for scanner data methods .................................................... 126
Appendix 5b: Abstract for 2013 Ottawa Group ........................................................... 127
6 Frequency of the CPI ................................................................................................. 128
6.1 Executive summary ............................................................................................... 128
4
Consumers price index advisory committee 2013 discussion paper
6.2 Issues to consider .................................................................................................. 128
6.3 2004 CPI Revision Advisory Committee ............................................................... 129
6.4 A monthly CPI ........................................................................................................ 129
6.5 International practice ............................................................................................. 131
6.6 Cost estimate for the current quarterly CPI ........................................................... 132
6.7 Current pricing frequency ...................................................................................... 132
6.8 Monthly option – full monthly CPI collection.......................................................... 132
6.9 Monthly option – spread pricing for items currently collected quarterly ................ 134
6.10 Implementation costs........................................................................................... 138
6.11 Other considerations ........................................................................................... 138
6.12 New developments .............................................................................................. 141
7 Seasonality in the CPI ............................................................................................... 142
7.1 Executive summary ............................................................................................... 142
7.2 Issue to consider.................................................................................................... 142
7.3 Introduction ............................................................................................................ 143
7.4 Recommendations of the 2004 CPI Revision Advisory Committee ...................... 143
7.5 History of the treatment of seasonality in CPI ....................................................... 144
7.6 Seasonal or variable baskets ................................................................................ 147
7.7 A fully seasonally adjusted CPI ............................................................................. 147
7.8 International practice ............................................................................................. 155
7.9 Seasonal adjustment options for the New Zealand CPI ....................................... 158
Appendix 7a: Additional seasonality tables ................................................................. 159
8 Dissemination of CPI and FPI information ............................................................. 162
8.1 Executive summary ............................................................................................... 162
8.2 Issue to consider.................................................................................................... 162
8.3 Purpose of this chapter.......................................................................................... 162
8.4. Publishing mediums ............................................................................................. 162
8.5 What is published .................................................................................................. 165
8.6 Data visualisation................................................................................................... 168
8.7 Personal inflation calculator................................................................................... 169
8.8 Index reference period........................................................................................... 169
Appendix 8a: Tables and supplementary tables published as part of information
releases ....................................................................................................................... 170
Appendix 8b: Infoshare – CPI and FPI time-series categories ................................... 173
Appendix 8c: Infoshare – published CPI average prices ............................................ 174
Appendix 8d: Infoshare – published FPI average prices ............................................ 176
Appendix 8e: Infoshare – analytical series .................................................................. 178
Appendix 8f: Recent infographics ................................................................................ 180
5
Introduction to the advisory committee
Statistics New Zealand has appointed a consumers price index (CPI) advisory committee.
The committee will meet to undertake an independent review of the methods and
practices used to compile the CPI, and to advise the Government Statistician on the CPI.
The Government Statistician appointed the following committee members:
• Diana Crossan, former Retirement Commissioner
• Dr Carla Houkamau, University of Auckland Business School
• Dr Kirdan Lees, New Zealand Institute of Economic Research
• Dr John McDermott, Reserve Bank of New Zealand
• Professor Jacques Poot, University of Waikato
• Associate Professor Marco Reale, University of Canterbury
• Dr Bill Rosenberg, New Zealand Council of Trade Unions
• Stephen Summers, Business New Zealand
• Keith Woolford (international expert), formerly of the Australian Bureau of Statistics.
Diana Crossan will chair the committee.
Members were selected to bring professional expertise, different perspectives, and the
confidence of the wide range of CPI users to the committee.
Terms of reference
The terms of reference for the 2013 CPI Advisory Committee are to:
1. Investigate, review, and form recommendations concerning the general nature
and objectives of the CPI, while taking account of the actual, potential, and
appropriate uses of the index.
2. Investigate and form recommendations concerning the general principles that
should be considered in the construction of the CPI with specific reference to:
• coverage of the CPI, including the range of goods and services
represented in the index and the coverage of the household population
• the practices used to compile the index
• the methods used to calculate the index.
Background
The 2013 CPI Advisory Committee is the latest in a longstanding series of committees
appointed to undertake an independent review of the methods and practices used to
compile the CPI. Similar committees have been convened periodically since 1948. The
most recent three committees were convened in 1991, 1997, and 2004.
The practice of holding CPI advisory committees is recommended by the International
Labour Organization (ILO) in their 2004 resolution concerning CPIs:
The agency responsible for the index should consult representatives of users on
issues of importance for the CPI, particularly during preparations for any changes
to the methodology used in compiling the CPI. One way of organizing such
consultations is through the establishment of advisory committee(s) on which
social partners, as well as other users and independent experts might be
represented.
6
Consumers price index advisory committee 2013 discussion paper
The committee will make recommendations to the Government Statistician, who will then
consider these recommendations relative to other initiatives in the Official Statistics
System. Statistics NZ has adopted the recommendations of past committees, where
possible. Progress made on the recommendations of the 2004 committee is summarised
in ‘4 Progress with recommendations from the 2004 CPI Revision Advisory Committee’.
Purpose of this paper
This paper provides members of the CPI advisory committee and the wider CPI user
community with information on key CPI topics, to inform submissions to and discussion
by the 2013 CPI Advisory Committee.
7
1 Principal purpose of the CPI
1.1 Executive summary
The consumers price index (CPI) measures the change in price of goods and services
acquired by New Zealand-resident private households. The primary purpose of the CPI
since the late 1990s has been to measure inflation for the purpose of informing monetary
policy setting. The CPI is also widely used as a compensation index (to adjust
government transfer payments, and wages and salaries) as well as a deflator to help
derive constant-price volume series, such as gross domestic product (GDP) and retail
trade sales.
The emphasis on inflation targeting has influenced some design features of the CPI,
which may make the index less suitable for compensation purposes. For example, the
New Zealand CPI is based on an ‘acquisition’ framework, which is suitable for inflation
measurement, while a ‘payment’ or ‘use’ framework would be suitable for an index
primarily used for compensation purposes.
However, there are also some aspects of the CPI that are better suited for a
compensation index – for example, the inclusion of non-market transactions, such as
local authority rates, where price change is not directly influenced by monetary policy.
This makes the New Zealand CPI more of a ‘general purpose’ CPI, taking on
characteristics of both an inflation index and a compensation index, rather than being
optimally suited for one or the other.
The 2004 CPI Revision Advisory Committee recognised the multiple uses of the CPI and
that a single index could not optimally meet all users’ needs. The committee also
acknowledged the advantages of maintaining a single ‘headline’ CPI and recommended
the CPI should remain an index better suited for measuring inflation.
The 2004 and 1997 committees recommended the production of additional conceptual
measures of price change compiled under alternative frameworks to supplement the
headline CPI. These indexes would adopt conceptual frameworks better suited to
indexing payments, and might adopt reference populations that relate to specific
subpopulations. These indexes have not yet been produced due to resource constraints,
and being a low priority relative to other initiatives across the suite of official statistics.
This chapter outlines the purpose and design of the New Zealand CPI and asks the
committee to consider whether these are still appropriate. The costs and benefits of a
single ‘general purpose’ CPI versus multiple indexes designed for specific purposes are
also discussed.
1.2 Issues to consider
Purpose of the CPI
• Should the primary purpose of the CPI remain inflation measurement for the
purpose of monetary policy setting?
Conceptual framework
• Should Statistics New Zealand continue producing the CPI using an acquisition
framework?
Scope and coverage
• Is the current scope and coverage adequate for the principal purpose of the CPI?
Possible areas to consider are:
8
Principal purpose of the CPI
o the treatment of transactions that are not fully influenced by market factors (eg
local authority rates)
o the treatment of households living in non-private and institutional dwellings
o the treatment of purchases made from overseas vendors via the Internet.
A ‘single’ CPI or multiple indexes
• Is there a need for additional indexes measuring household price change? If so,
what purpose are these indexes to serve, and how should they be constructed and
disseminated?
1.3 Purpose of this chapter
This chapter outlines how the purpose of a CPI influences its design. Central to this topic
are the three internationally recognised conceptual frameworks that can be used to
construct a CPI. These are set out by the International Labour Organization’s (ILO)
resolution concerning CPIs as:
• acquisition
• payment
• use.
Each framework is suited to a different principal purpose. The ILO resolution states that
an acquisition framework is most suitable for inflation measurement, a payment
framework is most suitable for indexing payments to maintain purchasing power, and a
use framework is most suitable for measuring changes in the cost of living.
This chapter also outlines the current principal purpose and design of the New Zealand
CPI, and asks the committee and CPI user community to consider whether this purpose
is still appropriate, and if any further enhancements are needed to the CPI’s design.
The 2004 CPI Revision Advisory Committee made three recommendations that
specifically relate to the principal purpose of the CPI.
Recommendation 1: The Revision Advisory Committee recognises the wide
ranging uses of the CPI, in terms of its application as a measure of inflation and as
a measure of changes in the cost of living. The committee acknowledges that a
single index cannot meet the needs of all users, but also recognises the value and
importance of maintaining a ‘single’ CPI. On balance, there should remain one
index branded as the ‘CPI’ and it should remain an acquisitions-based index
measuring inflation in the goods and services purchased by New Zealand-resident
private households.
Recommendation 2: Statistics New Zealand should provide another index (or
indexes) as separate series that would be more suited conceptually to measuring
changes in households’ cost of living. This additional index (or indexes) need not
be published on as frequent a basis as the CPI, and an annual frequency would
probably be sufficient to meet most users’ requirements.
The precise methodology for producing a credible and robust cost of living measure
would be left to Statistics New Zealand to explore. However, in producing an index,
or a range of indexes, Statistics New Zealand should take account of changes in
the cost of living for different population subgroups such as superannuitants, wage
and salary earners, low-income households, and recipients of government transfer
payments.
Recommendation 3: Statistics New Zealand should not make changes to further
enhance the CPI as a measure of inflation in the domestic economy (such as
9
Principal purpose of the CPI
including expenditure in New Zealand by overseas tourists or removing local
authority rates and similar non-market transactions) if such changes would result in
the index being less suitable for measuring changes in the cost of living for New
Zealand households. However, this decision could be reviewed at a time when
Statistics New Zealand makes available a robust and credible cost of living index
(or a range of cost of living indexes).
1.4 Background to CPIs
1.4.1 Main uses of a CPI
At the broadest level, three main uses are made of a CPI:
• as an inflation index – either for the household sector, or as a proxy for the wider
economy. A CPI is widely used to assist central banks in maintaining general price
stability
• as a compensation index – to compensate for increases in the cost of living, by
adjusting (indexing) government transfer payments, income tax thresholds, wages
and salaries, or other payments
• as a deflator – to derive (constant-price) volume series of household consumption
expenditure (HCE) in the national accounts.
The CPI is often regarded as an appropriate indicator of inflation in the wider economy,
beyond the household sector, although this view is generally not reflected in its design. In
many countries, targeting of CPI inflation has successfully resulted in stabilising
economy-wide inflation expectations.
1.4.2 Main uses in New Zealand
In New Zealand, the CPI measures price change for a fixed basket of consumption goods
and services acquired by New Zealand-resident private households living in permanent
dwellings.
The 2004 CPI Revision Advisory Committee confirmed the principal purpose of the New
Zealand CPI should be to measure inflation for the purposes of monetary policy. The CPI
is also used widely for indexing a range of payments (eg superannuation and welfare
payments), and as a deflator for deriving constant-price volume series (eg GDP and retail
trade sales).
In New Zealand, the use of the CPI as an inflation index for monetary policy purposes is
partly pragmatic. Other official price indexes, such as the producers price index and gross
domestic product implicit price deflator, are seen to be less suitable, or are not sufficiently
well understood to be accepted by the general public for use in monetary policy.
Using the CPI as a compensation index also partly reflects its high profile and wide
acceptance, although the design of the New Zealand CPI has not been optimised for this
use.
1.4.3 Conceptual issues in the CPI
Given the high profile use of the CPI, it is essential that the index maintains a good
reputation among key users and the wider public. The index’s reputation is influenced by
both the rules and practices that support the index’s production and its technical
construction.
Statistical agencies should follow good practice in producing the index. The ILO
resolution concerning CPIs outlines some of these practices, which include, but are not
limited to:
10
Principal purpose of the CPI
• releasing results as quickly as possible after the end of the period the results refer
to
• making information available to all users at the same time, in a convenient format
• disclosing the methods used to construct the index
• notifying publication dates in advance
• adhering to publication dates.
The technical construction of the index must also be considered. One challenge in
maintaining the CPI’s reputation is the user community’s wide range of needs and the
limited resources with which to fulfil them. In recent decades this has seen the New
Zealand CPI become a ‘general purpose’ CPI, taking on characteristics of both an
inflation index and a compensation index, rather than being optimally suited for one or the
other. The balance of user needs and perceptions is a key consideration when making
changes to the CPI’s composition.
The purpose for the index influences which conceptual framework should be used to
construct the CPI. Selecting one particular framework often involves trade-offs that may
make the index better suited for a specific purpose, but not as well suited for others. For
example, an index compiled under the ‘payment’ framework would be suitable for
compensation purposes, while an ‘acquisition’ framework would be suitable for measuring
inflation. To maintain the wider credibility of the index it could be argued that a pragmatic
balance is needed. (See the section ‘1.6.1 Conceptual framework’ below.)
The CPI’s purpose also influences the scope and coverage of transactions included in the
index. In principle, an inflation index would include expenditures by all households within
a country, regardless of resident status or the type of dwelling the household resides in.
In contrast, a compensation index may look to target a specific subpopulation, such as
superannuitants, or wage and salary earners. Similarly, an inflation index may exclude
prices that are partly or fully influenced by non-market factors, such as interest payments
or government charges, while these may be deemed appropriate for a compensation
index. (See the section ‘1.6.2 Scope and coverage’ below.)
Given the range of uses for a CPI, producing multiple indexes is an option. In practice,
there are reasons why a single CPI might be maintained; namely the credibility of the CPI
brand and the extra costs of producing more than one index. But where the potential
differences between indexes designed for inflation and for compensation purposes
become significant, separate indexes may be desirable. An index that is optimised for
compensating payments for a subpopulation (eg a superannuitants index) may even be
considered more credible for this purpose than the CPI, because it takes into account the
specific spending patterns of the target subpopulation. The 2004 CPI Revision Advisory
Committee recommended maintaining a single index designated the CPI, using the
acquisition framework, but that an additional index (or indexes) should be produced using
a payment and/or use framework. Due to resource constraints this has not been
implemented, but it remains a relevant issue for the 2013 CPI Advisory Committee to
consider. (See the section ‘1.8 A single CPI versus multiple indexes’ below for discussion
on the costs and benefits of producing multiple indexes.)
1.5 Purpose of the CPI
1.5.1 Recent history
In the 1970s, the emphasis was on using the CPI to adjust wages. During this period and
into the 1980s, wage bargaining took place at a national level and in a high-inflation
environment. The CPI was seen as a key indicator of the level of wage adjustment
required to maintain the purchasing power of incomes. This was reflected in the design of
the CPI, when the conceptual framework was changed from a use framework to a hybrid
11
Principal purpose of the CPI
acquisition/payment framework in 1974 following a recommendation from the 1971 CPI
Revision Advisory Committee.
A change in focus occurred in the late 1980s and early 1990s with the widespread
adoption of inflation targeting by central banks. After the Reserve Bank Act 1989 and
Employment Contracts Act 1991 were passed, the CPI’s use as an inflation indicator
became more important than previously. This eventually resulted in changes to the
design of the CPI, to make it better suited for inflation measurement. These changes
were implemented in the September 1999 quarter.
During the early phase of formal inflation targeting, the Reserve Bank focused primarily
on a variant of the CPI. The official all groups index was recognised as treating certain
items, such as housing, in a way that was not considered optimal for monetary policy
considerations. The first policy targets agreement (PTA) in 1990 stated that inflation was
to be measured using a housing adjusted (consumer) price index (HAPI). The HAPI was
an adjusted version of the CPI, produced by the bank, that used an imputed rents
approach for owner-occupied housing. The 1997 PTA formalised the policy target in
terms of the analytical series, CPIX. This series was the all groups CPI excluding credit
services. This series excluded interest rates, which were included in the headline
measure at the time.
The decision to align the official CPI more closely to an inflation index, suitable for
monetary policy setting, resulted in interest rates being excluded from the all groups CPI
from the September 1999 quarter onwards – a result of formally adopting an acquisition
conceptual framework. Residential section prices were also excluded. Since that time, the
monetary policy target has been defined in successive PTAs as changes in the all groups
CPI.
1.5.2 Past advisory committee recommendations
The 1997 CPI Revision Advisory Committee recognised the importance of having a
credible measure of the CPI that was suitable for monetary policy purposes. However,
there were differences in committee members’ views about the need to make changes to
the CPI, given its history as a compensation-type index. A compromise was agreed on,
and the 1997 CPI Revision Advisory Committee recommended that a change should be
made to the official all groups CPI, making it more suitable for measuring inflation. The
committee confirmed that an acquisition framework should be used to construct the CPI.
The CPI was to measure price change based on goods and services actually acquired by
households, and would exclude interest payments (and section prices). However, the
committee recommended that the official CPI be supported by two additional price
indexes that, in concept, would be more suitable for compensation purposes.
There was to be an index based on all household payments for goods and services. This
index would adopt the payment framework and include interest payments, allowing
continuity with the CPI before the changes. The other index was to be based on the use
framework and more closely approximate the concept of a cost of living index. The key
difference was that it would use an imputed-rent or user-cost approach to measure price
change for owner-occupied housing. (See the section ‘1.6.1 Conceptual framework’
below for further discussion.)
The recommendation for three price indexes was not implemented, due to resource
constraints and being viewed as a low priority relative to other initiatives across the suite
of official statistics.
The 2004 CPI Revision Advisory Committee recommended that Statistics NZ “provide
another index (or indexes) as separate series that would be more suited conceptually to
measuring changes in households’ cost of living”. This recommendation came from a
desire to maintain a single index branded as ‘the CPI’, rather than have three
12
Principal purpose of the CPI
conceptually different CPIs. The extra indexes would stand alongside, rather than be part
of, the headline CPI and would be published annually.
The 2004 recommendation for additional measures using different conceptual
frameworks was also not implemented, for similar reasons as in 1997.
Following interest being excluded from the headline CPI as part of the 1999 CPI review,
Statistics NZ has published an analytical series, ‘CPI plus interest’. This allows for the
continuation of a CPI series that includes interest.
1.6 Design of a CPI
CPIs designed for specific purposes have specific design criteria. Two interrelated
aspects of the design of a CPI are the:
• conceptual framework
• scope and coverage.
1.6.1 Conceptual framework
The ILO resolution concerning CPIs describes three conceptual frameworks that are used
to underpin CPI design: acquisition, payment, and use. Table 1.1 below summarises the
different conceptual frameworks that can be used to construct a CPI.
13
Principal purpose of the CPI
Table 1.1
Summary of CPI conceptual frameworks
Treatment for:
Acquisition
Payment
Use
Owner-occupied
housing
Net acquisition
of housing
during weight
reference
period, net
dwelling
insurance. Might
want to include
local authority
rates
Mortgage
interest, gross
dwelling
insurance, local
authority rates
Imputed-rent or
user-cost
approach
Durable/semidurable goods
Value of goods
acquired during
weight reference
period (whether
or not paid for or
used)
Value of
payments for
goods during
weight reference
period (whether
or not acquired
or used)
Value of the flow
of service from
goods used
during weight
reference
period.
Acquisition or
payment
framework used
in practice.
Non-durable
goods
Value of goods
acquired during
weight reference
period
Value of
payments for
goods during
weight reference
period
Value of goods
used during
weight reference
period
Services
Value of
services
acquired during
weight reference
period
Value of
payments for
services during
weight reference
period
Value of
services used
during weight
reference period
1.6.1.1 Acquisition and payment frameworks
Under the acquisition framework, the CPI weights are derived from expenditure on the
goods and services acquired by households during the weight reference period (the
period in time selected to weight the index, usually one year), irrespective of whether they
were wholly paid for or consumed during that period. Prices enter the CPI in the period
when consumers acquire the good or service, and at the full value agreed to, as opposed
to what is paid at the time.
Under the payment framework, expenditure weights are derived from the total payments
made for goods and services during the weight reference period, regardless of when the
goods or services were acquired or consumed. Prices enter the CPI in the period that
payment is made, which may or may not coincide with the period of acquisition or
consumption.
Both these frameworks are based on monetary expenditures. Most goods and services
are fully paid for when acquired, but a different treatment is given to items purchased on
credit. Under the acquisition concept, the full value of goods and services acquired during
the weight reference period is used to determine their expenditure weights, regardless of
14
Principal purpose of the CPI
whether those items were fully paid for when acquired. A service is acquired by a
household when it is provided by the service provider (eg a flight is taken).
Under the payment framework, only the actual payments made during the weight
reference period are used. These payments may represent partial payment for items
purchased during the weight reference period, plus any payments made for goods that
had been previously purchased on credit. This framework includes interest payments.
Payments for goods purchased before the weight reference period goods are not
included in an index measuring prices for weight reference period acquisitions, and are
therefore out of scope under the acquisition framework. Interest payments are also
excluded in this framework, as the amount of interest paid does not necessarily bear a
strong relationship to the actual quantities of goods and services acquired in the weight
reference period.
1.6.1.2 Use framework
Under the use framework, expenditure weights are based on the value of the goods and
services used or consumed during the weight reference period. The emphasis is on the
value of the commodity consumed, rather than what was paid for it. The value of a good
or a service consumed, and the price paid for it for ‘use’, will be the same for many items.
The most significant difference under the use framework is the treatment of capital (or
durable) goods.
Under the use framework, households are viewed as benefiting from the flow of services
derived from durable goods rather than the purchase of these goods. The most important
of these items in the CPI is owner-occupied housing. Owner-occupied housing has a
range of different treatments in CPIs. Under the acquisition framework, used in the New
Zealand CPI, its weight is based on the net increase in the stock of owner-occupied
housing during the weight reference period. Under the payment framework, its weight is
determined by the amounts actually paid out for housing (which includes interest
payments for dwellings acquired in earlier periods). With the use framework, it is the
value of the shelter services being consumed that is used to derive its expenditure
weight. Shelter value can be derived by estimating the implicit (ie imputed) rental value of
the stock of owner-occupied housing, or the user cost of the shelter consumed, which
may include real costs such as insurance or rates, as well as notional costs such as
depreciation.
These differentiations can be extended to other durable goods. For example, the weight
of motor vehicles in the CPI could be determined by estimating the value of the transport
service consumed during the weight reference period, and prices could be based on the
costs of rental car hire. In practice, most countries with a use framework use a flow of
services approach only for owner-occupied housing.
Aside from the practical difficulties of estimating the value of the service being provided,
the use framework requires modelled or notional expenditures and prices (eg imputed
rents) in the CPI. Furthermore, the methodology for valuing notional expenditure tends to
result in durables being given significantly higher expenditure weights than would be the
case under the acquisition framework. The higher expenditure weight and the use of
notional prices are factors that may affect the credibility of the CPI as a ‘representative’
measure of price change. Credibility issues can also arise under approaches based on
actual expenditures, for example the relatively low weight for owner-occupied housing
under an acquisition framework.
The treatment of services in general is similar to an acquisition framework. A service is
‘used’ by a household when it is provided by the service provider (eg a flight is taken).
The value of that service is the price the household paid for that service. This applies to
both subsidised (eg visits to the doctor) and non-subsidised services.
15
Principal purpose of the CPI
1.6.1.3 Summary
The design of New Zealand’s CPI has been formally based on the acquisition framework
since 1999. The ILO resolution considers this framework the most appropriate for an
inflation indicator, as it reflects the actual prices of goods and services acquired by
households and excludes interest payments.
The payment and use frameworks are viewed as being conceptually more appropriate for
compensation purposes. A decision to adopt a payment rather than a use approach may
reflect concerns over the non-acceptance of an index that makes extensive use of
notional prices.
Some may choose to adopt a payment framework where the aim of the index is to
maintain purchasing power, while a use framework implies the preservation of living
standards. The use framework is also conceptually more suitable for deflating household
consumption expenditures in the national accounts, because of the treatment of owneroccupied housing.
1.6.2 Scope and coverage
1.6.2.1 Scope and coverage of the New Zealand CPI
Table 1.2 below summarises the current scope and coverage of New Zealand’s CPI
16
Principal purpose of the CPI
Table 1.2
Scope and coverage of the New Zealand CPI
Coverage
dimension
Reference
population
Inclusions
• New Zealand-resident private
households living in
permanent dwellings
Exclusions
• New Zealand-resident
private households not
living in permanent
dwellings
• Institutional dwellings (eg
retirement homes and
prisons)
• Other non-private
dwellings (eg boarding
houses)
Goods and
services
• Goods and services
transacted at market prices
• Government charges (eg
local authority rates, vehicle
and driver licences)
• Interest
• Residential sections
• Assets
• Financial investment
• Services partly or fully
subsidised by government
(eg health and education)
and transacted at non-market
prices
Place of
acquisition
• In New Zealand – by the
reference population
• Purchases via the Internet or
mail order where the supplier
is located abroad – by the
reference population
• In New Zealand – by
other than the reference
population (eg by nonprivate households
(about 2 percent of the
population), foreign
visitors, non-profit
institutions serving
households, and general
government)
• Abroad – by the
reference population
The current scope and coverage of New Zealand’s CPI reflects its principal purpose as
an index that measures inflation for monetary policy. It also incorporates some aspects
that are, in theory, more suited for a compensation index, such as the inclusion of
government charges.
Like the conceptual framework, the scope and coverage of the index will also differ
depending on the principal purpose of the index. The categories of scope and coverage
covered in table 1.2 are discussed in more detail below.
1.6.2.2 Reference population
In theory, an inflation index should have broader reference population coverage than a
compensation index. It should encompass total final consumption expenditure, including
that by visitors from abroad (eg tourists) and by other non-private households. Further, it
17
Principal purpose of the CPI
can be argued that final consumption expenditure by non-profit institutions serving
households, and by general government, should also be included.
An index constructed to help derive constant-price volume series would ideally use a
similar reference population to the series it is deflating. For example, if an index was
being used to provide a constant price volume series of HCE in the national accounts,
then the reference population should match that of the HCE. In New Zealand, this would
mean covering expenditure by all New Zealand-resident households, including those in
non-private and institutional dwellings.
Similarly, for a compensation index, the reference population needs to be appropriately
considered. While an index could be constructed using a reference population of
households as a whole, it may be beneficial to target specific subpopulations. For
example, an index designed for adjusting New Zealand Superannuation rates could use
only superannuitant households as a reference population, using the spending patterns of
this subpopulation to derive expenditure weights. An index designed to adjust salaries
and wages may consider only the spending patterns of households earning salaries and
wages.
Identifying the expenditure patterns of different subpopulation groups can be difficult.
Statistics NZ established a superannuitants price index (SPI) in 1995, using expenditure
weights relating to this subpopulation. The index was discontinued after the June 1999
quarter. The main reasons given at the time were:
• The 1997 CPI Revision Advisory Committee recommended that further
development of the SPI was a low priority.
• There were known limitations to the SPI when it was first constructed in 1995 (eg it
assumed superannuitants purchased similar goods and services at similar outlets
and prices to the average New Zealander). Several organisations had expressed
reservations about the usefulness of a measure with these sorts of limitations.
• The major conceptual change to the CPI in 1999, which excluded interest
payments and section prices, made the headline measure less suited as a living
cost index.
• The SPI was originally based on a living cost definition. Continuing to calculate the
SPI based on the new CPI definition would mean covering a reduced range of
goods and services.
• It was concluded that Statistics NZ would not be able to update superannuitant
spending patterns as frequently as the CPI weights. Consequently, the difference
between the CPI and the SPI was viewed as likely to become less and less
meaningful.
The potential production of subpopulation indexes is discussed in the next chapter
‘Consumer price change for subpopulations’.
1.6.2.3 Goods and services
It could be argued that an inflation index should be narrower in scope than a
compensation index and incorporate only market-based transactions. Price changes for
goods and services such as health care and education, which are fully or partly funded by
government, are not directly influenced by monetary policy.
Similarly, government charges may not belong in an inflation index. These charges relate
to household payments, but there is not necessarily a direct relationship between the
payment made and the goods and services acquired. Local authority rates is an example
of this. One view is that such charges are a general tax, which, as with income tax, has
no place in an inflation index. An opposing view is that an expense like local authority
rates represents a tax for home ownership – an inescapable cost of owning a home. It
18
Principal purpose of the CPI
could then be included in the same way that prices are recorded including GST in the
CPI.
Notional prices, such as imputed rents, should also be excluded from an inflation index,
as inflation is considered to be something that affects only real monetary expenditure.
However, as these prices represent the service flow of a durable asset, they would be
suitable for an index constructed on a use framework.
1.6.2.4 Place of acquisition
It could be argued that an index designed to measure inflation in the domestic economy
would include only those goods and services purchased within New Zealand from New
Zealand-based retailers. This would exclude purchases made by households from
retailers based overseas (eg from overseas websites). Changes in the price of goods and
services imported by households would not accurately reflect inflationary pressures in the
New Zealand economy. Expenditure made by New Zealand households while abroad
would also not be desirable in an inflation index, for similar reasons.
Another view is that including purchases made by New Zealand-resident households, in
New Zealand, from overseas retailers (eg via the Internet) may still be of interest to
central banks. The inflation rates that households’ experience (which may include Internet
purchases from overseas vendors) can be different to that of the domestic economy. This
is because it is the rate of inflation experienced by households, not the rate of inflation
within the country’s borders, that influences household saving and spending decisions.
This is particularly true if, for example, households partly offset the domestic inflation they
experience by shifting consumption to off-shore goods and services that show lower
relative price change.
On the other hand, an index designed for compensation purposes would want to include
only the unique spending patterns of the target population, including payments made to
overseas retailers.
Similarly, an index designed to measure the changing cost of a fixed standard of living,
using a use framework, would want to incorporate all expenditure by the reference
population, regardless of where these purchases come from or were made.
1.6.3 CPI design – summary
Both the conceptual framework and the scope and coverage can be adjusted to suit CPIs
with different principal purposes. Table 1.3 below gives a summary of how the design of a
CPI can differ for each of the three main uses.
19
Principal purpose of the CPI
Table 1.3
Summary of CPI designs for different principal purposes
CPI design
aspect
Conceptual
framework
Reference
population
Goods and
services
Place of
acquisition
Inflation
measurement
Compensation
Deflating
national
accounts
New Zealand
CPI
Acquisition
Payment or Use
Use
Acquisition
New Zealandresident
households,
including nonprivate and
institutionalised
households.
Could also
include visitors
from abroad
and
government/nonprofit spending
on behalf of
households.
Target
subpopulation
(eg
superannuitants,
wage and salary
earners)
New Zealandresident
households,
including nonprivate and
institutionalised
households
New Zealandresident
households,
excluding nonprivate and
institutionalised
households
(about 2
percent of the
population)
Market-based
transactions only
Market-based
transactions.
Interest using
payment
framework.
Notional prices
(eg imputed
rents), if use
framework.
Government
charges (eg
local authority
rates).
Payments for
subsidised
goods/services.
Notional prices
(eg imputed
rents). Marketbased
transactions.
Market-based
transactions.
Government
charges.
Payments for
subsidised
goods/services.
In New Zealand,
from New
Zealand
retailers.
Could also
include
purchases made
from overseas
retailers via the
Internet
In New Zealand,
including
purchases made
from overseas
vendors via the
Internet and
while abroad,
by the reference
population
In New
Zealand,
including
purchases
made from
overseas
vendors via the
Internet and
while abroad,
by the
reference
population
In New
Zealand,
including
purchases
made from
overseas
vendors via the
Internet
20
Principal purpose of the CPI
1.7 International practice
Internationally, CPIs use different frameworks and have differing scope and coverage.
Below is a brief summary of CPIs produced in selected countries.
1.7.1 Australia
The Australian Bureau of Statistics (ABS) produces a CPI and also a suite of analytical
living cost indexes (ALCIs).
Australia’s CPI uses an acquisition framework and is designed to measure inflation for
the purpose of monetary policy. Owner-occupied housing costs are measured using a net
acquisition approach that aligns with that used in New Zealand. The CPI’s reference
population is private Australian households in the eight capital cities, which accounts for
about two-thirds of the Australian population. This excludes people living in institutional
and non-private dwellings.
The Australian ALCIs use a payment framework and have reference populations that
relate to specific Australian subpopulations. Mortgage interest payments are included in
the ALCIs for measuring owner-occupied housing costs, as are property rates and
dwelling insurance. Four ALCIs are published, for the following subpopulations:
• employee households
• age pensioner households
• other government transfer recipient households
• self-funded retiree households.
In addition, a pensioner and beneficiary living cost index (PBLCI) is published. The PBLCI
combines the age pensioner household and other government transfer recipient ALCIs.
From September 2009, most Australian social security pensions have been indexed by
the greater of the CPI and the PBLCI. Pension rates are also adjusted for improvements
in wages, as measured by male total average weekly earnings.
The ALCI suite has been produced by the ABS since 2000. Initially, quarterly index
movements were published on an annual basis as a by-product of the CPI. In 2009, a
government review identified the need for an index that was better suited for indexing
government pension payments. As a result of this review, the ABS was given additional
resources to produce a higher-quality PBLCI for this purpose. Additional pensioner and
beneficiary households were added to the 2009/10 Australian Household Expenditure
Survey, to increase the sample size or ‘over sample’ this subpopulation relative to other
households. By doing this, the ABS was able to derive higher-quality expenditure
estimates for pensioner and beneficiary households. The ALCIs (along with the PBLCI)
were reweighted at the same time as the Australian CPI using this data. From the
September 2009 quarter onwards the ALCIs and PBLCI have been published quarterly.
1.7.2 Canada
Statistics Canada produces a CPI that uses an acquisition framework, with the exception
of owner-occupied housing, which uses a use framework. Owner-occupied housing is
measured using a user-cost approach, which includes costs such as mortgage interest
payments and depreciation. The reference population of Canada’s CPI is both urban and
rural private households, excluding those living in institutional and non-private dwellings.
One difference between the Canadian reference population and those of other CPIs is
that tertiary students living in student accommodation are included. Typically, these types
of dwellings are excluded from other countries’ reference populations as ‘non-private’ (as
in New Zealand).
21
Principal purpose of the CPI
1.7.3 European Union
Each statistics agency within the European Union (EU) produces a harmonised index of
consumer prices (HICP) in addition to its own CPI. The primary purpose of the HICPs is
to measure inflation across countries in the EU, and there are specific guidelines for their
construction. In some countries, the resulting HICPs differ in scope and methodology
from the country’s existing CPI.
In constructing HICPs, countries adopt a consistent approach and a regulated set of
definitions. In particular, HICPs take a ‘domestic’ approach to determine the index’s
reference population. This means each HICP covers all household monetary
consumption expenditure within the country (including households living in institutional
and non-private dwellings). This includes all households, irrespective of national or
residence status, and excludes any expenditure by residents abroad. Prices are actual
prices faced rather than notional measures. At this stage, owner-occupied housing costs
are excluded from the HICPs. Pilot studies have been underway for some time to
determine the appropriate treatment of owner-occupied housing in the HICPs.
1.7.4 United Kingdom
The Office for National Statistics in the United Kingdom (UK) produces an HICP
excluding owner-occupied housing costs that is branded the ‘CPI’. In the UK, the CPI is
both the official measure of domestic inflation for monetary policy purposes, and the
index used for adjusting government payments.
The UK CPI sits alongside the retail prices index (RPI), an index that more closely
resembles a compensation index in construction. In the RPI, owner-occupied housing is
measured through a user-cost approach, incorporating mortgage interest payments,
depreciation, insurance on buildings, and building-related taxes. The reference population
is UK households, excluding the top 4 percent of households (by income), households
living in institutional and non-private dwellings, and excluding pensioner households
where at least three-quarters of household income is derived from government transfer
payments.
The ONS announced the introduction of two additional indexes, to be published from
March 2013. These are the CPI-H, which is the CPI including an imputed rents measure
for owner-occupied housing, and the RPI-J, which is the RPI using a geometic mean of
price relatives formula (Jevons) for elementary aggregation, rather than the arithmetic
mean of price relatives formula (Carli) that is largely used in the RPI.
1.7.5 United States of America
The Bureau of Labor Statistics (BLS) publishes three similar CPI series, they are:
• CPI for all urban consumers (CPI-U)
• chained CPI for all urban consumers (C-CPI-U)
• CPI for urban wage earners and clerical workers (CPI-W).
The CPI-U and CPI-W have different reference populations. The reference population of
the CPI-U covers all urban households, but excludes households living in rural nonmetropolitan areas and in institutional and non-private dwellings. In total, the reference
population of the CPI-U is about 87 percent of the USA’s total population. The CPI-W is a
subset of this population, covering households whose main source of income is from
wages and clerical workers’ earnings (about 32 percent of the total population) but does
not include those who earn salaries. Both indexes use the same price information, but
have different weighting patterns.
The C-CPI-U covers the same reference population as the CPI-U. However, it uses a
Törnqvist superlative index formula that uses expenditure weights from the current period
22
Principal purpose of the CPI
and the weight reference period. The C-CPI-U is produced to approximate substitution
bias in the USA’s CPI.
Preliminary results for the C-CPI-U are released at the same time as the CPI-U. There
are two scheduled revisions to the C-CPI-U, which change the preliminary results to
‘interim’ and then ‘final’ results as appropriate expenditure data becomes available. The
revisions happen in February each year. Currently, 2013 C-CPI-U results are preliminary,
2012 results are interim, and results before 2012 are final. Preliminary and interim results
are calculated using a geometric Young index formula to estimate price movement above
the elementary aggregate level. This formula keeps expenditure shares fixed, rather than
keeping quantities fixed, as in the Laspeyres-type formula used in most CPIs. The
geometric young formula assumes that households change the quantity of goods
purchased relative to changes in price. Final results are calculated using the Törnqvist
superlative index formula, which uses newly available expenditure data.
The BLS also produces an index called ‘the experimental CPI for Americans 62 years of
age or over’ (CPI-E). The reference population of the CPI-E is households headed by
someone aged 62 years or older. The CPI-E is simply a reweighted CPI, and no extra
products are tracked to account for the different spending patterns (eg specific brands) of
these households.
The BLS uses the cost of living (COLI) approach as the standard for its CPI. The COLI
approach measures the changing cost of maintaining a fixed standard of living, rather
than a fixed basket of goods and services. However, the changing cost of a fixed
standard of living is difficult to observe accurately. In practice, the USA CPI uses a fixed
basket approach with a use conceptual framework, including an imputed-rent
methodology for owner-occupied housing.
1.8 A single CPI versus multiple indexes
In principle, having multiple indexes designed for different purposes is desirable.
However, there are advantages to maintaining a single headline index, which the 2004
CPI Revision Advisory Committee recognised. Some countries produce measures of
consumer price change under more than one conceptual framework, but not without
some challenges. Table 1.4 below summarises the costs and benefits of a single CPI
versus multiple indexes.
Table 1.4
Costs and benefits of a single CPI versus multiple indexes
Number of indexes
A 'single’ CPI
Costs
Benefits
• Design may reflect a
'general purpose'
index, rather than a
specialised one
• A general index may
not be accepted by
subpopulations
Multiple indexes
• May confuse users
• Users may be
'selective' with the
index that they use
• Extra cost to produce
23
• High public profile of
a long-standing index
• Well understood and
trusted by users
• Indexes can be
designed and used
for specific purposes
• Subpopulation
indexes may be
better accepted by
subpopulations
Principal purpose of the CPI
1.8.1 A single CPI
The advantage of maintaining a single CPI is the ability to utilise the CPI ‘brand’. Longestablished CPIs have a high public profile and general credibility, which a new and
separately constructed index might lack.
As a result of the New Zealand CPI’s high profile, the Reserve Bank has an incentive to
use the official CPI as its yardstick for measuring inflation. Doing this is part of a broader
strategy of building general public support for monetary policy tools.
However, the Reserve Bank uses a range of indicators to gauge inflationary pressures,
rather than focusing exclusively on the CPI. These indicators include analytical series of
‘core’ inflation, such as tradables/non-tradables, weighted median and trimmed mean
CPI, sectoral factor model and factor model, and all groups CPI excluding selected goods
and services (eg CPI excluding food).
There is also an incentive to use the CPI as a compensation index. Recipients of
government payments, or wages and salaries, are directly affected by the way their
payments are adjusted. Knowledge that these payments are indexed by an official,
objective, and reputable measure of consumer price change may make the indexation
process better accepted by the public. On the other hand, because the CPI has a range
of uses, it may be seen to not accurately reflect price change for subpopulations. This
may reduce the relevance of the CPI for this compensation purpose.
While there is an advantage to having a single, reputable, and widely-accepted CPI,
using a single CPI in New Zealand has caused some design trade-offs. While largely
designed to measure inflation, the CPI also incorporates some characteristics considered
better suited to a compensation index (eg including local authority rates). These tradeoffs make the CPI more of a ‘general purpose’ index, rather than an index that is
optimally suited for one specific purpose.
A potential cost exists in using the CPI for multiple purposes. This is particularly true if, for
example, the price change of an inflation index is materially different to a compensation
index. This raises the question of whether multiple indexes are needed.
1.8.2 Multiple indexes
A consideration in producing multiple indexes is the level of user demand. If any
additional indexes are to be created, they would need to be well used to justify the extra
expense. Credibility may affect user acceptance of a new index – users may still choose
to use the headline CPI because of its reputation, rather than its conceptual design.
To date, the level of demand for new indexes is not clear. Previous recommendations by
the 2004 and 1997 CPI Revision Advisory Committees to produce indexes using a
payment and/or use framework have not been implemented. This was due to resource
constraints, and being considered a low priority relative to other initiatives across the
suite of official statistics.
Users may be confused by a range of indexes, and some users may not fully understand
which index is best suited to their needs. This could result in choosing the index that
shows the most favourable outcome, rather than the index that is conceptually most
appropriate.
The UK example highlights some of the challenges in producing multiple indexes on
different conceptual frameworks.
The Australian situation is somewhat different. The use of the PBLCI for adjusting
pensioner payments came largely from government and user demand for an index to
inform these adjustments. Initially the ALCIs were produced as a continuation of the
Australian CPI, after the headline measure switched from a payment framework to an
24
Principal purpose of the CPI
acquisition framework in 1999. The ALCIs were then able to help inform debate about
potential changes to Australian superannuation adjustment. This debate eventually led to
the creation of the PBLCI.
For details on the cost of producing subpopulation indexes in New Zealand, see the next
chapter ‘Consumer price change for subpopulations’.
1.9 Conclusion
The main purpose of the New Zealand CPI is to measure inflation for monetary policy.
This was explicitly stated after the 1997 CPI Revision Advisory Committee. This purpose
has influenced the design of the CPI; the most notable design factors are the use of the
acquisition framework, and the exclusion of interest rates.
The CPI is also used widely as an index for adjusting payments such as New Zealand
Superannuation and welfare payments, and for informing wage and salary negotiations.
The use of the CPI for compensation purposes is also reflected in the design of the CPI,
though less emphasis is placed on this purpose. Including items such as local authority
rates in the basket of goods and services is an example of this.
The result of having several purposes is that New Zealand’s CPI is more of a ‘general
purpose’ index, rather than one that is optimally designed for a specific purpose. One
advantage to maintaining a single CPI for multiple purposes is the ability to utilise the CPI
‘brand’ as a reputable, objective, and reliable measure of household price change.
However, there is a potential cost in using an index that is mainly designed to measure
inflation to adjust payments.
25
2 Consumer price change for subpopulations
2.1 Executive summary
This chapter explores the feasibility of producing additional indexes that are conceptually
better suited to measuring changes in households' living costs. The discussion includes
exploring the possibility of producing indexes of consumer price change for
subpopulations (population subgroups).
An empirical investigation of feasibility has been undertaken using data currently
collected. The following three subpopulation groupings were considered:
• ethnic groups
• recipients of government transfer payments (superannuitants and beneficiaries)
• income groups.
The choice of appropriate conceptual frameworks (acquisition, payment, use) is
discussed. The acquisition and payment frameworks are explored in the feasibility study.
Differences between the price indexes using different conceptual frameworks, and
analysis of the distribution of inflation across subpopulations, highlight the effect that the
choice of conceptual framework and reference population can have on consumer price
indexes.
The CPI user community and 2013 CPI Advisory Committee are invited to consider a
number of issues relevant to the topic of consumer price change for subpopulations.
These are outlined under ‘2.2 issues to consider’.
2.2 Issues to consider
2.2.1 Costs and benefits of producing subpopulation indexes
Potential uses of indexes of consumer price change for subpopulations are:
• as a compensation index
• to help understand how the CPI varies across subpopulations.
A compensation index is one where the index’s primary purpose is to adjust monetary
payments to maintain purchasing power or maintain living standards.
It is estimated that the ongoing cost of producing subpopulation indexes would be around
$40,000 per year including overheads. This cost would cover the calculation, quality
checking, and publication of the indexes. The cost to develop and maintain the relevance
of these indexes is estimated to be about $80,000 including overheads, initially and then
every three years. This cost would cover calculating and analysing expenditure weights,
and incorporating new weights into the production system.
The costs have been estimated with the following assumptions:
• a suite of five subpopulation indexes
• indexes published annually
• indexes published separately to the CPI
• indexes calculated quarterly
• quality checks carried out quarterly
• no extra households sampled in the Household Economic Survey
26
Consumer price change for subpopulations
• no additional price collection.
If it were considered necessary to increase data collection to produce fit-for-purpose
indexes then the costs would be considerably higher. Improving the quality of the
subpopulation expenditure weights would mean additional costs to increase the sample
size of the Household Economic Survey. For example, increasing the achieved sample
size of the survey by one-third every third year, would raise the three-yearly total cost
(direct costs and overheads) of conducting the survey by about 7 percent ($270,000).
Total, three-yearly, respondent load would increase about 21 percent – from about
16,500 hours to 20,000 hours (for comparison, the three-yearly respondent load for the
Household Labour Force Survey is about 30,000 hours). The cost increases for larger
sample size increases would be roughly proportional to this. The HES is currently being
redeveloped to simplify and rationalise the questionnaire for the 2015/16 survey. It is
anticipated that this will decrease the collection costs and reduce respondent load.
Improvements to the quality of the estimates of price change, for example to reflect
differences in where subpopulations shop, may require more price quotes to be collected.
Additional outlets and products that better represent the specific shopping habits of
subpopulations could be surveyed.
The International Labour Organization’s (ILO) (2003) resolution concerning consumer
price indices provides the following advice on subpopulation indexes:
Significant differences in the expenditure patterns and/or price movements
between specific population groups or regions may exist, and care should be
taken to ensure that they are represented in the index. Separate indices for these
population groups or regions may be computed if there is sufficient demand to
justify the additional cost.
2.2.2 Appropriate conceptual frameworks
The New Zealand CPI is currently complied on an acquisition framework. This
framework is most suited to measuring consumer price inflation for monetary policy
purposes. A payment or use framework is generally considered preferable to an
acquisition approach for a compensation index.
The main practical differences between these frameworks relate to the treatment of
owner-occupied housing and related costs. For additional details see the ‘Principal
purpose of the CPI’ chapter.
A payment framework is explored in the feasibility study. The payment framework is
generally considered suitable for a compensation index that aims to maintain purchasing
power. The payment index has been calculated as the CPI as currently defined but with
the following changes:
• plus interest
• excluding net acquisition of owner-occupied housing
• using gross expenditure weights for insurance.
The treatment of housing, including mortgage interest payments, other interest payments,
and insurance payments, represent major differences between the payment, acquisition,
and use frameworks.
A use framework is another generally considered suitable for a compensation index,
particularly for one that aims to maintain living standards. With a use framework, it is the
value of the shelter services being consumed that is used to derive the expenditure
weight for owner-occupied housing. Shelter value can be derived by estimating the
implicit (ie imputed) rental value of the stock of owner-occupied housing, or the user cost
of the shelter consumed, which may include real costs such as insurance or rates, as well
as notional costs such as depreciation.
27
Consumer price change for subpopulations
Publishing indexes of consumer price change using different conceptual frameworks, as
official analytical series, would require careful labelling and clear explaining of the
differences between indexes. This would help ensure users choose the index that is
conceptually most appropriate for their use. Subpopulation indexes could be released on
a different publication timetable to the CPI. Quarterly indexes could be published once a
year, as recommended by the 2004 CPI Revision Advisory Committee, if this would be
sufficient to meet user requirements.
2.2.3 Quality considerations
Using expenditure weight and price data that is currently collected, estimates of price
change for subpopulations would have the following limitations:
• Weight estimates would be less precise for subpopulations, compared with all
households, due to the smaller number of HES respondents for any given subgroup
(ie the sampling errors would be higher).
• The sample of outlets and products is designed to produce representative
measures of price change for the CPI reference population. Ideally, specific outlets
and products representative for each subpopulation would be used in each
subpopulation index.
A cost-neutral way to increase the quality of subpopulation indexes would be to reallocate
the HES sample, and/or the collection of price quotes, to be more representative of
particular subpopulations. However, with no additional data collection, this would be at
the expense of the quality of the ‘all household’ estimates from the HES and/or CPI.
Statistical quality has many dimensions. A balance between relevance and more
technical dimensions, such as accuracy, needs to be found. An important assessment of
quality is user feedback – do the statistics meet expectations, based on users’ knowledge
and judgement?
2.2.4 Questions to consider
Demand
• Is there sufficient demand for indexes of consumer price change for subpopulations
to justify the additional cost? If so, for which subpopulations? Why are they needed
and how would they be used?
CPI priorities
• If no additional resources were available, is the need for subpopulation indexes
sufficiently great to justify decreasing the current allocation of resources for other
areas of CPI measurement? (For example, decreasing the number of regional
centres used for price collection to fund subpopulation indexes.)
Conceptual framework
• What is the appropriate conceptual framework(s) for subpopulation CPIs?
Publication frequency
• If there is sufficient demand for indexes of consumer price change for
subpopulations, how frequently should they be published? Would annual
publication of quarterly indexes be sufficient?
28
Consumer price change for subpopulations
Quality requirements
• Would the quality of subpopulation indexes based on existing data collection (such
as those presented in the feasibility study) be sufficient to publish these indexes, or
would additional data collection be required?
Sample design
• Is the need for quality subpopulation indexes of sufficient priority to justify
reallocating the HES sample, and/or modifications to existing price collection, even
if this is at the expense of reduced quality of the all households CPI and/or HES
estimates for all households?
Additional data collection
• Should the HES sample size be increased to improve the quality of subpopulation
price indexes? Should price collection be expanded to better reflect the spending
patterns of subpopulations? If both, which is more important: expanded price
collection or an increase in the HES sample size?
Household Economic Survey priorities
• If no additional resources are available for the HES, or if there is a cap on total
respondent load, what is the balance of priorities for the quality dimensions of the
survey? For example, should the frequency of the annual income survey be
reduced (to two-yearly or three-yearly), or should some questions (eg annual
housing costs module, economic living standards module) be removed to increase
the quality of potential subpopulation CPIs?
2.3 Context for subpopulation indexes
2.3.1 Recommendations of recent CPI advisory committees
This chapter explores the feasibility of producing additional indexes that are conceptually
better suited to measuring changes in households' living costs. The costs and benefits of
producing these indexes are outlined, appropriate conceptual frameworks are discussed,
and international practices are presented. The chapter concludes with an empirical
investigation of feasibility using data currently collected.
The 2004 CPI Revision Advisory Committee concluded that the headline measure of the
CPI, branded as the ‘CPI’, should remain an acquisition-based index (recommendation 1)
and that a supplementary index (or indexes) should be produced that would be more
suited conceptually to measuring changes in households' cost of living. It was noted that
the supplementary index (or indexes) should take account of changes in the cost of living
for different subpopulations.
The 1997 CPI Revision Advisory Committee had reached a similar conclusion. This
committee recognised the importance of having a credible measure of the CPI that was
suitable for monetary policy purposes. However, there were differences in committee
members’ views about the need to make changes to the CPI, given its history as a
compensation-type index. A compromise was agreed on, and the 1997 CPI Revision
Advisory Committee recommended that a change should be made to the official all
groups CPI, making it more suitable for measuring inflation. The committee confirmed
that an acquisition framework should be used to construct the CPI. The CPI was to
measure price change based on goods and services actually acquired by households,
and would exclude interest payments (and residential section prices). However, the
committee recommended that the official CPI be supported by two additional price
indexes that, in concept, would be more suitable for compensation purposes.
29
Consumer price change for subpopulations
2004 CPI Revision Advisory Committee – Recommendation 2: Statistics New
Zealand should provide another index (or indexes) as separate series that would be
more suited conceptually to measuring changes in households’ cost of living. This
additional index (or indexes) need not be published on as frequent a basis as the
CPI, and an annual frequency would probably be sufficient to meet most users’
requirements. The precise methodology for producing a credible and robust cost of
living measure would be left to Statistics New Zealand to explore. However, in
producing an index, or a range of indexes, Statistics New Zealand should take
account of changes in the cost of living for different population subgroups such as
superannuitants, wage and salary earners, low-income households, and recipients
of government transfer payments.
1997 CPI Revision Advisory Committee – Recommendation 2: Statistics New
Zealand should publish a set of three measures of consumer price change, and the
Consumers Price Index should be an acquisitions measure which does not include
interest. The other two indexes will be an index of the price change of household
outlays and an index of the price change of household consumption. To maintain
public confidence in the measurement of price changes affecting households, the
new measures will be introduced when a suite of real disposable income indexes is
also available.
Little progress has been made towards implementing these recommendations, due to
resource constraints and other work being accorded a higher priority.
Following the 2004 CPI Revision Advisory Committee, Statistics New Zealand
commissioned Methods for Constructing Spatial Cost of Living Indexes (Melser & Hill,
2007). This report examined the differences between conceptual approaches to price
indexes, focusing on the impact of the differences in a spatial context. Spatial price
indexes compare differences in price level between regions, at a given point in time.
Regional-spatial cost of living indexes are discussed further in the ‘Sampling framework’
chapter.
Statistics NZ consulted widely with possible users of temporal and/or regional-spatial cost
of living indexes. The possible development of these measures received solid support
from potential users. Additional resources were sought to progress this work but the
budget bid was unsuccessful.
In 1999, interest was excluded from the headline CPI. The CPI plus interest is published
as an analytical series. The removal of interest was made on the grounds that the
principal purpose of the CPI is to measure inflation, under an acquisition framework. The
intention at the time was to subsequently develop other measures of price change (one
using the payment framework and one using the use framework). To date, such
additional indexes have not been produced due to resource constraints.
2.3.2 Māori statistical needs
Te Puni Kōkiri expressed interest in an index of consumer price change for Māori
households as part of the public call for topics for the 2013 CPI Advisory Committee.
Ethnic groups, including Māori and non-Māori households, have been included in the
section ‘2.10 Feasibility study for indexes of consumer price change’ of this chapter.
Statistics NZ’s Strategic Plan 2010–20 states that, in keeping with the Treaty of Waitangi,
Statistics NZ will recognise the uniqueness accorded Māori as tangata whenua. One of
the desired outcomes of the organisation’s Strategic Priority 1 (Lead the Official Statistics
System so that it efficiently meets the country’s needs for relevant, trustworthy,
accessible information) is that decision-making in the Official Statistics System is wellinformed and responsive to Māori statistical needs and interests. Also, that Statistics NZ
produces high-quality official statistics that are relevant to Māori, and promotes the
30
Consumer price change for subpopulations
recognition and consideration of Māori needs and aspirations across the Official Statistics
System.
This chapter on consumer price change for subpopulations provides an opportunity to
consider and explore the costs, benefits, and feasibility to provide relevant, trustworthy,
accessible information on Māori, and ultimately to reflect te ao Māori through the CPI.
2.3.3 Use of the CPI for indexation
The CPI is currently used to adjust a wide range of monetary payments – to maintain
purchasing power. Table 2.1 illustrates some of the important uses.
Table 2.1
Use of the CPI for indexation
Payment
Index(es) currently used
New Zealand Superannuation
•
•
Many benefit rates including:
•
•
•
•
•
unemployment benefit
sickness benefit
invalids benefit
domestic purposes benefit
Tobacco excise tax
•
•
Alcohol excise tax
•
All groups CPI excluding cigarettes
and tobacco
Benchmarked to a minimum of 66%
of average net ordinary time weekly
earnings
All groups CPI excluding cigarettes
and tobacco
CPI less credit services
Additional 10% increases have been
announced for 1 January 2014–16
CPI less credit services
2.4 Past practice of presenting price change for
subpopulations
Statistics NZ published an experimental series called the superannuitants price index
(SPI) in the late 1990s. The SPI measured change in consumer prices as it affected
superannuitant households (a household with one or more people aged 60 or over who
received New Zealand Superannuation and a superannuitant had the highest income
within the household). The SPI was published as two sub-indexes, split by tenure of the
household (‘in rented accommodation’ and ‘in their own accommodation’). The index time
series and weight information are in appendix 2a.
The index was discontinued after the June 1999 quarter. The main reasons given at the
time were:
• The 1997 CPI Revision Advisory Committee recommended that further
development of the SPI was a low priority.
• There were known limitations to the SPI when it was first constructed in 1995 (eg it
assumed superannuitants purchased similar goods and services at similar outlets
and prices to the average New Zealander). Several organisations had expressed
reservations about the usefulness of a measure with these sorts of limitations.
31
Consumer price change for subpopulations
• The major conceptual change to the CPI in 1999, which excluded interest
payments and section prices, made the headline measure less suited as a living
cost index.
• The SPI was originally based on a living cost definition. Continuing to calculate the
SPI based on the new CPI definition would mean covering a reduced range of
goods and services.
• It was concluded that Statistics NZ would not be able to update superannuitant
spending patterns as frequently as the CPI weights. Consequently, the difference
between the CPI and the SPI was viewed as likely to become less and less
meaningful.
In the feasibility study, undertaken for this paper, a superannuitant household price index
was calculated using the same household definition as the SPI, except the age was
raised to 65 years, to reflect the current age of eligibility for New Zealand
Superannuation.
An age beneficiaries price index was produced in 1975, but stopped because it did not
show significantly different results from the main CPI. The 1988 Report on the Revision of
the CPI noted that this was "at least partly attributable to the fact that it used the same
price survey data as the main index and that price indexes are more sensitive to what is
price-surveyed than to the expenditure weights assigned to those items."
The 1988 CPI Revision Advisory Committee rejected the idea of separate indexes for
specific population subgroups. The 1988 Report on the Revision of the CPI concluded
“there was little demand or need for such series and that they would not therefore justify
the expense of creating them”. The same report also noted “the concern that a multiplicity
of CPIs for different groups could lead to confusion and acrimony when parties met to
consider what was the appropriate series to use in specific circumstances”. In contrast,
the 2004 CPI Revision Advisory Committee recommended that Statistics NZ should take
account of changes in the cost of living for different population subgroups.
2.5 Conceptual frameworks for subpopulations
The choice of an appropriate conceptual framework depends on the use of the price
index. The ILO’s (2003) resolution concerning CPIs provides the following advice:
• The ‘acquisition’ approach is often used when the primary purpose of the index is to
serve as a macroeconomic indicator.
• The ‘payment’ approach is often used when the primary purpose of the index is for
the adjustment of compensation or income.
• the ‘use’ approach may be most suitable when the primary purpose of the index is
to measure changes in the cost of living.
Potential uses of indexes of consumer price change for subpopulations are:
• as a compensation index
• to help understand how the CPI varies across subpopulations.
As a compensation index, subpopulation CPIs could be used to: adjust government
transfer payments to households, inform employees and employers when negotiating
wages, or adjust income tax thresholds. Either a payment or use framework would be
most suited to this purpose.
32
Consumer price change for subpopulations
A payment framework is explored in the feasibility study. The payment index has been
calculated as the CPI:
• plus interest
• excluding net acquisition of owner-occupied housing
• using gross expenditure weights for insurance.
The treatment of housing, including mortgage interest payments, other interest payments,
and insurance payments, represents major differences between the payment, acquisition,
and use frameworks.
With the use framework, it is the value of the shelter services being consumed that is
used to derive its expenditure weight. Shelter value can be derived by estimating the
imputed rental value of the stock of owner-occupied housing, or the user cost of the
shelter consumed, which may include real costs such as insurance or rates, as well as
notional costs such as depreciation. The use framework was not investigated in the
feasibility study due to time constraints.
Subpopulation CPIs on the same, acquisition conceptual framework as the current CPI
are likely to be most useful in understanding how subpopulations experience price
change. Such indexes would allow analysis of the distribution of acquisition-based
household inflation across the population.
2.6 Dissemination of subpopulation indexes
Publishing indexes of consumer price change for subpopulations as official analytical
series would require clear differentiation from the CPI. If the headline, acquisition-based
CPI were used for monetary policy targeting, and other subpopulation-specific indexes
were used for compensation, then users may be unsure which series is the most
appropriate for their purpose. This could result in users picking the index that shows the
most favourable outcome, rather than the index that is conceptually most appropriate.
This risk has been managed internationally by carefully labelling and explaining the
different indexes. To some extent, publishing and explaining multiple price indexes is an
area in which Statistics NZ is already well versed. The organisation currently publishes
indexes of producer and labour prices alongside the CPI and the current range of
analytical CPI series.
The subpopulation indexes could be differentiated by clearly naming the indexes, and
explaining the differences. Subpopulation indexes could be released on a different
publication timetable to the CPI. They could be published once a year if this was sufficient
to meet user demand.
2.7 Quality of indexes of price change for
subpopulations
2.7.1 Dimensions of statistical quality
The Principles and protocols for producers of Tier 1 statistics provides the following
guidance on achieving an acceptable level of statistical quality:
For many years the focus was on accuracy, but during the last decade a much
broader understanding of quality has emerged. Quality is now usually defined as
fitness for use in terms of user perspectives and includes a number of
dimensions, the priorities of which may vary across different groups of users.
Quality of statistics refers to all aspects of how well statistics meet users’ needs
and expectations of statistical information, once disseminated. The decision and
33
Consumer price change for subpopulations
actions that achieve [an acceptable level of quality] balance are based on
knowledge, experience, reviews, feedback, consultation, and judgment.
The Quality Management Protocol for the Official Statistics System identifies six key
dimensions of statistical quality:
• relevance
• accuracy
• timeliness
• accessibility
• coherence
• interpretability.
Relevance is explored in the section ‘2.3 Context for subpopulation indexes’. Producing
consumer price indexes for subpopulations would fulfil recommendation 2 from the 2004
CPI Revision Advisory Committee, and align with more recent user interest.
Accuracy, reliability, and methodological soundness are discussed in the sections that
follow. There is a balance between ideal methods and data sources, costs, and other
dimensions of statistical quality.
Coherence is explored in the feasibility study by looking at the consistency of
subpopulation weights between the 2006/7 and 2009/10 HES. Some of the subpopulation
weighting patterns have been confronted against other data sources. For example, the
New Zealand Tobacco Use Survey found that in 2009 the smoking rate for Māori was 44
percent compared with 18 percent for non-Māori (Ministry of Health, 2011). Tobacco
weights were found to be higher for Māori households than non-Māori households in our
consumer price change for subpopulations feasibility study. Many consistent patterns are
found when comparing the weighting patterns for superannuitant and beneficiary
households, presented in the feasibility study, with the subpopulation weights in the
Australian analytical living cost indexes for similar household types (see appendix 2c).
An important assessment of quality is user feedback. The CPI user community and 2013
CPI Advisory Committee are invited to provide comment on how the feasibility study
results (particularly the differences in the expenditure weights for the subpopulations)
align with their expectations.
2.7.2 Basket weights for subpopulations (upper-level aggregation)
The precision of the expenditure weights will be lower for subpopulations than for the
CPI, as only a proportion of respondent households in the HES belong to a given
subgroup. Whether or not the current sample size of the survey can support fit-forpurpose expenditure-weight estimates for subgroups is explored in the feasibility study.
Some CPI expenditure weights are estimated, wholly or in part, using data sources other
than the HES. This is for a variety of reasons explained in the expenditure weighting
section of the paper Consumers Price Index Advisory Committee 2013 Background
Paper. The majority of these additional data sources, primarily business or government
administrative or survey data, are unlikely to have information that allows estimation of
expenditure by population subgroup. For example, expenditure weights for alcohol and
tobacco are estimated from New Zealand Customs data – expenditure on these products
is underestimated in the HES. The Customs data is available only at a national aggregate
level.
Where alternative data is used for CPI weights, the all-households estimate provides a
population benchmark. The ratio of the differences between the HES estimates and those
calculated from other sources provides a scaling factor for all households. This factor can
be applied to the HES estimates for each subpopulation. This will adjust the survey
34
Consumer price change for subpopulations
estimates, up or down, to account for survey variance, response bias, or conceptual
differences between the survey and the CPI.
Expenditure weights for construction of new dwellings in the CPI are currently estimated
wholly from data sources other than the HES. The survey does not ask questions relating
to the purchase of new housing. It is therefore necessary to estimate these weights for
subpopulations separately. An apportioning method was used in the feasibility study. Net
additions to housing for each population subgroup were estimated by multiplying the total
expenditure weight for net additions to housing by the proportion of the housing stock
owned by each respective subpopulation. An alternative method is described in the
section ‘2.10 further developments’.
The defined target population for the HES is private New Zealand resident households.
This definition is sufficient for constructing weight estimates for the current scope and
coverage of the CPI. The exclusion of institutionalised households, such as people living
in rest homes or hospitals, may present more acute coverage limitations when estimating
expenditure for subpopulations.
2.7.3 Basket-level price change for subpopulations (elementary
aggregates)
Ideally, the sample of price quotes used in each subpopulation index would be specific to
that subpopulation. It is likely that the outlets, brands, and pack sizes that best represent
the spending patterns for specific subpopulations would be slightly different for each. The
relative importance of outlet types and regions would also ideally be specific for each
subpopulation.
In the feasibility study, no modifications were made to the calculation of price change for
each of the goods and services in the CPI basket. This assumes the price change for all
households represents price change for a specific subpopulation. The extent to which
price change, at the CPI basket level, varies for each subpopulation, compared with the
estimate of national price change, would be a limitation of estimates of price change for
subpopulations – using currently collected data. Suggestions to test and enhance the
quality of CPI basket-level measures of price change for subpopulations are given in the
section ‘2.10 further developments’.
In principle, region-specific weighting for subpopulations is a desirable way to capture the
geospatial distribution of a subgroup, and their expenditure patterns, across regions. This
would better reflect the price change for a given group. In practice, the sample size of the
HES, by population subgroup and region, may be too small to increase the overall quality
of estimates of price change for subpopulations. Region-specific subpopulation weights
could be estimated using national subpopulation weights, apportioned using subgroupspecific population shares. The current method of estimating regional expenditure
weighting uses this way of population apportionment to calculate regional expenditure
weights.
2.8 International comparisons of measuring price
change for subpopulations
The scope and coverage of CPIs varies internationally. The ‘Principal purpose of the CPI’
chapter provides an overview of international practice for the conceptual design of CPIs.
Calculating and publishing CPIs for subpopulations also varies across countries. Some
national statistical agencies produce official CPI estimates for subpopulations. Research
on this topic has been done in many countries that do not have official estimates. The
research and publication practices in selected countries are outlined in this section.
35
Consumer price change for subpopulations
The ILO’s (2003) resolution concerning consumer price indices provides the following
advice:
Significant differences in the expenditure patterns and/or price movements
between specific population groups or regions may exist, and care should be
taken to ensure that they are represented in the index. Separate indices for these
population groups or regions may be computed if there is sufficient demand to
justify the additional cost.
The Report by the Commission on the Measurement of Economic Performance and
Social Progress (Stiglitz et al, 2009), which has been helping to shape the future
development of economic statistics internationally, highlighted the importance of
understanding the distributional aspects of inflation from a welfare perspective.
A point of particular relevance from a welfare perspective is the question about
whose price index is evaluated. Often, conceptual discussions about price indices
are conducted as if there were a single representative consumer. Statistical
agencies calculate the increase in prices by looking at the costs of an average
bundle of goods. However, different people buy different bundles of goods (eg
poor people spend more on food and less on entertainment) and they may buy
their goods and services in different types of stores (which sell “similar” products
at very different prices). When all prices move together, having different indices
for different people may not make much of a difference. But recently, with soaring
oil and food prices, these differences may have become more marked and people
at the bottom of the income distribution may have seen real incomes fall by much
more than those at the top of the income distribution.
2.8.1 Australia
Australia’s headline CPI is based on the acquisition framework. The Australian Bureau of
Statistics also publishes analytical living cost indexes (ALCIs) for four household
groupings:
• employee
• age pensioner
• other government transfer recipient
• self-funded retiree.
In addition, a pensioner and beneficiary living cost index (PBLCI) is published. The PBLCI
combines the age pensioner household and other government transfer recipient ALCIs.
The analytical living cost indexes are based on a payment conceptual framework. Interest
payments are included in the living cost indexes and gross (rather than net) insurance
premiums are used.
Analytical living cost indexes have been published by the Australian Bureau of Statistics
since June 2000. Initially they were published once a year, but have been published
quarterly since September 2009 (Australian Bureau of Statistics, 2011a). From
September 2009, most Australian social security pensions have been indexed by the
greater of the CPI and the pensioner and beneficiary living cost index. Pension rates are
also adjusted for improvements in wages, as measured by male total average weekly
earnings (Department of Families, Housing, Community Services and Indigenous Affairs,
2012).
The 2009/10 Australian Household Expenditure Survey sample was increased by about
3,000 households (from the 2003/04 survey) to include additional sampling of age
pensioner and other government transfer recipient households (Australian Bureau of
Statistics, 2011b). The achieved sample size for the 2009/10 survey was nearly 10,000
36
Consumer price change for subpopulations
households, compared with an achieved sample size of about 3,000 households for New
Zealand’s HES.
For further explanation of the indexes see Selected Living Cost Indexes, Australia Sep
2012 (Australian Bureau of Statistics, 2012).
2.8.2 Canada
Canada’s CPI is available with a regional breakdown. No further subpopulation CPIs are
currently published.
2.8.3 Germany
The German Federal Statistical Office produced three subpopulation indexes until
December 2002 (Mehrhoff et al, 2009). The subpopulations were:
• four-person households with higher income
• four-person households with middle income
• two-person pensioner households with low income.
The German sample survey of income and expenditure (EVS), used to update CPI
weights, is a quota survey conducted every five years with an achieved sample size of
about 53,000 households. The continuous household budget survey (LWR) is run
continuously, based on a subsample of the EVS, with an achieved sample size of about
8,000 households a year.
2.8.4 Japan
The Statistics Bureau of Japan has published a CPI for ‘retired elderly households’ since
August 2011 (Maruyama, 2011). Japan’s CPI is also published for selected cities and city
groups (small, medium, large city), and income quintiles for workers’ households.
Differences in regional price levels are also published in Japan.
Expenditure weights for Japan’s CPI are updated from two household surveys. The
Family Income and Expenditure Survey is a monthly survey of 9,000 households (oneperson households are surveyed for three months, larger households for six months).
The National Survey of Family Income and Expenditure runs every five years and has a
sample size of 57,000 households (Statistics Bureau of Japan, 2009).
2.8.5 United Kingdom
Two consumer price indexes are currently published in the UK. The consumer prices
index (CPI) is an acquisition-based index complied using the methodology of the
European harmonised index of consumer prices (HICP). HICPs exclude most elements of
owner-occupied housing costs, due to lack of consensus on appropriate methods. The
retail prices index (RPI) includes mortgage interest payments, house depreciation,
insurance, council tax, and house purchase costs (eg estate agents' and conveyancing
fees).
The CPI reference population covers all UK households, including institutional
households and foreign visitors to the UK. Expenditure by UK households abroad is
excluded. The RPI reference population excludes institutional households, pensioner
households (households with at least 75 percent of their income from state pensions and
benefits), and high-income households (the top 4 percent of the income distribution). For
further details of calculation methods and coverage see Differences between the RPI and
CPI measures of inflation (Office for National Statistics, 2010). In March 2013, an
additional index will be launched, titled CPI-H, which will be the CPI including owneroccupied housing costs, using the imputed-rent method (Office for National Statistics,
2012a).
37
Consumer price change for subpopulations
RPI pensioner indexes are published for one- and two-person pensioner households
(pensioner households are not in-scope for the main RPI). Housing costs are excluded
from these subpopulation indexes (Office for National Statistics, 2013). In 2011, the
Institute for Fiscal Studies produced estimates of the Retail Prices Index, for 2000–10, by
income deciles, benefit dependency, age of head of household, family composition, and
housing tenure. For details of their findings and methods used see The spending patterns
and inflation experience of low-income households over the past decade (Levell &
Oldfield, 2011).
The sample size of the Living Cost and Food Survey, equivalent to New Zealand’s HES,
is around 6,000 responding households per year (Office for National Statistics, 2012b).
Around 3,000 households responded to the 2009/10 HES.
2.8.6 United States of America
The Bureau of Labor Statistics publishes several CPIs with different reference
populations. The CPI for all urban consumers (CPI-U) excludes households living in rural
non-metropolitan areas, and institutional households. The CPI for urban wage earners
and clerical workers (CPI-W) is based on the expenditure of households included in the
CPI-U definition that also meet two requirements: more than half the household's income
comes from clerical or wage occupations, and at least one of the household's earners has
been employed for 37 weeks or more during the previous 12 months.
Since 1987, the Bureau of Labor Statistics has produced an experimental CPI for
Americans aged 62 years and older (CPI-E) (Stewart, 2008). The index is calculated
using the same basket items, retail outlets, and price quotes as the headline CPI. The
upper-level weighting is calculated from the expenditure patterns of older Americans.
Social security benefits are adjusted using the CPI for urban wage earners and clerical
workers (CPI-W). Some policymakers have advocated using the CPI-E to adjust benefits
instead. In 2009, the achieved sample size of the US Consumer Expenditure Survey,
equivalent to the NZ HES, was about 36,000 in the interview survey and 7,100 in the
diary survey (Mathiowetz, 2011).
2.9 Feasibility study for indexes of consumer price
change for subpopulations
To better understand the feasibility of producing indexes of consumer price change for
subpopulations a study based on currently available data was undertaken.
The following three subpopulation groupings were considered:
• recipients of government transfer payments (superannuitants and other
beneficiaries)
• income groups
• ethnic groups.
The first two groupings were suggested by the 2004 CPI Revision Advisory Committee (in
recommendation 2). Te Puni Kōkiri expressed an interest in an index of consumer price
change for Māori households as part of the public call for topics for the 2013 CPI
Advisory Committee.
Regional population estimates are discussed in the ‘Sampling framework’ chapter.
2.9.1 Economic context of the study period
The study period is the June 2008 quarter to the September 2012 quarter. CPI weights
were updated in the June 2008 quarter and June 2011 quarter, based on the 2006/7 and
2009/10 HESs, respectively. New Zealand experienced a six-quarter recession from the
38
Consumer price change for subpopulations
March 2008 quarter to the June 2009 quarter. The Official Cash Rate was progressively
reduced from 8.25 percent in June 2008 to 2.5 percent in June 2009.
Figure 2.1
Gross domestic product
(1)
Quarterly change
March 2008–September 2012 quarters
2.0
Percent
1.5
1.0
0.5
0.0
-0.5
-1.0
-1.5
Mar Jun Sep Dec Mar Jun Sep Dec Mar Jun Sep Dec Mar Jun Sep Dec Mar Jun Sep
2008
2009
2010
2011
2012
1. Seasonally adjusted chain-volume series.
Source: Statistics New Zealand
Figure 2.2
Official cash rate
March 2008–September 2012
10
Percent
8
6
4
2
0
Mar Jun
2008
Sep Dec Mar Jun
2009
Sep Dec Mar Jun
2010
Sep Dec
Mar Jun Sep Dec Mar Jun
2011
2012
Source: Reserve Bank of New Zealand
2.9.2 CPI on a payment conceptual framework
An index of price change under a payment conceptual framework was calculated for each
subpopulation and for the CPI reference population. A payment approach is generally
considered preferable to an acquisition approach for a compensation index (ie one where
the index’s primary purpose is to adjust monetary payments to maintain purchasing
power or living standards). An important potential use of indexes of consumer price
change for subpopulations is as a compensation index. This index was calculated as the
CPI plus interest, excluding net acquisition of housing, and using gross expenditure
weights for insurance (this series is labelled CPI – payment in figure 2.3). The treatment
39
Sep
Consumer price change for subpopulations
of housing, including mortgage interest payments, other interest payments, and insurance
payments, represents major differences between the payment and acquisition
frameworks.
The price change for interest has been calculated using the interest rates analytical
series (as currently published) multiplied by the price change for purchase of newly built
houses. Combining interest rates and house prices aims to ensure that the nominal value
of debt represents the same quantity and quality over time. Alternative methods for
calculating price change for interest payments are discussed in the section ‘2.10 Further
developments’.
In the acquisition-based CPI, insurance expenditure weights are calculated using a net
approach. Under this approach, the weight given to insurance relates to the
administrative costs of providing the service (ie collecting premiums and paying claims),
and the profits of insurance companies. Spending on goods and services that is funded
by insurance claims is allocated to the expenditure weights of those goods and services.
Under the payment framework, insurance weights are calculated using a gross approach.
This approach views the cost to the consumer of having insurance cover as being the full
amount of the premium paid to the insurance company. The service obtained by the
household in exchange for this payment (the insurance premium) is the security of
knowing that, when necessary, property will be repaired or replaced, or the health
services or income will be supplied as specified in the policy. When payments are made
for repairs, replacements, medical services, or income, these are regarded as being met
by the insurance company not by the households.
Figure 2.3
Comparison of CPIs using different conceptual frameworks
June 2002–September 2012 quarters
1200
Base: June 2006 quarter (=1000)
Index
1150
CPI - analytical
CPI - payment
1100
1050
1000
950
Study period
900
850
Jun
2002
Jun
2003
Jun
2004
Jun
2005
Jun
2006
Jun
2007
Jun
2008
Jun
2009
Jun
2010
Jun
2011
Jun
2012
Source: Statistics New Zealand
Figure 2.3 shows the payment-based CPI compared with an analytical, acquisition-based,
all groups CPI. The analytical CPI has been calculated as the CPI with the following
modifications to aid comparability:
• alternative June 2002 quarter housing weights
• alternative June 2002 quarter car insurance weights
• seasonally unadjusted fresh fruit and vegetable treatment.
The alternative 2002 housing weights, which were compiled using methods that are
consistent with the 2006, 2008, and 2011 housing weights, better reflect falling homeownership rates in New Zealand.
40
Consumer price change for subpopulations
The alternative 2002 car insurance weights removed unintended double counting of
expenditure in the original data, which would have been magnified under the payment
approach.
At the 2006 reweight, the official all groups index became fully seasonally unadjusted. Up
until the June 2006 quarter, fresh fruit and vegetable prices used in calculating the all
groups index were seasonally adjusted. In the series compiled for this paper, the 2002
weights for fresh fruit and vegetables and the fresh fruit and vegetable prices from 2002
to 2006 were not seasonally adjusted.
The payment-based CPI has been calculated using the full methods described above for
the study period (June 2008–September 2012 quarters). The payment series has been
backcast to the June 2002 quarter, using the analytical CPI series and a ratio adjustment
to convert insurance weights from a net to gross basis.
Over the study period, the payment-based CPI had a lower rate of price change. Interest
rates fell over this period. The influence of lower price change for the interest group was
partly offset by a larger expenditure weight for insurance, which increased at a faster rate
than the CPI overall.
Looking at the longer timeseries (June 2002–September 2012 quarters) it can be seen
that the payment-based CPI had a higher rate of price change from the June 2002
quarter to the start of the study period. Differences between acquisition- and paymentbased indexes are often cyclical, influenced by changes in interest rates. This means that
caution should be used when making conclusions about the differences between
acquisition- and payment-based indexes over the four-year study period.
41
Consumer price change for subpopulations
Figure 2.4
Effect of conceptual frameworks on CPI weights
June 2008 and June 2011 quarters
100
Percent
7.1
90
1.8
9.5
80
3.2
10.9
8.8
1.6
6.8
1.8
9.1
3.5
16.2
60
5.1
3.0
14.9
50
15.1
5.4
4.4
5.3
8.6
1.7
8.5
8.8
70
8.2
4.7
3.3
14.1
5.1
4.1
4.8
40
23.6
22.7
4.5
20
10
0
6.8
6.2
17.8
16.4
CPI
CPI - Payment
June 2008 quarter
Education
Recreation & culture
Communication
Transport
Health
Household contents & services
Housing & household utilities
Clothing & footwear
4.4
4.1
Misc. goods & services
18.3
15.8
30
Interest
6.9
4.1
Alcoholic beverages & tobacco
6.5
Food
18.8
17.6
CPI
CPI - Payment
June 2011 quarter
Source: Statistics New Zealand
Figure 2.4 shows the relative expenditure weights under the payment framework,
compared with current CPI expenditure weights. Including interest payments in the CPI
decreases the relative importance of the other expenditure groups. The weight for
housing and household utilities is lower under the payment framework, due to
expenditure on the purchase of newly built houses being excluded. The weight for
miscellaneous goods and services increases under the payment conceptual framework
due to the inclusion of gross insurance weights.
2.9.3 Sampling errors
An important factor influencing the reliability of expenditure estimates by subpopulation is
the sample size of the subpopulation in the HES. The sampling errors of expenditure
estimates are inversely proportional to the sample size. The sampling errors of
subpopulations broadly follow the rule that a decrease in the sample size by a factor of m,
results in an increase in the sampling errors by a factor of one over the square root of m.
For example, in the 2009/10 survey, Māori households were about 15 percent of the
responding households. This means the sampling errors of expenditure estimates for
Māori households are about two and a half times the sampling errors of expenditure
estimates for all households.
42
Consumer price change for subpopulations
2.9.4 Ethnic groups
Ethnicity is the ethnic group or groups that people identify with or feel they belong to.
Ethnicity is self-perceived and people can belong to more than one ethnic group.
Respondent households have been allocated to one or more ethnic subpopulation(s)
based on one or more people in the household reporting they belong to that ethnic group.
The ethnic groups are not mutually exclusive. Households can belong to more than one
ethnic group as individuals can have multiple ethnicities and/or because different
members of a household have different ethnicities. The exception to these coding rules is
the non-Māori ethnic group, which has been defined as households not classified as
Māori households.
The definition of household ethnicity used is consistent with that used by Te Puni Kōkiri in
Māori families and households (Te Puni Kōkiri, 2011), where a Māori household is
defined as any household with at least one Māori member.
Table 2.2 shows the sample sizes of ethnic subpopulations in the latest two HESs. Pacific
people have the smallest sample sizes in both surveys. The sample size for Māori is
broadly comparable with the smallest sample size for the published regional breakdown
(five broad regions). Wellington region had a sample size of 382 in the 2006/7 survey.
Rest of the South Island had a sample size of 490 in the 2009/10 survey.
Regional sample sizes are shown in the ‘Sampling framework’ chapter.
Table 2.2
Household Economic Survey achieved sample sizes
By ethnicity
Subpopulation
European
Māori
Pacific people
Asian
Other
Non-Māori
All households
Sample size (households)
2006/07
2009/10
2,164
2,607
321
476
120
187
186
255
162
246
2,229
2,650
2,550
3,126
The HES datasets used in the feasibility study are those used to calculate the CPI
weights. Survey method changes made after the initial survey publication were not
incorporated, to maintain consistency with the CPI weights.
Imputation methods were introduced for the 2009/10 HES. Imputation replaces missing
values with actual values from similar respondents. This was applied to a household
where the household did not supply all the required information, but supplied sufficient
information to be retained in the sample. The effect of introducing imputation was to
increase the number of usable households, and so increase the achieved sample size.
43
Consumer price change for subpopulations
Figure 2.5
Expenditure weights for households classified by ethnicity (payment framework)
Expenditure weights for households classified by ethnic group (1)
June 2008 and June 2011 quarters
100
Percent
11.1
8.9
80
1.2
9.2
2.9
60
14.6
5.1
8.0
9.3
8.9
7.3
1.7
1.3
9.0
3.2
13.9
8.2
6.5
1.7
7.6
7.7
7.6
7.2
3.1
6.0
3.0
3.4
0.4
6.4
1.0
5.8
13.0
13.2
3.9
13.4
16.6
3.2
3.3
4.7
1.7
3.0
5.0
14.8
4.3
20
0
17.1
21.3
7.3
3.7
6.8
6.6
3.7
3.8
27.5
22.8
16.3
16.5
3.5
3.2
2.6
4.1
4.1
6.6
6.8
8.9
8.1
16.4
17.4
17.0
17.8
16.1
2008
2011
2008
2011
2008
European
Māori
22.8
4.0
3.1
6.5
6.5
19.3
2011
Pacific people
Ethnic group
1. Payment framework
Source: Statistics New Zealand
44
Misc. goods & services
Education
Recreation & culture
Transport
Health
Household contents & services
23.8
3.4
Interest
Communication
19.4
4.3
10.0
1.2
2.8
4.3
40
11.3
6.9
3.3
5.5
8.2
11.5
Housing & household utilities
Clothing & footwear
2.7
2.9
3.2
2.6
16.5
16.9
2008
2011
Alcoholic beverages & tobacco
Food
Asian
Consumer price change for subpopulations
Figure 2.6
Difference in expenditure shares for Māori households compared with
Difference in expenditure shares for Māori
households when compared with all households (1)
June 2008 and June 2011 quarters
CPI group
Housing & household utilities
Alcoholic beverages & tobacco
Food
Communication
Interest
Education
Clothing & footwear
Household contents & services
Transport
Recreation & culture
Jun-11
Misc. goods & services
Jun-08
Health
-2
0
2
4
Percentage points
6
1. Payment framework
Source: Statistics New Zealand
Figure 2.6 shows the percentage-point difference in the group-level expenditure shares
for Māori households compared with all households. This was calculated as the Māori
households expenditure share less the all-households expenditure share. Some
consistent patterns were found in the 2008 and 2011 expenditure weights for Māori
households.
The estimated expenditure weights for Māori households were higher than for all
households for:
• housing and household utilities
• alcoholic beverages and tobacco
• food.
The estimated expenditure weights for Māori households were lower than for all
households for:
• household contents and services
• transport
• recreation and culture
• miscellaneous goods and services
• health.
No consistent pattern was found for other expenditure groups.
45
Consumer price change for subpopulations
Figure 2.7
Acquisition- and payment-based consumer price indexes for ethnic groups
Base: June 2008 quarter (=1000)
June 2008–September 2012 quarters
1120
Index
1100
1080
1060
1040
1020
1000
J
2008
S
D
M
2009
J
S
D
M
2010
J
S
D
M
2011
J
All households - acquisition
All households - payment
European - acquisition
European - payment
Māori - acquisition
Māori - payment
Pacific people - acquisition
Pacific people - payment
Asian - acquisition
Asian - payment
Non-Māori - acquisition
Non-Māori - payment
S
D
M
2012
J
Source: Statistics New Zealand
Figure 2.7 shows the point estimates of price change for ethnic groups from the feasibility
study. From the June 2008 quarter to the September 2012 quarter consumer price
indexes for European households tracked all households very closely. This is not
surprising, as European households are the largest ethnic group. Māori households had
the largest price increases – as measured by both the acquisition and payment
conceptual frameworks. Asian households recorded the lowest price increases.
Due to the small sample sizes for ethnic groups in the HES, there are relatively few cases
where the survey provides enough statistical power to make strong conclusions about the
statistical significance of the differences in rates of price change between ethnic groups.
46
S
Consumer price change for subpopulations
Table 2.3
Average annual percent change – June 2008 quarter to September 2012 quarter
By ethnic group
Acquisitionbased
framework
Group
All households
European
Māori
Pacific people
Asian
Non-Māori
Percent
2.35
2.37
2.40
2.27
2.12
2.35
Absolute
sampling error
due to index
weight
estimation
Percentage
point
0.03
0.04
0.19
0.21
0.17
0.04
Paymentbased
framework
Percent
1.65
1.66
1.75
1.48
1.32
1.63
Absolute
sampling error
due to index
weight
estimation
Percentage
point
0.04
0.05
0.23
0.44
0.31
0.05
Sampling errors in table 2.3 and error bars in figure 2.8 show the uncertainty in the
estimates of price change, due to the sampling of households in the HES. Uncertainty
due to other sampling methods used to calculate the CPI, such as sampling of products
and outlets, is not accounted for in these errors. The error bars represent 95 percent
confidence intervals for the estimates of price change. They were calculated using the
delete-a-group jacknife method. One hundred replicate groups were used. The sampling
errors are 95 percent confidence interval half-widths; adding or subtracting the sampling
errors provides the upper and lower confidence limits, respectively.
Figure 2.8
Annual percent change in consumer price indexes for ethnic groups(1)
September quarters, 2009–12
5
Percent
Pacific people
4
Non-Māori
Māori
All households
European
Asian
3
2
1
0
2009
2011
2010
2012
September quarter
1. Acquisition framework
Source: Statistics New Zealand
Under the acquisition framework, all households experienced greater price change in the
year to the September 2011 quarter. This was influenced by the increase in the rate of
goods and services tax (GST) from 12.5 percent to 15 percent on 1 October 2010.
Analysis of the point estimates, shown in figure 2.8, suggests that European households
experienced annual price change that was very similar to price change for all households,
under the acquisition framework. European households had higher price change than all
households in the years to September 2009 and 2011, but lower price change in the
years to September 2010 and 2012.
47
Consumer price change for subpopulations
In the feasibility study, the point estimates for Māori household price change, under the
acquisition framework, were higher than for all households in three of the four years. Over
the period from the June 2008 quarter to the September 2012 quarter Māori households
had annual average price change of 2.40 percent, compared with all households’ annual
average price change of 2.35 percent. The 95 percent confidence intervals for these
estimates are [2.21, 2.58] and [2.32, 2.39] for Māori and all households, respectively. The
confidence intervals are conditional on the basket-level price change being the true price
change for each population. The 95 percent confidence intervals for the annual average
price change under the payment framework were [1.52, 1.99] for Māori households and
[1.61, 1.69] for all households.
48
Consumer price change for subpopulations
Figure 2.9
Main contributions to annual percent change (1)
Māori and non-Māori households
June 2008 quarter–September 2012 quarter
5
Percentage point
Misc. goods & services
Recreation & culture
4
Transport
1.4
Health
1.2
3
0.2
2
0.3
0.4
0.2
0.2
0.3
0.5
1
0.4
1.0
0.7
0.4
0.8
-0.6
-2
-3
-1.6
-1.0
Māori
-1.9
0.3
Other
0.3
0.2
0.5
(2)
Housing & household utilities
Alcoholic beverages & tobacco
0
-1
0.6
0.5
0.2
0.9
0.7
0.5
Interest
0.4
0.4
1.2
1.1
0.6
0.3
-0.7
-0.2
0.3
-0.3
-0.2
-0.3
-0.3
-0.2
-0.2
Māori
NonMāori
Food
All groups
-0.8
NonMāori
Māori
NonMāori
Māori
NonMāori
September 2009 September 2010 September 2011 September 2012
1. Payment framework
2. Includes clothing & footwear, household contents & services, communication, and education groups
Source: Statistics New Zealand
Under the payment framework, the contributions accounting for the largest differences in
the average annual change in prices for Māori households, compared with non-Māori
households, were:
• alcoholic beverages and tobacco – mainly due to tobacco contributions being
higher for Māori households
• miscellaneous goods and services – mainly due to dwelling insurance contributions
being higher for non-Māori households
• interest – mainly due to mortgage interest payments having a greater downward
contribution for non-Māori households.
49
Consumer price change for subpopulations
Figure 2.10
Differences in percentage point contributions to average annual change
For ethnic groups when compared with all households
June 2008 quarter–September 2012 quarter
0.6
Percentage point difference
Interest
Misc. goods & services
0.4
Education
Recreation & culture
0.2
Transport
Health
0.0
Housing & household utilities
-0.2
Alcoholic beverages & tobacco
Other
(1)
-0.4
-0.6
Māori
Pacific
people
Asian
Acquisition-based framework
Māori
Pacific
people
Asian
Payment-based framework
Ethnic group
1. Includes food, clothing & footwear, household contents and services, and communication groups.
Source: Statistics New Zealand
During the study period, the point estimates for Pacific people showed lower overall price
change than all households. Lower contributions came from health, education, interest
(payment framework), and miscellaneous, which were partly offset by higher contributions
for alcohol and tobacco, housing, and transport.
The Asian ethnic group also had point estimates that showed lower overall price change
than all households, during the study period. Lower contributions came from alcohol and
tobacco, and interest (payment framework), somewhat offset by higher contributions for
education and housing.
50
Consumer price change for subpopulations
2.9.5 Government transfer payments
For the purpose of this feasibility study households were grouped as:
• Superannuitant – households with one or more people aged 65 years or older,
who received New Zealand Superannuation, and where a superannuitant had the
highest household income.
• Beneficiary – households where the highest income recipient received a benefit
payment (see appendix 2b for a list of benefits covered).
• Main beneficiary – households where the highest income recipient received a
benefit payment, classified as a ‘main benefit’ in appendix 2b.
The superannuitant and beneficiary group comprises households belonging to one, or
both, of the separate groups.
The definition of superannuitant households is consistent with the classification of
households used in the superannuitants price index (published in the late 1990s – see
the section ‘2.4 Past practice of presenting price change for subpopulations’), except the
age criterion is 65 years to reflect the current age of eligibility for New Zealand
Superannuation.
Table 2.4 shows the achieved sample sizes in the most recent HESs for each of the
Government transfer payment subpopulations. The superannuitant and beneficiary
subpopulation covers around one-third of the respondent households.
Table 2.4
Household Economic Survey achieved sample sizes
By government transfer status
Sample size (households)
Subpopulation
Superannuitant and beneficiary
Superannuitant
Beneficiary
Main beneficiary
All households
2006/07
926
547
494
324
2,550
51
2009/10
1,098
663
559
370
3,126
Consumer price change for subpopulations
Figure 2.11
Expenditure weights for subpopulations (1)
Classified by government transfer recipient status
June 2008 and June 2011 quarters
2008
2011
Government transfer recipient status
All households
17.6
Superannuitant & beneficiary
19.1
Superannuitant
19.1
Beneficiary
19.4
Main beneficiary
19.3
All households
16.4
Superannuitant & beneficiary
17.0
Superannuitant
17.2
Beneficiary
17.0
Main beneficiary
16.6
0
18.3
5.1
23.1
14.1
7.4
17.1
4.7
17.3
26.7
5.5
6.2 5.6
5.8 3.9
8.8
8.8
10.9
16.4
3.6 11.6
28.4
20
9.7
5.1
14.1
9.7
40
8.1 3.5
3.5 11.4
14.9
6.4
8.2
10.9
4.1 11.0
32.3
22.0
8.4
12.9
30.1
8.6
12.1
10.6
15.8
8.5
8.5
9.7
7.3
2.8 12.5
7.0
60
80
9.3
5.1
10.4
8.5
8.2
7.4
6.1
100
Percent
Food
Clothing & footwear
Household contents & services
Transport
Recreation & culture
Miscellaneous goods & services
Alcoholic beverages & tobacco
Housing & household utilities
Health
Communication
Education
Interest
1. Payment framework
Source: Statistics New Zealand
Figures 2.11 and 2.12 show that beneficiary households spent relatively more than all
households on: housing and household utilities, food, and alcoholic beverages and
tobacco under the payment framework. Superannuitant households spent relatively more
than all households on: health, food, recreation and culture, miscellaneous goods and
services, and household contents and services. Relatively consistent patterns were found
in the June 2008 quarter and June 2011 quarter weights.
52
Consumer price change for subpopulations
Figure 2.12
Difference in expenditure patterns
(1)
For government transfer recipients compared with all households
June 2008 and June 2011 quarters
CPI Group
Superannuitant
Jun-11
Health
Recreation & culture
Superannuitant
Jun-08
Food
Main beneficiary
Jun-11
Miscellaneous goods & services
Household contents & services
Main beneficiary
Jun-08
Communication
Clothing & footwear
Alcoholic beverages & tobacco
Housing & household utilities
Transport
Education
Interest
-10
0
Percent
10
20
1. Payment framework
Source: Statistics New Zealand
Over the June 2008 quarter to September 2012 quarter period the point estimates for all
the government transfer groups showed greater price change than all households.
Superannuitant household point estimates had the greatest price change in the study
under both the acquisition- and payment-based frameworks.
53
Consumer price change for subpopulations
Table 2.5
Average annual percent change by government transfer recipient status
June 2008 quarter to September 2012 quarter
Acquisitionbased
framework
Absolute
sampling
error due to
index weight
estimation
Percentage
point
0.03
0.10
Paymentbased
framework
Absolute
sampling error
due to index
weight
estimation
Percentage
point
0.04
0.15
Percent
Percent
Group
2.35*
1.65*
All households
Superannuitant
2.60*
2.43*
and Beneficiary
Superannuitant
2.69*
0.10
2.87*
0.12
Beneficiary
2.55*
0.16
2.10*
0.26
Main beneficiary
2.57*
0.23
2.21*
0.30
* Statistically significant difference from all households, 95 percent confidence level.
Using the 95 percent confidence intervals, conditional on the basket-level price change
being the true price change for each subpopulation, there is strong statistical evidence
that:
• the annual average price change over the study period for superannuitant
households, and for superannuitant and beneficiary households, was different to
the all household price change, under both the acquisition and payment
frameworks
• the annual average price change over the study period, for all government transfer
subpopulations considered, was different from the all household price change
under the payment framework.
54
Consumer price change for subpopulations
Figure 2.13
Acquisition- and payment-based consumer price indexes for subpopulations
Classified by government transfer recipient status
Base: June 2008 quarter (=1000)
2008 quarter–September 2012 quarter
1250
Index
1200
1150
1100
1050
1000
J
2008
S
D
M
2009
J
S
D
M
2010
J
S
D
M
2011
J
S
D
M
2012
J
All households - acquisition
All households - payment
Superannuitant - acquisition
Superannuitant - payment
Beneficiary - acquisition
Beneficiary - payment
Main beneficiary - acquisition
Main beneficiary - payment
Superannuitant and beneficiary - acquisition
Superannuitant and beneficiary - payment
NZ Superannuation rate
Source: Statistics New Zealand, Ministry of Social Development
Figure 2.13 shows the cumulative effect of price change over the study period, for each
government transfer group, under the two conceptual frameworks investigated. It can be
seen that the rate of New Zealand Superannuation increased more than any of the
consumer price change indexes shown.
55
S
Consumer price change for subpopulations
Table 2.6
Superannuation rates
Date from
Superannuation –
married couple net
(1)
rate
Explanation for increase
1 April 2008
$439.80
1 Oct 2008
$462.74
Increase due to reduction in tax rates – same
gross rate as 1 April 2008
1 April 2009
$478.38
Consistent with all groups CPI increase year
ended December 2008
1 April 2010
$489.42
Higher than all groups CPI increase to meet the
66% net average wage commitment
1 Oct 2010
$500.94
Increase due to reduction in tax rates – same
gross rate as 1 April 2010
1 April 2011
$522.96
Higher than all groups CPI excluding cigarettes
(2)
and tobacco increase, to meet the 66% net
average wage commitment
1 April 2012
$536.80
Higher than all groups CPI excluding cigarettes
and tobacco increase, to meet the 66% net
average wage commitment
1.
2.
Other superannuation rates are adjusted in a consistent way
From 1 April 2011, the CPI excluding cigarettes and tobacco used as a basis for adjusting
rates of social security benefits and New Zealand Superannuation. This was to ensure
that beneficiaries were not compensated for the increase in tobacco excise.
Source: Ministry of Social Development
56
Consumer price change for subpopulations
Figure 2.14
Contributions to average annual percent change (1)
By government transfer recipient status
June 2008 quarter–September 2012 quarter
Government transfer recipient status
All households
-0.1
Superannuitant
& beneficiary
0.5
0.7
-0.1
Main
beneficiary
-1.0
0.4
0.7
-0.1 -0.2
Superannuitant
Beneficiary
0.3
0.3
0.5
-0.6
-0.1
-0.5
0.3
0.8
-0.4
0.2
0.8
-0.3
0.0
0.5
1.0
0.6
0.2
0.3
0.3
0.4
0.3
0.5
1.5
0.6
0.1
0.3
0.6
0.7
0.4
0.3
0.2
0.6
0.3
0.1
0.6
2.0
2.5
Percent
Interest
Communication
Recreation & culture
Housing & household utilities
Education
Misc. goods & services
Alcoholic beverages & tobacco
Transport
Health
Other (2)
1. Payment framework
2. Includes food, clothing & footwear, and household contents & services groups
Source: Statistics New Zealand
Figure 2.14 shows that alcoholic beverages and tobacco, housing and household utilities,
and interest payments explain most of the difference in price change, over the study
period, for the government transfer recipient groups compared with all households. A
higher upward contribution from the health group was a noticeable difference for the
superannuitant group households.
2.9.6 Income groups
Households were grouped into five income groups (quintiles), using before-tax regular
and recurring income. Table 2.7 shows the household income thresholds used.
Table 2.7
Household income boundaries used to define income quintiles
Income quintile
Quintile 1
Quintile 2
Quintile 3
Quintile 4
Quintile 5
2006/07
Less than $26,021
$26,021–$44,939
$44,940–$68,324
$68,325–$99,999
$100,000 +
2009/10
Less than $28,932
$28,932–$51,440
$51,441–$76,092
$76,093–$110,799
$110,800 +
Shown in figure 2.15, expenditure weights by household income display some clear
patterns under the payment framework. The weight for housing and household utilities
decreases as household income increases. The relative importance of interest, and
recreation and culture increases with household income.
57
3.0
Consumer price change for subpopulations
Figure 2.15
Expenditure weight by household income quintile (1)
June 2008 and June 2011 quarters
100
Percent
3.8
10.6
1.1
80
8.2
3.8
12.1
60
5.0
4.6
7.4
11.6
1.6
8.6
7.2
1.2
2.8
8.4
3.3
14.1
14.3
20
0
8.6
8.6
1.8
1.9
8.2
10.1
2.8
15.8
5.6
3.5
4.6
24.4
12.6
9.2
4.6
40
12.6
21.4
17.2
3.9
5.0
14.6
5.1
4.8
7.9
8.2
2.3
0.7
7.9
6.8
7.9
1.7
3.9
4.1
10.5
13.9
14.7
6.8
9.4
9.6
9.4
8.7
1.4
2.2
8.4
9.6
3.0
3.7
3.7
Interest
Misc. goods & services
Education
Recreation & culture
2.8
Communication
13.9
5.6
5.5
5.0
8.1
3.6
2.7
15.2
8.2
5.1
4.3
4.1
4.1
15.0
4.9
Transport
Health
4.5
26.8
22.6
Household contents & services
18.1
17.0
11.4
2.9
3.8
3.8
6.9
6.3
7.1
14.8
5.5
Clothing & footwear
6.1
Alcoholic beverages & tobacco
2.6
2.8
4.1
4.0
5.1
5.5
6.5
6.6
6.0
6.3
18.2
16.9
16.5
16.7
15.5
17.7
18.8
18.3
18.0
16.3
1
2
3
4
5
1
2
3
4
5
2.8
5.7
June 2008 quarter
June 2011 quarter
Household income quintile
1. Payment framework
Source: Statistics New Zealand
58
Housing & household utilities
Food
Consumer price change for subpopulations
Figure 2.16
Acquisition- and payment-based consumers price indexes for income quintiles
Base: June 2008 quarter (=1000)
June 2008 quarter–September 2012 quarter
Index
1120
1100
1080
1060
1040
1020
1000
J
2008
S
D
M
2009
J
S
D
M
2010
J
S
D
M
2011
J
S
D
M
2012
J
Quintile 1
- acquisition
Quintile 2
- acquisition
Quintile 3
- acquisition
Quintile 4
- acquisition
Quintile 5
- acquisition
Quintile 1
- payment
Quintile 2
- payment
Quintile 3
- payment
Quintile 4
- payment
Quintile 5
- payment
Source: Statistics New Zealand
Shown in figure 2.16, clear patterns were found in the analysis of the point estimates of
consumer price change by household income. Regardless of the conceptual framework,
the distribution of price change was inversely proportional to household income for the
June 2008 quarter to September 2012 quarter period. Lower-income households had the
highest price change and higher-income households had the lowest price change.
59
S
Consumer price change for subpopulations
Figure 2.17
Contributions to average annual percent change
By household income quintile
June 2008 quarter–September 2012 quarter
3.0
Percent
Interest
Misc. goods & services
2.5
Education
Recreation & culture
2.0
Communication
Transport
1.5
1.0
0.8
0.7
0.8
0.6
Health
0.7
0.5
0.5
Household contents & services
0.5
0.4
1.0
0.4
0.4
0.4
0.4
0.3
0.4
0.4
0.4
0.4
0.3
0.5
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.5
Clothing & footwear
Alcoholic beverages & tobacco
Food
All groups
0.0
-0.2
-0.3
-0.5
-1.0
Housing & household utilities
1
2
3
4
5
Acquisition-based framework
1
2
-0.6
-0.7
-0.7
3
4
5
Payment-based framework
Household income quintile
Source: Statistics New Zealand
In figure 2.17, error bands shown for the all groups indexes are 95 percent confidence
intervals, conditional on the basket-level price change being the true price change for
each population. There is strong statistical evidence (regardless of the conceptual
framework) that:
• the highest income quintile experienced lower price change over the study period
than the lowest income quintile
• the highest income quintile experienced lower price change over the study period
than all households
• the lowest income quintile experienced greater price change over the study period
than all households.
Figure 2.17 shows the interest group made a large contribution to the differences in price
change between income groups under the payment framework. The relative weights for
interest payments increase with income. Interest rates decreased over the study period.
In periods when interest rates increase this pattern may be reversed.
60
Consumer price change for subpopulations
The housing and household utilities group helps to explain the differences in average
annual price change between income groups under both conceptual frameworks
investigated. The relative weights of housing and household utilities decreases as income
increases. Price change for this group was greater than for the all groups price change
over the study period.
2.10 Further developments
There are a number of areas of potential further research that may improve the quality of
the estimates of price change presented in the feasibility study. If the effects were thought
to be substantial, they would need to be addressed before official publication of indexes
of consumer price change for subpopulations.
2.10.1 Basket-level price change
An important assumption was required to undertake the work presented in the feasibility
study. For each subpopulation, national estimates of price change at the CPI basket level
have used the measures of price change used to compile the national CPI.
Subpopulation-specific weighting was used to aggregate price change in the upper-level
aggregation. This assumes the national, all households, basket-level indexes are
representative measures of price change for each subpopulation. The same assumption
is required to use the all households CPI for compensation of subpopulations (along with
an additional assumption that the all households upper-level weighting is appropriate).
The validity of the assumption could be tested empirically by studying the outlet and
product specific expenditure patterns of subpopulations. HES data could be used for this
purpose. The sample size may need to increase to make reliable conclusions. Depending
on the findings, additional price collection may be required and/or subpopulation specific
outlet/product weights may need to be used. A study by the Australian Bureau of
Statistics suggested the quality of the Australian pensioner and beneficiary living cost
index could be improved by including additional items and outlets in the price sample
(Australian Bureau of Statistics, 2013).
The earlier section ‘2.7 quality of indexes of price change for subpopulations’ discussed
regional weighting in the context of subpopulations. National elementary aggregates were
used in the feasibility study. The impact of regional specific weights could be investigated.
Incorporating subpopulation considerations into the overall design of the CPI sample also
has the potential to increase the quality of subpopulation estimates. For example,
regional, outlet, or product coverage could be modified to better reflect subpopulations. It
should be noted that additional subpopulation constraints would need to be balanced
against the effect on the quality of the national, all households CPI.
Regional coverage of ethnic groups is discussed in the ‘Sampling framework’ chapter.
2.10.2 ‘Use’ conceptual framework
The use framework was not investigated in the feasibility study due to time constraints.
Such work could be undertaken for comparison. An estimate of rental-equivalent price
change for owner-occupied households is calculated for the national accounts.
Indexes based on the use framework would be particularly helpful for users who
conceptually prefer this framework. Subpopulation indexes based on all three conceptual
frameworks (acquisition, payment, use) could be produced. Doing this would be
consistent with recommendation 2 of the 1997 CPI Revision Advisory Committee.
2.10.3 Price change for interest payments
Under the payment framework, the aim of the interest payments price index is to measure
the change, over time, in the interest that would be payable on a set of mortgages
61
Consumer price change for subpopulations
existing in the weight reference period. In the feasibility study, the price change for
interest payments was calculated as the interest payments analytical series (as currently
published) indexed by the price change for the purchase of newly built dwellings.
Combining interest rates and house prices aims to ensure that the nominal value of debt
represents the same quantity and quality over time.
This method could be enhanced by considering the debt profile of mortgage debt. Under
this method, interest payments price change is calculated separately for each age cohort
of debt and then combined.
It would also be desirable to consider, in greater depth, the most appropriate index to use
to index interest payments. For example, the dwelling price index could cover just new
dwellings (as in the feasibility study) or could also cover established dwellings and/or
land. Another alternative is to index the interest payments using a broader measure of
price change, such as the CPI, since mortgages are also used to fund other types of
expenditure.
2.10.4 Group definitions
The group definitions used could be refined or expanded – based on user demand and
the availability of reliable weight information from the Household Economic Survey.
Other subpopulations that could be investigated include:
• household composition
• equivalised household income quintiles (eg to account for size of household and/or
household composition)
• age demographics (eg age of household reference person).
2.10.5 Split of owner-occupied housing
Under the acquisition conceptual approach used to compile New Zealand’s CPI, the
expenditure weight allocated to ‘purchase of housing’ represents the value of the net
increase in the stock of owner-occupied housing during the weight reference period. This
includes expenditure on both newly constructed dwellings by owner-occupiers, and
alterations and additions to existing owner-occupied dwellings. Net transfers of
established housing into or out of the household sector either increase or decrease the
expenditure weight.
In the feasibility study, an apportioning method was used to estimate net additions to
housing for each subpopulation. The total expenditure weight for net additions to housing
was multiplied by the proportion of the housing stock owned by the subpopulation.
The method used in the 2008 and 2011 CPI reviews to calculate expenditure for all
households involved: applying an average new private-dwelling value to the estimated
net change in the number of owner-occupied dwellings, then adding an estimate of
owner-occupiers’ share of the value of residential building additions and alterations to
established dwellings. The estimated change in the number of owner-occupied dwellings
was based on population census data (extrapolated to 2008 and 2011, respectively).
Using census tenure data specific to each subpopulation, this method could be extended
to estimate the net increase (or decrease) in the stock of owner-occupied housing for
each subpopulation. However it is possible this method could lead to negative
expenditure weights for owner-occupied housing for some subpopulations.
Another alternative method would be to ask questions relating to the sale and purchase
of housing in the HES. Such an approach may require an increase in the sample size to
62
Consumer price change for subpopulations
produce estimates that are sufficiently reliable. It was decided that net capital outlay was
no longer in scope for the 2006/07 HES, so questions on the following expenditure areas
were removed from the questionnaire:
• construction of new dwellings
• purchase of land and buildings
• sale of land and buildings.
This decision partly reflected concerns over the reliability of estimates for these types of
expenditure at the ‘all households’ level.
The payment framework does not require estimates of owner-occupied housing for each
subpopulation. The purchase of housing is not considered consumption expenditure
under this framework (it is considered the purchase of an asset). Repayments of
mortgage principal are excluded as there is no change in net worth on household balance
sheets. Debt repayment results in a decrease in household cash reserves and an
increase in assets (or reduction in liabilities). Other housing-related costs, such as
interest, insurance, real estate fees, repairs and maintenance, and local authority rates,
are included under the payment framework.
2.10.6 Insurance weights
In the feasibility study, the gross insurance weights required under the payment
framework were calculated using information only from the HES. In contrast, the net
insurance weights calculated for the acquisition-based CPI used additional data sources.
Net insurance weights were calculated using household consumption expenditure
estimates from the national accounts and data from the insurance industry. It would be
desirable to explore if these additional data sources could also be used to provide better
estimates of gross insurance weights.
Using the net approach for insurance services, spending on goods and services that is
funded by insurance claims is allocated to the expenditure weights of those goods and
services. Such expenditure is not in scope under the payment approach. However, due to
time constraints, it was not possible to tailor the expenditure weights for the payment
approach to exclude expenditure funded by claims.
2.10.7 Potential changes to the Household Economic Survey
The reliability of estimates of price change for subpopulations is heavily dependent on the
reliability of the HES estimates. As the international comparisons earlier in this chapter
show, the sample size of the New Zealand survey is relatively small compared with
similar surveys used in some other countries. The smaller population of New Zealand is
unlikely to have a noticeable effect on the sampling errors. The sample size of the HES is
also among the smaller social surveys undertaken in New Zealand – see Te Waharoa
(Statistics New Zealand, 2012) for a comparison of sample sizes.
A cost-neutral way to increase the quality of subpopulation indexes would be to reallocate
the HES sample to better represent particular subpopulations.
However, with no additional data collection this would be at the expense of the quality of
the all-household estimates. For a discussion of methods to oversample ethnic
subpopulations see Sampling for Subpopulations in Household Surveys with Application
to Māori and Pacific Sampling (Clark RG et al, 2009).
Widening the coverage of the HES to cover rest homes and/or hospitals may provide a
quality increase for estimates of subpopulations with greater-than-average density in
these institutions (eg superannuitants).
63
Consumer price change for subpopulations
The quality of estimates of net acquisition of housing, for subpopulations and overall, may
be increased by including questions in the HES about the sale and purchase of dwellings.
See the section ‘2.10.5 split of owner-occupied housing’ above for a discussion of
reliability.
2.10.8 Democratically weighted CPI
Using aggregate expenditure for index weighting is often referred to as ‘plutocratic’
weighting. The expenditure patterns of high-spending households have more influence on
the index. The use of plutocratic weighting is generally considered appropriate for
measuring consumer price inflation for macroeconomic purposes.
An alternative weighting method is possible that gives the expenditure pattern of each
household an equal weight. Such weighting is often called ‘democratic’ weighting. It
would be possible to estimate indexes of democratically weighted consumer price change
using the HES. It can be argued that democratic weighting better reflects the inflation
experience of a typical household. Internationally, democratically weighted CPIs are
rarely computed (United Nations et al, 2009). The practice of the UK RPI, excluding the
top 4 percent of the income distribution for weight calculations, limits the impact of highincome households on plutocratic expenditure weights.
64
Consumer price change for subpopulations
2.11 Options for indexes of consumer price change for
subpopulations
Table 2.8
Options and implications for indexes of consumer price change for subpopulations
Option
Implications
Conceptual framework
•
•
•
•
acquisition
or
payment
or
use
•
•
•
Ethnic groups
•
•
•
•
Māori households
or
Māori and non-Māori
households
or
multiple ethnic groups
•
•
•
Government transfer payment groups
•
•
•
•
Superannuitant and
beneficiary households
separately or combined
•
Data collection
•
•
•
•
current data collection
increase the HES sample
size
increase collection of price
quotes
•
65
Acquisition framework is suitable for
macroeconomic purposes. Subpopulation
indexes on this basis would be useful to help
understand how the CPI varies across
subpopulations. Methods to split owneroccupied housing would require further
investigation.
Payment framework is suitable for a
compensation index that aims to maintain
purchasing power. Methods to measure price
change for interest payments would require
further investigation.
Use framework is suitable for a compensation
index that aims to maintain living standards.
Feasibility investigations would be required.
Subpopulation indexes could be produced on
one or more conceptual frameworks.
Producing subpopulation indexes for Māori
households would align with Statistics NZ’s
Strategic Plan 2010-20 and recent user
interest.
If indexes for only Māori households were
produced, these could be compared with ‘all
households’ indexes.
Producing indexes for non-Māori households,
as well, would allow all households to be split
into two mutually exclusive groups.
Producing multiple ethnic subpopulation
indexes would allow analysis for a greater
range of ethnicities. HES sample sizes are
smallest for Pacific people, Asian and ‘Other’
ethnic groups so the weight estimates, and
consumer price indexes, are likely to be less
precise for these groups than for Māori, nonMāori, European, and all-household
estimates.
Group definition(s) depend on user needs.
Estimated expenditure weights, and
consumer price indexes, are likely to be more
precise for the broader group –
superannuitant and beneficiary households –
than finer groupings.
Fit-for-purpose quality considerations depend
on user needs.
Cost and benefit trade-offs could be made to
avoid additional costs or load, but at the
expense of the quality of the ‘all households’
CPI.
Increasing sample sizes has cost implications
for Statistics NZ and increases respondent
load.
Consumer price change for subpopulations
References
Australian Bureau of Statistics (2013). Information paper: Experimental data in the
Pensioner and Beneficiary Living Cost Index. Feb 2013. Available from www.abs.gov.au
Australian Bureau of Statistics (2012). Selected living cost indexes, Australia. Sep 2012.
Available from www.abs.gov.au
Australian Bureau of Statistics (2011a). Analytical living cost indexes for selected
Australian household types. Mar 2011. Available from www.abs.gov.au
Australian Bureau of Statistics (2011b). Household Expenditure Survey, Australia:
Summary of results, 2009-10. Available from www.abs.gov.au
Clark, RG, et al (2009). Sampling for subpopulations in household surveys with
application to Māori and Pacific sampling. Official Statistics Research Series, 4. Available
from www.statisphere.govt.nz
Department of Families, Housing, Community Services and Indigenous Affairs (2012).
More pension increases – pension indexation. Available from www.fahcsia.gov.au
International Labour Organization (2003). “Resolution concerning consumer price
indices”, adopted at the Seventeenth International Conference of Labour Statisticians.
Levell, P, & Oldfield, Z (2011). The Spending Patterns and Inflation Experience of LowIncome Households over the Past Decade, Institute for Fiscal Studies Commentary
C119. Available from www.ifs.org.uk
Maruyama, A (2011). The index for retired elderly households, International Working
Group on Price Indices. Available from www.ottawagroup.org
Mathiowetz, et al (2011). Redesign Options for the Consumer Expenditure Survey,
Bureau of Labour Statistics. Available from www.bls.gov
Mehrhoff, et al (2009). Is inflation heterogeneously distributed among income groups?
International Working Group On Price Indices. Available from www.ottawagroup.org
Melser, D, & Hill, RJ (2007). Methods for constructing spatial cost of living indexes.
Official Statistics Research Series, Vol 1. Available from www.statisphere.govt.nz/
Ministry of Health (2011). Māori Smoking and Tobacco Use 2011. Available from
www.health.govt.nz
Moulton BR, & Stewart KJ (1997). An overview of experimental U.S. consumer price
indexes, International Working Group on Price Indices. Available from
www.ottawagroup.org
Office for National Statistics (2010). Differences between the RPI and CPI measures of
inflation. Available from www.ons.gov.uk
Office for National Statistics (2012a). Response to the consultation on: the recommended
method of reflecting owner occupiers’ housing costs in a new additional measure of
consumer price inflation; and the strategy for consumer price statistics. Available from
www.ons.gov.uk
Office for National Statistics (2012b). Living cost and food survey quality and
methodology information. Available from www.ons.gov.uk
Office for National Statistics (2013). CPI and RPI reference tables, January 2013, table
40. Available from www.ons.gov.uk
66
Consumer price change for subpopulations
Statistics Bureau of Japan (2009). 2009 National Survey of Family Income and
Expenditure Overview. Available from www.stat.go.jp
Statistics New Zealand (2012). Te Waharoa: directory of Māori statistics. Available from
www.stats.govt.nz
Stewart, KJ (2008). The experimental consumer price index for elderly Americans (CPIE): 1982–2007. Bureau of Labor Statistics. Available from www.bls.gov
Stiglitz, et al (2009). Report by the Commission on the Measurement of Economic
Performance and Social Progress. Available from www.stiglitz-sen-fitoussi.fr
Te Puni Kōkiri (2011). Māori families and households. Fact sheet 004-2011. Available
from www.tpk.govt.nz
United Nations, et al (2009). Practical guide to producing consumer price indices.
Available from www.unece.org
67
Consumer price change for subpopulations
Appendix 2a: Discontinued series
Appendix table 1
Superannuitants price index and consumers price index
Index series and percentage change
Base: December 1993 quarter (=1000)
Superannuitants price index
All households
All
superannuitant
households
Series ref:- SPIQ
SE9A
In rented
accommodation
SEAA
Consumers
price index –
In their own
accommodation
SEBA
All
groups
CPIQ.SE9A
Quarter
1997
Jun
Sep
Dec
1071
1078
1083
1118
1126
1129
1066
1072
1078
1083
1088
1094
1998
Mar
Jun
Sep
Dec
1085
1089
1096
1095
1135
1141
1154
1153
1078
1082
1088
1088
1096
1101
1107
1098
1999
Mar
Jun
1096
1103
1153
1159
1089
1096
1095
1097
Percentage change from previous quarter
Quarter
1997
Jun
Sep
Dec
0.4
0.7
0.5
0.4
0.7
0.3
0.4
0.6
0.6
0.1
0.5
0.6
1998
Mar
Jun
Sep
Dec
0.2
0.4
0.6
-0.1
0.5
0.5
1.1
-0.1
0.4
0.6
-
0.2
0.5
0.5
-0.8
1999
Mar
Jun
0.1
0.6
0.5
0.1
0.6
-0.3
0.2
Percentage change from same quarter of previous year
Quarter
1997
Jun
Sep
Dec
1.8
2.0
1.6
2.6
2.6
2.0
1.7
1.9
1.6
1.1
1.0
0.8
1998
Mar
Jun
Sep
Dec
1.7
1.7
1.7
1.1
2.0
2.1
2.5
2.1
1.5
1.5
1.5
0.9
1.3
1.7
1.7
0.4
1999
Mar
Jun
1.0
1.3
1.6
1.6
1.0
1.3
-0.1
-0.4
68
Consumer price change for subpopulations
Appendix table 2
Expenditure weights by group(1)
Percentage of total expenditure weight
Superannuitants price index
Group
All
superannuitant
households
Food
Housing
Household operation
Apparel
Transportation
Tobacco and alcoholic drinks
Personal and health care
Recreation and education
Credit services
All groups
In rented
accommodation
Total
17.76
19.50
14.92
4.50
14.58
8.46
6.27
7.50
6.50
100.00
100.00
100.00
100.00
Consumers
Price Index –
All
groups
4.48
24.31
5.83
6.13
3.78
5.75
3.34
2.16
4.67
10.73
4.11
13.20
2.87
5.53
3.11
4.14
30.29
6.39
4.37
4.50
5.35
2.91
1.82
4.30
11.57
3.36
11.53
1.98
4.66
2.83
100.00
100.00
1. Percentages may not add to 100 percent due to rounding.
69
All
groups
18.83
19.94
18.87
3.81
15.91
7.12
8.58
6.11
0.84
Percentage of total expenditure weight
Whangarei
Auckland
Hamilton
Tauranga
Rotorua
Napier-Hastings
New Plymouth
Wanganui
Palmerston North
Wellington
Nelson
Christchurch
Timaru
Dunedin
Invercargill
In their own
accommodation
20.14
25.77
19.46
2.31
9.31
8.95
9.50
4.08
0.47
Regional weights(1)
Superannuitants
price index –
All
households
price index –
18.79
20.71
18.97
3.66
15.16
7.32
8.69
5.92
0.79
Appendix table 3
Region
Consumers
All households
Consumer price change for subpopulations
Appendix 2b: Coding of government transfer payments
The coding used is consistent with the HES coding of income. Note that payments from
Inland Revenue, such as Working for Families tax credits, are not included under benefit
payments.
•
•
•
Supplements, benefits, and main benefits are used to define beneficiary.
Main benefits only are used to define main beneficiary.
New Zealand Government pensions are used to define superannuitant.
New Zealand Government pensions
•
•
•
Main benefits
Supplements
Benefits
70
New Zealand Superannuation paid by
WINZ
Overseas pension paid by WINZ using
special banking
Overseas pension top-up payment from
WINZ
•
Surviving spouse pension
•
Veterans pension
•
War disablement pension
•
Domestic purposes benefit
•
•
Emergency benefit
Emergency maintenance
benefit
•
Independent youth benefit
•
•
Invalids benefit
Orphans or unsupported
child’s benefit
•
Other type of main benefit
•
Sickness benefit
•
Student allowance
•
Unemployment benefit
•
Widow’s benefit
•
Accommodation supplement
•
Child disability allowance
•
•
Disability allowance
Family support paid by WINZ (Family
tax credit)
•
OSCAR subsidy
•
Other
•
Recover payment
•
Recoverable assistance payment
•
•
Replace payment
Special benefit (Temporary additional
support)
•
•
Special needs grant
Unspecified
Consumer price change for subpopulations
Appendix 2c: Australian expenditure patterns of
selected household types
Estimated average weekly expenditure during 2009–10, Australia
Household type by commodity group(a)(b)
Commodity group
Food and non-alcoholic
beverages
Alcohol and tobacco
Other
government
transfer
recipient
Selffunded
retiree(e)
CPI
PBLCI
Employee
Age
pensioner
AU$
AU$
AU$
AU$
AU$
AU$
135.92
262.48
120.72
150.86
176.66
230.87
56.19
114.63
36.12
75.92
68.16
96.87
32.27
64.39
26.31
38.12
43.29
54.58
135.91
205.32
92.43
178.64
113.89
305.75
Furnishings, household
equipment and services
60.00
142.82
59.81
60.20
119.83
124.79
Health
35.91
80.73
47.69
24.32
87.22
72.56
Transport
63.26
182.25
50.65
75.65
118.13
158.39
Communication
26.26
46.45
19.80
32.62
31.84
41.81
Recreation and culture
67.90
194.82
61.91
73.79
215.24
172.30
Education
Insurance and financial
services(d)
All groups
10.07
45.13
1.54
18.45
9.06
43.67
37.44
218.75
22.19
52.42
39.40
69.71
661.13
1,557.77
539.17
780.99
1,022.72
1,371.30
Clothing and footwear
Housing(c)
(a) Based on 2009-10 Household Expenditure Survey (HES) at June quarter 2011 prices.
(b) Figures may not add up due to rounding.
(c) House purchases are included in the CPI but excluded from the population subgroup indexes.
(d) Includes interest charges and general insurance. Interest charges are excluded from the CPI
and general insurance is calculated on a different basis.
(e) The average weekly expenditures for self-funded retiree households are based on estimates
at the State/Territory level, whereas estimates for the other household types (and the CPI) are
based on estimates at the capital city level.
Source: Australian Bureau of Statistics
71
3 Sampling framework
3.1 Executive summary
This chapter explains the current CPI sampling framework. It examines potential
enhancements and trade-offs that could be made to best meet users’ fit-for-purpose
quality requirements.
Changes made to the CPI sample design since the 2004 CPI Revision Advisory
Committee met are explained, as are recent investigations and potential future research.
The costs and benefits of regional-spatial indexes are also outlined.
There are many dimensions to sampling prices used in the CPI, including coverage of:
•
•
•
•
•
regions and subpopulations
goods and services
retail outlets and service providers
products
time (frequency of collection).
The CPI user community and 2013 CPI Advisory Committee are invited to consider
several issues relevant to the CPI sample design.
3.2 Issues to consider
3.2.1 Balance of competing regional and national data
requirements
Many choices are required in deciding how to allocate price collection effort. Given limited
resources, there is often a trade-off between the optimal sample allocation for the
national CPI and the requirements for fit-for-purpose regional indexes. For example, price
changes for lower-weighted items in smaller regions are unlikely to have a noticeable
impact on the national CPI. Yet, they may have a noticeable effect on the indexes at a
regional level, particularly at group level and below.
The current design is a balance between regional and national requirements. This
chapter, and ‘Consumer price change for subpopulations’ and ‘Dissemination of CPI and
FPI information’ explore the trade-off further, outlining potential alternative strategies. For
example, if the quality requirement for regional indexes were relaxed this could free-up
resources for potential alternatives – such as a monthly CPI or measures of consumer
price change for subpopulations. Larger differences were found between indexes of
consumer price change for subpopulations (in the feasibility study) than between the
regional CPIs for the June 2008 to September 2012 quarter period.
(See the ‘Consumer price change for subpopulations’ chapter for information about the
feasibility study mentioned above.)
An integrated sampling review was undertaken in 2006, as recommended by the 2004
CPI Revision Advisory Committee. This resulted in reallocating price collection towards
larger regions.
3.2.2 Regional coverage of subpopulations
The current sample for price collection in terms of products, and for field-collected items,
regions, and outlets, was designed to achieve reliable measures of price change for the
CPI reference population as a whole. Regional and outlet-type stratification are used to
help ensure good coverage.
72
Sampling framework
Increasing the design criteria to include additional assessment of the subpopulation
coverage (eg ethnic groups) could be beneficial for two reasons:
• To ensure that price collection for the national CPI is a representative average of all
households – including enhancing the public perception of coverage for specific
subpopulations.
• To increase the quality of measures of consumer price change for subpopulations
(if these were to be developed).
In some cases, improvements to coverage of where certain subpopulations shop may
oppose the efficiency of the optimal sample design for the all households, national CPI.
3.2.3 Modes of data collection
The current sample design for the CPI is multi-modal. Prices for about half the CPI (by
weight) are collected in the field by local price collectors visiting outlets. Other prices are
collected by postal survey, online, or from government administrative data. Retail
transaction data is being investigated as a potential future mode.
3.2.4 Costs and benefits of regional-spatial indexes
Regional-spatial price indexes compare price level differences between regions, at a
given point in time. Such indexes can be useful in comparing living costs across regions.
Regional-spatial price indexes require similar products to be priced in each region to be
compared, or more sophisticated methods that control for the differences in quality
characteristics. Since the current sample of price quotes is designed to measure price
change over time, the products priced are not necessarily comparable across regions.
The production of these indexes would require additional data collection or the
development of sophisticated methods. They have not yet been produced due to
resource constraints and limited user demand in recent years.
3.2.5 Questions to consider
• What balance should be struck between quality requirements for the national,
regional, and possible subpopulation CPIs? What sampling trade-offs should be
made to achieve this balance?
• Should coverage of subpopulations be considered when making decisions about
the sampling of prices; for example, the distribution of subpopulation expenditure
across regions, outlets, and products?
• Do the quality requirements for regional CPIs, and perceptions of credibility related
to regional coverage of New Zealand, justify the costs of field price collection in 15
urban centres or would fewer be sufficient? Alternatively, should prices be collected
in more regions or in smaller towns and/or rural areas?
• Is there sufficient demand for regional-spatial price indexes, which compare price
levels between regions, to justify the additional cost?
73
Sampling framework
3.3 Recommendations of the 2004 CPI Revision
Advisory Committee
The 2004 CPI Revision Advisory Committee made four recommendations that specifically
relate to the sampling framework. The recommendations cover the following topics:
• integrated sample review
• regional expenditure weights
• number of regional pricing centres
• regional-spatial indexes.
3.3.1 Integrated sample review
Recommendation 13: Statistics New Zealand should undertake an integrated
sample review of items, regions, field outlets and brands as part of the 2006 CPI
revision.
An integrated sample review was undertaken as part of the 2006 CPI review. The sample
of retail outlets that price collectors visit was reselected. The specifications of the
representative basket of goods and services were updated, and the sample of product
sizes, brands, and varieties was reselected (also done as part of the 2008 and 2011 CPI
reviews).
An ongoing, rolling review of the sample of prices for goods and services collected from
field outlets was established after the 2011 CPI review. The rolling review considers the
sample of outlets prices are collected from, as well as the product specifications and
brands of the goods and services collected from field outlets.
3.3.2 Regional expenditure weights
Recommendation 14: Statistics New Zealand should explore the possibility of
moving from national expenditure weights to using regional expenditure weights for
a number of broad regions, if the use of regional weights would improve the
accuracy of the national CPI. The use of regional weights would have the added
benefit of supporting the derivation of ‘fit-for-purpose’ indexes for regions such as
each of Auckland, Wellington, the rest of the North Island, Canterbury, and the rest
of the South Island.
Statistics NZ investigated the possibility of adopting regional expenditure weights and
reported on the investigation in a public consultation paper published in 2005, Review of
consumers price index regions. The outcomes of this consultation were published in
2006. In principle, Statistics NZ is in favour of adopting a regional expenditure weighting
approach. However, there was little evidence that this would improve the accuracy of the
national CPI. Regional expenditure weights have not been implemented in the CPI.
3.3.3 Number of regional pricing centres
Recommendation 15: Statistics New Zealand should consult with users in
reviewing the number of regional centres from which prices are to be collected.
Statistics NZ investigated the possibility of changing the regions that CPI prices are
collected from in a public consultation paper published in 2005, Review of consumers
price index regions. The outcomes of this consultation were published in 2006. Statistics
NZ decided to continue to collect prices from the existing 15 regional centres, but to
reallocate resources to align more closely with the population shares of the regions. This
resulted in more prices being collected from the larger pricing centres, particularly
Auckland. Reducing the number of regions that prices are collected from could affect
74
Sampling framework
some users’ perception of how credible the national CPI is as a national measure of price
change.
3.3.4 Regional-spatial indexes
Recommendation 16: The committee recognises that there are different regional
data requirements. One such requirement is for an absolute price level
measurement at a point in time, via ‘spatial’ indexes, to measure differences in the
cost of living in different regions. Regional spatial indexes would also complement
the range of indexes designed to measure changes in the cost of living over time
(Recommendation 2). Statistics New Zealand should seek to develop regional
spatial indexes. Consistent with Recommendation 2, it will not be necessary to
publish such indexes more frequently than on an annual basis.
Statistics NZ commissioned Daniel Melser and Robert Hill of the University of New South
Wales to develop a suitable methodology for constructing regional-spatial cost of living
indexes and to document findings in a research report. In Developing a methodology for
constructing spatial cost of living indexes, the authors outlined possible uses for these
indexes, explored possible conceptual frameworks, and discussed possible
methodological approaches. They concluded that a conditional cost of living index
constructed under a use framework would appear to be most appropriate.
The possible development of these measures received solid support from potential users.
However, it was not seen as a high priority relative to other initiatives across the suite of
official statistics, there has been little demand for them, and additional resources have not
been obtained.
3.4 Sampling framework
The ideal data requirements (target population) for producing a CPI are details of all inscope transactions undertaken by the CPI reference population. In practice, such data is
not readily available and complete collection is not feasible. Sampling of transactions is
required across the following domains:
• goods and services (basket items)
• products (brands, sizes, varieties)
• regions
• outlets
Geospatial dimension
• time dimension.
75
Product dimension
Sampling framework
Figure 3.1
CPI sample space
The sampling framework for collecting data to compile the CPI is multi-stage and multimodal. The CPI measures price change for a representative basket of goods and
services. Expenditure weights are calculated from the Household Economic Survey
(HES) and other sources on a three-yearly basis. For more details on the calculation of
expenditure weights, see Consumers Price Index Advisory Committee 2013 Background
Paper.
Prices are collected using the following modes:
• price collectors visiting retail outlets
• postal and electronic surveys to businesses
• online price collection
• administrative data.
Fuel, and fresh fruit and vegetable prices are collected weekly. The remaining food and
grocery prices are collected monthly, as are alcohol, tobacco, car hire, international air
fares, Internet charges, cut flowers, and newspapers. Most other prices are collected
quarterly. Prices that change only annually, such as local authority rates and school fees,
are collected annually.
Prices are collected in the field for just over half the items in the CPI basket (by
expenditure weight). Outlets are sampled in 15 urban centres. Within outlets, specific
products are tracked for each CPI item priced at that outlet.
Sampling based on judgement is used to choose representative goods and services,
products, regions, outlets, and time periods. Choices are guided by dissemination
requirements, available data (eg retail transaction data for supermarket products),
historical precedent, and international practice. Probabilistic sampling techniques are
used to select respondents to some postal surveys, where suitable sampling frames are
available.
The 2004 CPI Revision Advisory Committee recommended that Statistics NZ undertake
an integrated sample review of items, regions, field outlets, and brands (recommendation
13). This was done as part of the 2006 CPI review. An important outcome of this review
was to reallocate resources to align more closely with the population shares of the
regions. This resulted in more prices being collected from the larger pricing centres,
particularly Auckland.
As part of the 2006, 2008, and 2011 CPI reviews the specifications of the representative
basket of goods and services were updated, and the sample of product sizes, brands,
and varieties was reselected.
76
Sampling framework
The sample of retail outlets from which prices are collected was fully reselected in 2006.
In 2012, Statistics NZ began a rolling review of retail outlets visited. The review
considers:
• the retail outlets visited by Statistics NZ price collectors
• the detailed pricing specifications of products for the representative goods and
services (items) that are tracked for the CPI at retail outlets.
The first changes were implemented when the September 2012 quarter index was
released in October 2012. Further changes will be incorporated in each quarterly release
over a two-year cycle.
Retail transaction data is proposed as a future mode of price (and quantity) collection for
some areas of expenditure. This data provides the potential to produce higher-quality
estimates of price change.
See the ‘Retail transaction data’ chapter for further discussion on issues and
implementation.
3.5 Basket coverage
Factors to consider when selecting items to include in the CPI basket of goods and
services are that they:
• collectively comprise a representative sample of goods and services purchased by
consumers
• have price movements that reflect those of broad groupings of similar products
• have a high probability of being available in the future – this reduces the difficulties
should items have to be replaced.
The basket is reviewed at each three-yearly CPI review, using information from the HES
and other sources. There are currently 710 items in the basket.
3.6 Regional pricing centres
3.6.1 Number of regional pricing centres
Prices are collected for about half the CPI basket (by weight) by Statistics NZ price
collectors personally visiting over 3,000 different shops in 15 main urban centres
throughout the country. The types of outlets visited include, for example, supermarkets,
department stores, and appliance stores. The current 15 regional pricing centres, for
items priced in the field, are shown in table 3.1.
77
Sampling framework
Table 3.1
Regional pricing centres for CPI
Region / pricing centre
Base
population
weight
Proportion of
Proportion of
weekly price
monthly price
quotes
quotes
June 2011 quarter (percent)
Proportion of
quarterly price
quotes
Auckland
33.4
13.4
13.2
18.4
Wellington
11.1
8.1
7.4
13.0
Rest of North Island
Whangarei
Hamilton
Tauranga
Rotorua
Napier-Hastings
New Plymouth
Wanganui
Palmerston North
31.7
3.6
9.4
4.5
1.8
4.6
2.5
1.5
3.8
47.3
5.7
7.2
5.8
6.0
6.0
5.7
6.7
4.2
45.6
5.5
6.3
5.6
5.5
5.6
5.4
6.5
5.4
35.7
3.8
6.9
4.9
3.9
4.8
3.8
3.8
3.9
Canterbury
Christchurch
Timaru
13.0
11.6
1.4
12.8
8.9
3.9
13.7
8.3
5.4
16.5
12.6
3.8
Rest of South Island
Nelson
Dunedin
Invercargill
10.8
3.9
4.8
2.2
18.4
6.5
7.0
5.0
20.1
6.6
7.2
6.3
16.5
4.8
7.0
4.7
Total
100.0
100.0
100.0
100.0
The sample of pricing centres is determined by regional data requirements, coverage of
the national population and, to a lesser extent, consistency with historical practice. The
number of regional pricing centres has reduced over time as follows:
• 1965 – increased to 25 centres (up from 21)
• 1988 – reduced to 20 centres
• 1991 – reduced to 15 centres.
Recommendation 15 of the 2004 CPI Revision Advisory Committee asked that Statistics
NZ should consult with users in reviewing the number of regional centres from which
prices are collected. Following public consultation in December 2005/January 2006,
based on an issues and options paper entitled Review of CPI regions, Statistics NZ
decided to continue to collect prices from the existing 15 regional centres, but to
reallocate resources to align more closely with the population shares of the regions. This
resulted in more prices being collected from the larger pricing centres, particularly
Auckland. The regional population weight for Auckland increased from 27.5 percent in
1988 to 33.4 percent in 2011.
Analysis undertaken for the Review of CPI regions (2005) showed that excluding the
three smallest regions (Timaru, Wanganui, and Rotorua) would have a negligible impact
on the national all groups CPI. The difference in the quarterly change for the national CPI
less the three smallest regions, compared with the published national CPI, was found to
be mostly less than or equal to 0.01 of a percentage point over the period from the June
1999 quarter to the March 2005 quarter. The same study found that excluding the eight
smallest regions affected the quarterly changes by up to about 0.04 of a percentage
point.
78
Sampling framework
Similarly, research for the 1997 CPI Revision Advisory Committee found that excluding
the four smallest regional pricing centres, or more generally price quotes with small
weights (the bottom 16 percent of cumulative weights), from the calculation of the CPI
had a negligible impact on the national all groups CPI, based on a two-year period
(December 1993 to December 1995 quarters).
These studies do not describe the effect on disaggregated indexes (eg the group level
and below), nor that on regional CPIs. There is also a risk that successively reducing the
number of pricing centres might miss the cumulative effect of reducing regional pricing
centres over a longer time period (ie from 25 centres before 1988).
3.6.2 Population coverage
Two aspects of the sampling framework that the 2013 CPI Advisory Committee may wish
to consider are the alignment of the regional pricing centres with regional council
boundaries, and how well the pricing centres cover different ethnic groups.
79
Sampling framework
Figure 3.2
Current and alternati ve r egional pricing c entres
Current and alternative regional pricing centres
80
Sampling framework
These regional council areas do not currently have a CPI regional pricing centre:
• Gisborne
• Marlborough
• West Coast.
Prices are collected in Richmond (Tasman regional council) as part of the Nelson pricing
centre. For weighting purposes, Gisborne’s population is allocated to the Napier-Hastings
pricing centre. Marlborough, and the West Coast populations are allocated to the Nelson
pricing centre.
The representation of price change for households in these regional council areas could
be enhanced by including a regional pricing centre in one or more of these regions. Given
resource constraints, adding additional pricing centres may not be an efficient use of
collection costs. A more attractive option may be to align the location of the current 15
pricing centres to more-closely cover regional council areas. The three areas that do not
currently have a regional pricing centre all have relatively small populations. As shown in
table 3.2, the distribution of ethnic groups varies across regions.
The grouping of regional council areas for CPI weights is consistent with the published
regional breakdown in the Household Labour Force Survey. Labour force statistics are
published at the regional council area level – the Gisborne and Hawke’s Bay regions are
combined as are the Tasman, Nelson, Marlborough, and West Coast regions. The
Regional GDP feasibility study investigated producing estimates of GDP at the regional
council area level, with the Tasman and Nelson regions combined. Future work on
regional GDP is likely to use the same regional breakdown. The Retail Trade Survey
publishes retail sales for six geographic regions (Auckland, Waikato, Wellington, Rest of
North Island, Canterbury, Rest of South Island).
An option that the committee may wish to consider is to reallocate the CPI regional
pricing centres as follows:
• drop Wanganui and Timaru
• add Gisborne and Westport-Greymouth.
Westport-Greymouth could be treated as one regional pricing centre (Napier-Hastings is
currently one region for price collection). Timaru and Wanganui are the two smallest
pricing centres (their current population weights are 1.4 percent and 1.5 percent,
respectively). Canterbury would retain Christchurch as a regional pricing centre and
Manawatu-Wanganui would retain Palmerston North. Including Gisborne as a regional
pricing centre would increase the proportion of prices directly representing Māori
households.
81
Sampling framework
Table 3.2
Regional council areas by ethnic group
2006 Census usually resident population count
Regional
council area
Northland
Auckland
Waikato
Bay of Plenty
Gisborne
Hawke's Bay
Taranaki
ManawatuWanganui
Wellington
West Coast
Canterbury
Otago
Southland
Tasman
Nelson
Marlborough
Total
European
Ethnic group
Pacific
Māori
Asian
peoples
Other
Total
people
Regional pricing
centre
Whangarei
Auckland
Hamilton
Tauranga, Rotorua
[Napier-Hastings]
Napier-Hastings
New Plymouth
Wanganui,
Palmerston North
Wellington
[Nelson]
Christchurch, Timaru
Dunedin
Invercargill
Nelson
Nelson
[Nelson]
4
27
10
6
1
4
3
8
24
14
12
3
6
3
Percent
1
67
4
2
0
2
1
1
66
5
2
0
1
1
3
25
10
7
1
4
3
4
32
9
6
1
4
3
6
7
2
2
6
6
12
1
15
6
3
1
1
1
100
10
1
6
2
2
1
1
1
100
13
0
4
1
1
0
0
0
100
10
0
8
2
0
0
0
0
100
11
1
16
6
3
1
1
1
100
11
1
13
5
2
1
1
1
100
About 97 percent of the 2006 Census usually resident population is within regional
council areas with a regional pricing centre. Table 3.3 shows population coverage, by
ethnic group, of areas with a regional pricing centre.
Table 3.3
Population covered by regional council areas that have a regional pricing centre
By ethnic group
Current coverage
Coverage with change in
(1)
pricing centres
European
97.0
98.8
Māori
95.2
99.2
Pacific peoples
99.2
99.8
Asian
99.5
99.8
Other
96.6
98.5
All households
97.1
98.9
Ethnic group
1. Gisborne and Westport-Greymouth instead of Wanganui and Timaru.
At the territorial authority level, about 73 percent of the 2006 Census usually resident
population is within a territorial authority with a regional pricing centre. Under the option
above, this would decrease slightly to 72 percent coverage of all households. Coverage
for the Māori ethnic group would increase from about 64 percent now to about 65
percent. See Appendix 3b for coverage of other ethnic groups.
82
Sampling framework
About 78 percent of total retail sales in 2012 were in territorial authorities with a regional
pricing centre. See Appendix 3c for coverage of retail sales by industry.
3.6.3 Price collection in large cities
Retail transaction data is a potential future mode for price collection, which may increase
the coverage of outlets within large cities (eg Auckland, Wellington, Christchurch). See
the ‘Retail transaction data’ chapter for discussion on current research in this area.
There is potential to investigate the feasibility to increase the proportion of price quotes
collected in Auckland, to more closely align price quote proportions with the population
shares of the regions. This would increase the quality of the CPI for Auckland, and
potentially the national CPI. However, without additional data collection this would be at
the expense of the quality of other regional CPIs.
Given the scale of price collection in Auckland, the city is treated as two pricing centres to
allocate price collector workload. For practical reasons, it is split roughly north-south.
There is the potential to explore expanding the stratification of Auckland to further
increase geospatial coverage within the Auckland Council area.
Further stratification of Auckland would also provide an opportunity to consider
subpopulation coverage, such as ethnic groups. Table 3.4 shows the distribution of
Auckland’s population, by ethnicity, in the 2006 Census. There are notable differences in
the distribution of ethnic groups across Auckland.
83
Sampling framework
Table 3.4
Auckland local board areas, by ethnic group
2006 Census usually resident population count
Local board area
European
Albert-Eden
Devonport-Takapuna
Great Barrier
Henderson-Massey
Hibiscus and Bays
Kaipatiki
Puketapapa
Rodney
Upper Harbour
Waiheke
Waitakere Ranges
Waitemata
Whau
Franklin
Mangere-Otahuhu
Manurewa
Maungakiekie-Tamaki
Orakei
Otara-Papatoetoe
Papakura
Howick
Total
Māori
7
5
0
7
9
7
3
5
4
1
5
5
4
6
2
4
4
7
2
3
9
100
4
2
0
11
3
5
2
3
2
1
3
3
4
5
8
14
6
2
9
8
4
100
Pacific
peoples
Percent
4
1
0
10
1
3
4
1
1
0
2
2
7
1
21
11
10
1
17
2
2
100
Asian
10
4
0
7
3
7
8
0
4
0
1
5
9
1
4
5
5
5
6
1
15
100
Total
people
7
4
0
8
6
6
4
4
3
1
3
5
5
4
5
6
5
6
6
3
9
100
Changing the design criteria to include additional assessment of subpopulation coverage
(eg ethnic groups) could be beneficial for two reasons:
• To ensure that price collection for the national CPI is a representative average of all
households – including enhancing the public perception of coverage for specific
subpopulations.
• To increase the quality of measures of consumer price change for subpopulations
(if developed).
In some cases, improvements to coverage of where certain subpopulations shop (or the
range of brands and pack sizes) may oppose the efficiency of the optimal sample design
for the ‘all households’ national CPI.
3.7 Outlet coverage
Prices are collected from outlets by different modes. For about 53 percent of the overall
basket weight, prices are collected in the field, by local price collectors visiting retail
outlets. Prices of items with about 43 percent of the basket weight are collected through
postal surveys. The remaining 4 percent of prices are collected in other ways, such as the
Internet, or brochures. The samples for postal surveys and other sources are reviewed on
a rolling basis and determined using both staff judgement and probability sampling
techniques.
For the current field outlet sample, two elements were considered. The first was storetype
selection, where information from the HES was used to determine the representative
types of outlets at which consumers buy (eg supermarkets, department stores, specialist
retailers).
84
Sampling framework
The second element was the regional allocation of field outlets. The 15 pricing regions
were divided into five groups, based on the size of the population served by each urban
area. The groupings were:
• Auckland
• Wellington, Christchurch
• Hamilton, Tauranga, Napier/Hastings, Dunedin
• Whangarei, Palmerston North, Nelson, Invercargill
• Rotorua, New Plymouth, Wanganui, Timaru.
For each group, a specific number of outlets was recommended to be priced – in each
region for particular groups of items. The number was based on judgements about how
many outlets were required to represent the population of outlets, given resource
constraints and outlet population sizes. Price collectors then selected representative
outlets based on their local knowledge, with, in some cases, guidance from head office.
Outlet sampling is a major component of the rolling field outlet review. Outlet samples are
refreshed to ensure the outlets visited reflect where consumers shop. The review
analyses the coverage of retailers and the mix of storetypes for each item. Information
from the HES, the Annual Enterprise Survey, Statistics NZ's register of businesses (the
Business Frame), and other sources, including retail transaction data, is used.
If an existing outlet closes, or is no longer considered representative by the local price
collectors, then a replacement outlet of a similar type and in the same area is chosen
based on price collectors’ local knowledge. New outlets deemed to be significant players
are added to the sample when they enter the local market.
3.8 Product coverage
Product selection is based on a combination of price collectors’ knowledge and
observation, feedback from outlets, and retail transaction data. The aim is to collect
prices for a representative range of products across a range of outlets. In practice, one
representative product per item is priced per outlet (eg one brand, size, and type of
instant coffee is priced in each outlet). Summary retail transaction data is used for
supermarket items and consumer electronics to ensure that representative ranges of
brands and pack sizes are priced for each item.
3.9 Temporal coverage
The frequency of data collection is determined by the publication frequency and the
variability of price change over time. Redistribution of price collection across months
and/or additional monthly data collection could be required to publish the CPI on a
monthly basis.
See the ‘Frequency of the CPI’ chapter for a full discussion of the work involved in
producing a monthly CPI.
3.10 Sampling trade-offs
Given resource constraints, decisions have to be made on the:
• size and composition of the CPI basket of goods and services (items)
• geographic coverage of the index
• outlet selection
85
Sampling framework
• product selection
• when and how often price (and quantity) data is collected.
Judgements are required to balance sampling trade-offs, such as whether there should
be more items in the CPI basket at the expense of fewer outlets, or more products
sampled at the expense of fewer items.
An important decision is to weigh the relative importance of the national CPI against the
quality of regional CPIs. If prices were collected from fewer regional centres then
resource could be available to boost the coverage of the prices sampled within the
remaining regions. Another option would be to collect prices for only a subset of the CPI
basket in smaller regional pricing centres. This may mean not collecting prices for CPI
items with relatively small expenditure weights in some regions.
Relaxing regional requirements could allow resources to be directed towards the data
requirements of a monthly CPI. Expanded monthly price collection is one of several
factors that might be necessary to publish a monthly CPI.
See the ‘Frequency of the CPI’ chapter for details of other requirements for a monthly
CPI.
Statistics NZ has begun work on a tool to help determine how best to allocate the CPI
sample. The tool uses the Neyman optimal sample allocation formula. This formula
allocates a fixed sample size to strata in a way that maximises survey precision, given
known information about strata population sizes, variance, and costs of data collection. It
has been adapted to be suitable for use in the CPI context, using the CPI expenditure
weights and the variability of price change. The tool will provide additional data to help
inform the size and composition of the CPI basket, and to assist in efficient outlet
sampling.
Quality requirements for regional CPIs can be added or relaxed depending on the
balance of user requirements. The allocation formula can be run to provide the most
efficient sample allocation for the all groups, national CPI. Alternatively, additional
constraints can be added to specify precision requirements for indexes disaggregated by
region or expenditure classification.
3.11 Regional weighting
New Zealand’s CPI is constructed using national expenditure weights, which assumes
the population in each region purchases the same basket of goods and services. For
example, the same expenditure weight is given to electricity in the regional indexes of
Whangarei and Invercargill. The exceptions to this are council refuse bags, reticulated
gas, suburban rail services, and child (0–5 years) GP fees, which are all charged
differently or not available in all regions.
An alternative approach is regional expenditure weighting, which reflects any regional
differences in the mix of goods and services purchased by households and any regional
differences in spending per person. Australia, the United Kingdom, the United States, and
Canada have adopted the regional expenditure weighting approach.
Recommendation 14 from the 2004 CPI Revision Advisory Committee stated that
Statistics NZ should explore the possibility of moving from national expenditure weights to
using regional expenditure weights for a number of broad regions, if the use of regional
weights would improve the accuracy of the national CPI. Using regional weights would
also support the derivation of ‘fit-for-purpose’ indexes for regions (Auckland, Wellington,
the rest of the North Island, Canterbury, and the rest of the South Island).
According to the ILO’s Consumer price index manual, theory and practice, regional
weights are not universally used in CPIs. The choice depends on factors such as the size
86
Sampling framework
and structure of the country, data and resource availability, and the purposes of the
index. However, the manual does indicate a preference for using regional expenditure
weights over population weights, "In the absence of any [regional] expenditure statistics,
population statistics might be used as the basis for regional weights".
In 2005, Statistics NZ investigated the possibility of adopting regional expenditure weights
and reported on the investigation in a public consultation paper, Review of consumers
price index regions (2005). The outcome of the review was reported in Outcome of the
review of consumers price index regions (2006). In principle, Statistics NZ is in favour of
adopting a regional expenditure weighting approach. Analysis of historical CPI data
suggests this approach would have a negligible effect on the accuracy of the CPI.
Using regional expenditure weighting would eliminate the potential bias in the current
index (due to the use of national expenditure weights) and would improve the consistency
of the CPI weights. Regional expenditure weights would improve the reliability of regional
estimates in the CPI – if regional estimates of expenditure can be calculated with
sufficient quality. The current method of using national expenditure weights in each
region assumes that expenditure is distributed identically throughout New Zealand. This
assumption needs to be balanced against the quality of regional expenditure weights
derived from the HES, which has higher sampling variation at the regional level. One
measure of the quality of regional expenditure estimates is to look at the consistency of
the regional differences over successive surveys.
87
Sampling framework
Figure 3.3
Regional expenditure weights
2008 and 2011
100
Percent
7.1
6.8
7.0
9.5
9.1
8.8
90
6.3
8.2
6.9
7.6
10.8
9.4
7.4
6.3
9.6
9.7
7.4
9.4
8.9
9.5
6.8
6.4
10.0
9.4
70
60
5.1
5.3
5.4
4.4
5.4
4.9
15.7
6.0
4.1
14.7 14.3 16.5 15.6 15.2
5.2
4.6
5.2
4.6
5.0
5.5
5.6
4.7
13.9
16.6 14.1
40
23.6
23.2
26.0
23.8 23.9
23.5
Communication
Transport
4.6
5.1
4.9
4.9
4.5
5.9
5.1
4.6
Health
Household contents & services
50
22.7
Education
Recreation & culture
80
16.2 15.1 16.7
Misc. goods & services
23.0 20.5 20.3 20.9
21.4
Housing & household utilities
Clothing & footwear
30
5.7
5.2
4.4
8.0
8.1
7.8
5.1
4.5
4.4
20
6.8
6.9
10
17.8 18.8 17.9 19.0 17.8 17.5 17.4 19.2 18.1 18.2 18.5 19.6
0
4.6
4.0
3.9
4.9
6.1
6.0
7.2
7.1
4.1
6.3
3.9
6.7
Alcoholic beverages & tobacco
8.6
Food
2008 2011 2008 2011 2008 2011 2008 2011 2008 2011 2008 2011
New
Zealand
Auckland
Wellington
Rest of
North
Island
Canterbury
Rest of
South
Island
Region
Source: Statistics New Zealand
Figure 3.3 illustrates the difference between estimates of CPI weights for each of the five
broad regions and aggregated for all regions – using regional expenditure weighting
compared with the currently published series using national expenditure shares
apportioned to regions using population shares.
88
Sampling framework
Figure 3.4
CPI using national and regional expenditure weights
June 2008–September 2012
1140
Index
National
weights
Akld
1120
Regional
weights
Akld
1100
Wgtn
Wgtn
1080
RNI
RNI
Cant
Cant
RSI
RSI
NZ
NZ
1060
1040
1020
1000
J
08
S
D
M
09
J
S
D
M
10
J
S
D
M
11
J
S
D
M
12
J
S
Code
Akld - Auckland
Wgtn - Wellington
RNI - Rest of North Island
Cant - Canterbury
RSI - Rest of South Island
NZ - New Zealand
Source: Statistics New Zealand
Over the period from the June 2008 quarter to the September 2012 quarter, the New
Zealand CPI tracked almost identically when comparing national basket weights in each
region with using region-specific expenditure weights. There are minor differences in
some of the regional indexes. Using regional expenditure weights, the Auckland CPI
increased slightly less than the Auckland CPI using national expenditure weights. In
contrast, using regional expenditure weights results in the CPIs for Canterbury and the
Rest of the South Island increasing slightly more than the respective indexes calculated
using national expenditure weights.
3.12 Regional-spatial indexes
Regional-spatial price indexes compare differences in price levels across regions, at a
given point in time. These indexes can be useful in comparing living costs between
regions.
Regional-spatial price indexes require the same products to be priced in each region to
be compared, or have sophisticated methods that control for the differences in quality
characteristics. Since the current sample of price quotes is designed to measure price
change over time, the products priced are not necessarily comparable across regions.
The 2004 CPI Revision Advisory Committee recommended that Statistics NZ should seek
to develop regional-spatial indexes. Statistics NZ commissioned Daniel Melser and
Robert Hill of the University of New South Wales to develop a suitable methodology for
constructing regional-spatial cost of living indexes and to document findings in a research
report. In Developing a methodology for constructing spatial cost of living indexes, the
authors outlined possible uses of regional-spatial cost of living indexes, explored possible
conceptual frameworks, and discussed possible methodological approaches. They
concluded that a conditional cost of living index constructed under a use framework
would appear to be most appropriate.
Statistics NZ consulted widely with possible users of temporal and/or regional-spatial cost
of living indexes. The possible development of these measures received some support.
Resources have not been made available for developing and producing these measures.
In recent years there have been limited enquiries about regional differences in the cost of
living.
89
Sampling framework
3.13 Modes of data collection
Prices used in the CPI are collected by four main methods:
• visiting retail outlets
• postal surveys
• online price collection from the Internet
• administrative data.
Statistics NZ price collectors personally visit over 3,000 different outlets in 15 main
centres throughout the country. The types of outlets visited include supermarkets,
department stores, and appliance stores.
Price collection in the field is currently conducted by Statistics NZ staff who record prices
using pen and paper. Over the past two decades, it has become standard practice for
most national statistics offices to use electronic handheld devices to collect prices from
retail outlets. Using these devices for CPI price collection has proven internationally to be
efficient, effective, and reliable.
Statistics NZ has a project in place to replace pen and paper with handheld devices in
2014. This is expected to improve the efficiency of collection and processing activities,
and enable Statistics NZ to undertake data quality checks at the point of collection.
Postal surveys sent out each month, quarter, or year are used primarily to collect prices
for services, such as rent, construction, and education. The surveys go directly to service
providers. The survey population and sampling frames used for some of these surveys
cover the whole of New Zealand (not just the 15 urban centres used for personal visits).
These surveys are designed to produce fit-for-purpose estimates at the national level and
for the published five broad regions (Auckland, Wellington, Rest of North Island,
Canterbury, Rest of South Island).
Prices for some products and services (eg digital downloads, package holidays, and
international air fares) are collected each month or quarter from the Internet. About 3,000
of the 120,000 price quotes collected each quarter are from online retailers. Prices are
collected from websites regardless of where the retailers are located (New Zealand or
overseas). Online price collection is being increasingly used. Collecting prices online can
be used:
• to reflect where consumers shop
• as a more-efficient mode of price collection, potentially.
Increasing online outlet coverage has been an important part of the rolling field outlet
review – for example, prices are now collected from online retailers for books, CDs, and
magazines. A feasibility project will investigate the potential to use an automated data
scraping tool to aid efficient collection of prices online.
Administrative data is used to track price change for Housing New Zealand Corporation
rents. Retail transaction data is a potential future mode for price collection.
See the ‘Retail transaction data’ chapter for discussion on current research in this area.
90
Sampling framework
3.14 Options for sampling framework
Table 3.5
Options and implications for the sampling framework
Option
Implications
Reallocate regional price collection
•
improve alignment of
regional pricing centres with
regional council boundaries
and/or subpopulations
better align proportion of
price quotes with regional
population shares
•
Reduce regional price collection to
reallocate resources towards other
areas of CPI measurement – priority
of regional indexes against:
•
•
•
•
•
consumer price change for
subpopulations
monthly CPI
Regional expenditure weights
•
•
precision vs bias trade-offs
priority of these
•
•
•
•
•
•
Regional-spatial indexes
•
priority of these
•
•
91
Cost implications of changing pricing centres
(and sample changes within pricing centres).
Alignment with regional council boundaries
and/or subpopulations depends on user
perceptions and requirements.
Closer alignment with regional population
shares could improve efficiency of price
collection and quality of national CPI, at the
expense of a potential quality reduction for
some regional indexes.
Reduced regional coverage may affect
perceptions of credibility of CPI.
Risk that successively reducing the number of
pricing centres might miss the cumulative
impact (ie from 25 centres prior to 1988).
Improved range or frequency of indexes, at
the expense of a quality reduction for some
regional indexes
Cost implications of making changes,
including potential additional analysis required
to quality assure regional expenditure weights
(at each three-yearly weight update).
Using regional expenditure weights would
eliminate the potential bias in the current index
due to the use of national expenditure weights.
Analysis of historical CPI data suggests
negligible impact on the accuracy of the
national CPI.
Accuracy of regional CPIs would increase
using regional expenditure weights, but would
be less precise due to higher sampling errors
for expenditure weights (from the HES).
Cost implications of developing and compiling
regional-spatial indexes. Limited number of
enquiries in recent years.
Improved information on differences in prices
across regions
Sampling framework
Appendix 3a: Household Economic Survey – regional
sample sizes
Appendix table 1
Household Economic Survey achieved sample sizes
By CPI region
CPI region
Auckland
Wellington
Rest of North Island
Canterbury
Rest of South Island
All households
Sample size (households)
(1)
2006/07
2009/10
597
762
382
511
690
816
463
547
418
490
2,550
3,126
1. Imputation methods were introduced for the 2009/10 HES. Imputation replaces missing values with
actual values from similar respondents. The effect of introducing imputation was to increase the number
of usable households, and so increase the achieved sample size.
92
Sampling framework
Appendix 3b: Population covered by territorial
authorities with pricing centres
Appendix table 2
Population covered by territorial authorities that have CPI regional pricing
centres(1)
By ethnic group
Current coverage
Coverage with change in
(2)
pricing centres
European
70.2
68.9
Māori
63.5
65.0
Pacific peoples
92.4
92.4
Asian
94.7
94.5
Other
70.0
68.7
All households
72.8
72.0
Ethnic group
1.
2.
Includes seven territorial authorities that are now part of Auckland city, and Tasman territorial
authority as some prices for Nelson are collected in Richmond.
Gisborne and Westport-Greymouth instead of Wanganui and Timaru.
93
Sampling framework
Appendix 3c: Retail sales coverage by territorial
authority areas
Appendix table 3
Retail sales covered by territorial authorities that have CPI regional pricing
centres(1)
By industry
Current
coverage
Industry
Coverage with
change in
pricing
(2)
centres
Percent
Motor vehicle and parts
Fuel
Supermarket and grocery stores
Specialised food
Liquor
Furniture, floor coverings, houseware,
textiles
Electrical and electronic goods
Hardware, building, and garden supplies
Recreational goods
Clothing, footwear, and accessories
Department stores
Pharmaceutical and other store-based
retailing
Food and beverage services
Total
1.
2.
79
67
63
81
61
80
65
65
82
62
84
88
75
80
90
79
85
87
76
79
89
78
81
81
78
80
80
78
Includes seven territorial authorities that are now part of Auckland city, and Tasman territorial
authority as some prices for Nelson are collected in Richmond.
Gisborne and Westport-Greymouth instead of Wanganui and Timaru.
94
4 Frequency of weight updates
4.1 Executive summary
This chapter considers whether it is appropriate to reweight certain parts of the
consumers price index (CPI) on a more frequent basis, using either administrative data or
retail transaction data.
New Zealand’s CPI is reweighted once every three years, on average. This reweighting
frequency is well within the International Labour Organization’s (ILO) recommendation of
at least once every five years. Regular CPI reweights help to maintain the relevance of
the index by reducing the effect of households substituting towards goods and services
showing low relative price change, known as commodity substitution.
Commodity substitution can cause the CPI to overstate price change. The more
frequently the CPI is reweighted the less effect commodity substitution will have on the
index. It is estimated that the cumulative cost to government of not reweighting the CPI
regularly could quite quickly run to hundreds of millions of dollars.
4.1.1 Why reweighting is three-yearly
The choice to reweight every three years is for practical reasons, balancing fit-forpurpose quality requirements, resource constraints, and respondent load. More frequent
reweights would be expensive. This is due to the time and resources required to collect
comprehensive information on household spending, and turn this information into CPI
expenditure weights, and the relatively high respondent load for people completing the
Household Economic Survey (HES).
Comprehensive information on household spending patterns is only available every three
years, when HES results become available. The HES expenditure module runs every
three years to coincide with CPI reviews, as it is the main source of CPI expenditure
weights. However, for some goods and services, information from other sources is
available more frequently. Administrative data sources provide information for some
goods and services, and retail transaction data is potentially available for supermarket
groceries and consumer electronics.
Currently, CPI expenditure weights (or rather, the underlying quantities) are held fixed
between reweights, down to the 108 categories at the class level of the classification.
Quantity shares below this level are monitored. They may be adjusted where necessary
to reflect volume-related shifts in the relative importance of goods or services within the
108 expenditure classes between three-yearly reweights. In practice, this has been done
only occasionally.
4.1.2 Options for reweighting
Retail transaction data and administrative data could be used to update some CPI
weights on a more frequent basis. This could improve the relevance of the index,
particularly for higher-weighted areas such as audio-visual equipment, and cigarettes and
tobacco, where low/high relative price changes are associated with high/low relative
quantity change.
Retail transaction data is available for supermarket groceries and consumer electronics.
This data potentially allows superlative index formulae, such as the Fisher or Törnqvist
index, to be used. A superlative index is one that utilises both current- and referenceperiod quantity shares symmetrically, which results in substitution between products
being accounted for in the index appropriately.
This is discussed in further detail in the ‘Retail transaction data’ chapter.
95
Frequency of weight updates
Weights for some other goods and services could be updated on an annual basis. This
could be done for items where administrative data is available. This would be an explicit
weight update, rather than occurring implicitly when the price and quantity data enter the
index formula itself.
Updates to CPI weights could occur within expenditure classes (consistent with current
practice) or they could happen at higher levels of the index.
Updating some, but not other, CPI weights on a more frequent basis could make the
weight reference period of the CPI basket less coherent. By updating part of the CPI
basket, the weight reference period for items would be less uniform across the basket. To
a certain extent this is already the case. About 5 percent of the CPI basket (by weight) is
‘volume adjusted’ during CPI reviews, to reflect significant changes in quantities between
the weight reference period and price reference period, as there is a lag in calculating
new expenditure weights. Similarly, a further 15 percent of the CPI basket (by weight)
uses a weight reference period that is an average of three years of expenditure to
partially smooth the impact of cyclical highs or lows in activity. This means the underlying
quantity weights for some CPI items are from a different period of time than for most
other items.
A framework would be developed for updating part of the CPI basket more frequently.
The framework could be available to users, to inform them of the reasons for updating
weights, and the principles and process for carrying out the updates. This would make
the process more transparent and objective.
4.2 Issues to consider
The frequency of weight updates in the CPI
• Is a three-yearly HES and CPI reweighting cycle still appropriate? If not, does the
frequency need to be increased, or decreased?
The use of administrative data in the CPI for updating weights between CPI reviews
• Should Statistics NZ use alternative data sources to update expenditure weights
between CPI reweights, for areas where data is available to do this? If so, at what
level of the index should this be done?
4.3 Purpose of this chapter
This chapter considers whether it is appropriate to reweight certain parts of the CPI on a
more frequent basis, using either administrative data or retail transaction data. There are
no specific recommendations related to this topic from the 2004 CPI Revision Advisory
Committee, but three closely related recommendations are:
Recommendation 6: Statistics New Zealand should continue with the current
three-yearly re-weighting cycle. At this stage, there is no clear evidence that the
benefits of more frequent re-weighting justify the additional cost.
Recommendation 8: Within its current resources, and subject to
Recommendations 1 and 3, Statistics New Zealand should continue to update its
index methodologies to improve the quality of the CPI, taking into account changes
in the economy, changing user requirements, and changes in international
standards of good practice.
Recommendation 10: At each reweighting of the CPI basket, Statistics New
Zealand should calculate a superlative index on a retrospective basis to provide
information on the effect of item substitution on the fixed-weight CPI. Consistent
96
Frequency of weight updates
with recommendation 8, Statistics New Zealand should also assess the value of
providing users with real-time estimates of the effect of item substitution on the CPI.
4.4 Background – why weights are updated
CPI weights are updated regularly to help maintain the CPI’s relevance. One main factor
that can influence the CPI’s relevance is commodity substitution. Under normal economic
conditions, households tend to react to changes in relative prices by reducing purchases
of goods and services that show higher relative price change, and instead buying more of
those with lower relative price change. For example, if apple prices increased a lot, but
pear prices only a little, households might buy more pears and fewer apples than before.
Continuing to price the same quantities of apples and pears would over-weight price
change for apples and under-weight price change for pears. This would overstate the
actual price change faced by households. Commodity substitution occurs when current
household spending patterns begin to differ from those used to calculate the CPI.
For practical reasons, the quantities used to weight CPI price movements represent
household acquisitions from a historical period, the ‘weight reference period’. These
quantities remain fixed between reweights, because comprehensive information on
household spending is collected through the HES only every three years. Commodity
substitution is known to occur in the years between reweights.
Commodity substitution can cause the CPI to overstate price change. This is known as
‘substitution bias’. Statistics NZ produces a retrospective superlative index to indicate the
impact of commodity substitution, and changes to the data sources and methods used to
compile expenditure weights. This analytical time series uses a superlative index formula
– the Fisher index formula, which is compared with a Laspeyres-type index that is
consistent with the all groups CPI. The Fisher index uses weights from the latest review
period, and the previous review period. Statistics NZ has produced a retrospective
superlative index following each three-yearly CPI review since the 2006 CPI Review.
From the June 2002 quarter to the June 2011 quarter, the Laspeyres-type index rose an
average of 2.7 percent a year, compared with 2.6 percent for the Fisher index, a
difference of 0.1 of a percentage point a year. This result is broadly consistent with, and
at the lower end of, international studies. Figure 4.1 shows how the Fisher index tracks
compared with a Laspeyres-type index consistent with the all groups CPI.
97
Frequency of weight updates
Figure 4.1
Analytical CPI indexes
(1)(2)
Quarterly
Base: June 2002 quarter (=1000)
1300
Index
Laspeyres
1250
Fisher
1200
1150
1100
1050
1000
J S D M J S D M J S D M J S D M J S D M J S D M J S D M J S D M J S D M J
02
03
04
05
06
07
08
09
10
11
1. Alternative housing weights were used from the June 2002 to June 2006 quarters.
2. Prices are not seasonally adjusted.
Source: Statistics New Zealand
The 2011 superlative index paper also included a new analysis of how the CPI might
have tracked had it not been reweighted. The analysis found that without reweighting in
2006 nor 2008, the CPI’s average annual increase would have been 3.0 percent between
the June 2002 quarter and June 2011 quarter, compared with 2.7 percent for the
reweighted CPI. This demonstrates the value of regular CPI weight updates. Figure 4.2
shows how the CPI might have tracked had it not been reweighted in 2006 nor 2008.
Figure 4.2
Analytical CPI indexes (1)(2)
Quarterly
Base: June 2002 quarter (=1000)
1300
Index
Reweight in 2006 and 2008
No reweight in 2006 nor 2008
1250
1200
1150
1100
1050
1000
J S D M J S D M J S D M J S D M J S D M J S D M J S D M J S D M J S D M J
02
03
04
05
06
07
08
09
10
11
1. Alternative housing weights were used from the June 2002 to June 2006 quarters.
2. Prices are not seasonally adjusted.
Source: Statistics New Zealand
98
Frequency of weight updates
See Analytical retrospective superlative index based on New Zealand’s CPI: 2011 for
more details.
Another impact of reweighting the basket less frequently is the potential effect of the
CPI's use in indexing Government transfer payments, social assistance thresholds, and
excise duties. For every 0.1 of a percentage point that the CPI overstates true annual
inflation, there is a potential adverse impact of about $20 million on central government
fiscal net transfers of $19.9 billion ($22 billion of spending less $2.1 billion of revenue).
This cost could quite quickly run in to the hundreds of millions of dollars. These figures
include government spending on New Zealand Superannuation, which can be further
indexed to ensure that the superannuation rate for a married couple is at least 66 percent
of average weekly after tax earnings. This means that some years the annual increase in
superannuation rates reflects more than just increases in the CPI, and any overstating by
the CPI is superseded by further adjustments to the superannuation rate to maintain the
average weekly earnings threshold.
More-frequent weight updates minimise the effect of commodity substitution within the
index but are more costly. This is due to the need to collect more frequent information on
household spending patterns through the HES, which is the main source of CPI weights
and carries a relatively high respondent load. However, administrative and retail
transaction data could be used to update parts of the CPI basket at a relatively low cost,
to help reduce the effect of commodity substitution on the CPI.
4.5 Current practice
The CPI is reweighted at all levels of the classification (ie down to the basket level) once
every three years, on average. A three-yearly reweighting frequency has been in place
since 1999, after a recommendation made by the 1997 CPI Revision Advisory
Committee. Before the 1999 CPI review, the CPI was reweighted in 1949, 1955, 1965,
1974, 1977, 1980, 1983, 1988, and 1993. The 1997 recommendation was confirmed by
the 2004 committee. The frequency of reweighting is well within the ILO recommendation
of at least once every five years.
A three-yearly reweighting frequency sits about in the middle of international practice. At
one end of the spectrum, the United Kingdom’s CPI and retail prices index are both
reweighted annually, and CPIs for Canada and the United States are reweighted every
two years. Towards the other end, Australia’s CPI is reweighted once every six years at
the expenditure class level, but there is a rolling review to maintain the relevance within
expenditure classes between the six-yearly reweights.
The choice to reweight every three years is largely for practical reasons, as more
frequent reweights would be expensive. This is due to the time and resources required to
collect comprehensive information on household spending and turn this information in to
CPI expenditure weights, and the high respondent load for those completing the HES, the
main source of CPI expenditure weights.
Between reweights, the effective expenditure share of each individual basket item, which
is used to weight period-on-period movements, is implicitly updated each quarter (or
month, for the FPI) as prices change. Under the Laspeyres-type index used to calculate
the CPI, each item in the index is assigned an expenditure share – the expenditure on a
particular item (ie the quantity purchased in the weight reference period multiplied by the
prices of the price reference period) expressed as a percentage of total household
spending. As prices change from period to period, the price component of each
expenditure share is updated for each item – to reflect the change in prices. The quantity
component remains fixed, and is updated at each CPI review, although weights are
calculated as a total expenditure rather than as explicit quantities and prices. An increase
in price for a particular item, relative to other items, increases the effective expenditure
share of that item. A decrease in price for a particular item, relative to other items,
decreases the effective expenditure share of that item. Table 4.1 illustrates how the
99
Frequency of weight updates
effective expenditure shares of the groups have changed since the CPI was last
reweighted in the June 2011 quarter.
Table 4.1
Consumers price index – effective expenditure share, group level
Effective expenditure
Price change
share (percent)
Group
Jun 2011–Dec 2012
Jun 2011
Dec 2012
quarter (percent)
quarter
quarter
Food
18.79
18.40
-1.1
Alcoholic beverages and tobacco
6.91
7.26
6.1
Clothing and footwear
4.42
4.33
-1.0
Housing and household utilities
23.55
24.27
4.1
Household contents and services
4.44
4.31
-2.0
Health
5.44
5.61
4.1
Transport
15.12
15.01
0.3
Communication
3.53
3.04
-12.8
Recreation and culture
9.12
8.78
-2.8
Education
1.84
1.90
4.0
Miscellaneous goods and
services
6.85
7.10
4.7
All groups CPI
100.00
100.00
1.0
It is the effective expenditure share that is used to weight period-on-period price
movements. For example, price change for food from the December 2012 quarter to the
March 2013 quarter will be weighted at 18.40 percent, not 18.79 percent. Thus, an item
showing high relative price change will have an increased influence on the overall index,
while an item showing low relative price change will have a decreased influence.
Although the effective expenditure share of each expenditure category (eg group,
subgroup, class) changes along with price movements, the effective expenditure shares
for individual categories are not currently published. Instead, it is the expenditure share
that was calculated at the most-recent CPI review that is published. One option being
considered is to publish the updated effective expenditure shares in addition to those
calculated at the most-recent review. CPI weights are included in tables 8.01 and 8.02 of
the CPI information release, which show the influence price change for each group,
subgroup, and class has had on the all groups CPI.
Expenditure weights (or rather, the underlying quantities) are held fixed between
reweights, down to the third level of the classification (class level). Quantity shares below
this level are monitored. They may be adjusted where necessary to reflect volume-related
shifts in the relative importance of goods or services within the expenditure classes. In
practice this has been done only occasionally.
Outlet weights are also calculated for items with prices collected from different types of
outlets (eg food items collected from both supermarkets and convenience stores). This is
to ensure that price change at the types of outlets where households do much of their
spending (eg supermarkets) has more effect on the CPI than that at outlets where
households spend less.
Outlet weights, and the samples of retail outlets that prices are collected from for the
different parts of the CPI, are updated on a rolling basis as part of the CPI rolling review
of field outlets, which commenced following the 2011 CPI review. This review is carried
out in the years between full CPI reweights, to help maintain the CPI’s relevance.
100
Frequency of weight updates
4.6 Future possibilities
For some goods and services, data is available on the quantity and quality of the specific
products acquired in a certain period. This data could be used to update CPI weights on a
more-frequent basis than at the full, three-yearly CPI reweights, allowing regular weight
updates for some items tracked in the CPI between CPI reviews. In effect, this would be a
partial weight update for the CPI. It is estimated that up to half the CPI basket (by weight)
could be updated more frequently using this data. Examples of items this data is available
for are:
• supermarket groceries
• cigarettes and tobacco
• consumer electronics
• new cars and motorcycles.
More detailed retail transaction data could also be used to calculate price change in the
CPI – using up-to-date price and quantity information in real time. This chapter focuses
on using transaction data only for weighting purposes. Using retail transaction data to
measure price change is discussed in the ‘Retail transaction data’ chapter.
4.6.1 Updating the relative importance of items within an
expenditure class
Currently, expenditure weights (or rather, the underlying quantities) can be adjusted
below the class level of the classification. This is to reflect volume-related shifts in the
relative importance of goods or services within the expenditure classes, although this has
been done only occasionally.
Updating the relative importance of goods and services within expenditure classes could
be done more often. An example is new cars. Vehicle registration data could be used to
explicitly update the mix of car types within the ‘purchase of new motor cars’ class, to
reflect shifts between small, medium, and large cars. Such an update could be carried out
annually.
Similarly, if retail transaction data is used to measure price change for supermarket
groceries, superlative index formulae (eg Fisher or Törnqvist index) could be used to
implicitly update the relative importance of items within an expenditure class on a monthly
basis. For example, if the price of butter rose significantly and households shifted to
buying margarine, the superlative index formula could update the relative importance of
butter and margarine within the ‘oils and fats’ class to account for that shift.
While it may sound desirable to update quantity and quality information on a monthly
basis, this can cause ‘chain drift’ – a result of price and quantity ‘bouncing’ due to
discounting and seasonality. Chain drift is the bias that occurs when a chained index
diverges, or systematically ‘drifts’ away, from its direct (ie unchained) counterpart.
A chained index in which the return of prices and quantities to previous levels does not
correspond to the index also returning to the previous level, is exhibiting chain drift. The
effects of chain drift are mitigated if weights are not updated sub-annually, or if an
appropriate chaining methodology is used. Current research into using retail transaction
data for measuring price change aims to eliminate the problem of chain drift from monthly
chained superlative formulae.
For more details, see the ‘Retail transaction data’ chapter.
Another issue with updating the relative importance of goods and services within an
expenditure class is the completeness of the data used to make the update. If data exists
for some items within an expenditure class, but not for others, there is a risk that updated
weights may not reflect the true relative importance of all goods and services within that
101
Frequency of weight updates
expenditure class. At lower levels of the index (eg class level or below) this is less of a
problem as information tends to be more complete. For example, if supermarket
transaction data was available to update the relative importance of items within the oils
and fats expenditure class, it could be safely assumed the data would be comprehensive
enough to accurately update the relative expenditure weights within the class.
4.6.2 Updating weights at or above class level
Administrative data and transaction data could also be used to update expenditure
weights at or above the class level. This would be particularly useful for parts of the
basket where strong relative price changes cause significant commodity substitution.
4.6.2.1 Audio-visual equipment
The audio-visual equipment class could be updated more frequently. Audio-visual
equipment is subject to strong price decreases that decrease the effective expenditure
weight of the class. These price decreases are partly the result of quality adjustments,
which are made to remove the effect of quality changes in the products being priced from
period to period. Advances in technology can lead to rapid changes in the ‘characteristics’
(or features) of the audio-visual products households acquire.
Under a fixed-basket approach, the underlying quality and quantity of goods and services
acquired in the weight reference period are held fixed until the next reweight. A change in
the quality of goods and services purchased by households has a similar effect on a price
index to a change in the quantity of goods and services purchased. This is because
households effectively get ‘more for their money’ by shifting their purchases towards
higher quality goods over time, even if the overall number of items purchased does not
change.
The quality of an individual audio-visual equipment product is reflected mainly in its
features. For example, a new television model may cost the same in the current period as
the older model did in the previous period, but have a higher screen resolution. In this
case, households are able to receive more/better features for the same price, so this is
shown as a price decrease in the CPI.
Strong price decreases for audio-visual equipment reduce the effective expenditure
weight of this class between CPI reweights. This reduction in the effective expenditure
weight could understate price change in the CPI if the relative quantity or quality of audiovisual products were increasing, as the quality-adjusted price decreases for the audiovisual class will become underweighted.
The effective expenditure weight at each quarter is used to weight quarterly price
movements. The effective expenditure share of the audio-visual equipment class begins
to decrease (as quantity and quality are fixed) relative to actual household spending
patterns, which reflect a shift towards more highly featured technology. This effect is
highlighted when the weights are updated. Had the CPI not been reweighted in 2011,
price movements for the audio-visual equipment class between the June 2011 and
September 2011 quarters would have been weighted using an expenditure share of 0.52
percent, compared with an updated expenditure share of 0.81 percent, which was
calculated as part of the 2011 CPI review. Figure 4.3 highlights this difference.
102
Frequency of weight updates
Figure 4.3
CPI effective expenditure share
Audio-visual equipment class
1.2
Percent
2006 reweight
2008 reweight
2011 reweight
1.0
0.8
0.6
0.4
0.2
0.0
J S
06
D
M
07
J
S
D
M
08
J
S
D
M
09
J
S
D
M
10
J
S
D
M
11
J
S
D
M
12
J
S
D
Source: Statistics New Zealand
Decreases in the effective weight of audio-visual equipment, which are mainly due to
quality adjusted price decreases, result in a pattern that sees the effective weight of
audio-visual equipment progressively decrease until the next CPI review. At that time, the
expenditure weights are recalculated to reflect the more up-to-date quantities and quality
underlying household spending patterns.
Updating the expenditure share for audio-visual equipment on an annual basis would
make the effective expenditure shares of the audio-visual equipment class more relevant
over time, reducing the difference in expenditure shares at reweight periods. This would
mean that price changes for audio-visual equipment would have an influence on the all
groups CPI that better reflected the level of spending on this class. Figure 4.4 illustrates
how the effective expenditure shares of audio-visual equipment might look if they were
updated annually.
Figure 4.4
Analytical CPI effective expenditure share
Audio-visual equipment class - annual update
1.2
Percent
2006
2007
2008
2009
2010
2011
2012
1.0
0.8
0.6
0.4
0.2
0.0
J S
06
D
M
07
J
S
D
M
08
J
S
D
M
09
Source: Statistics New Zealand
103
J
S
D
M
10
J
S
D
M
11
J
S
D
M
12
J
S
D
Frequency of weight updates
Expenditure weights for the June 2007, 2009, and 2010 quarters were estimated by
simply interpolating the estimated expenditure levels between review periods, while the
expenditure weight for the June 2012 quarter was extrapolated from the 2008 and 2011
weights. These expenditure weights were then expressed as a proportion of total
household spending (an expenditure share). Weights could be updated using transaction
data for consumer electronics, provided by GfK, and adjusted for any population
changes. Adjusting for changes in the population ensures that changes in quantities
reflected in the new weights do not reflect changes resulting from population change
alone.
While this would go some way towards solving the problem of a rapidly decreasing
expenditure share, updating weights on an annual basis lessens rather than removes the
problem. It could be possible to update weights on a quarterly basis, using an appropriate
superlative index formula.
See the ‘Retail transaction data’ chapter for more details on measuring price change
using retail transaction data.
A major issue with updating part of the CPI basket is the lack of data for the remainder of
the basket. This is more of a problem at higher levels of the index, where non-HES
expenditure information sources are less complete. For example, if households
substituted from eating in restaurants towards home-cooked meals, the relative spending
on grocery food would increase and the relative spending on restaurant meals would
decrease, and the relative spending of food as a whole may stay the same or fall. If retail
transaction data were used to update the weight of grocery food only, this would show as
an increase in the relative weight of grocery food, without a related drop in restaurant
meals. This would mean the relative shares of grocery food and restaurant meals would
not accurately reflect current household spending patterns.
4.6.3 Frequency of CPI reviews
Three-yearly CPI reviews focus on reweighting and reselecting the basket of
representative goods and services. Reviews of the CPI coincide with running the HES.
Currently, the HES expenditure module is run every three years, while the income and
housing expenditure module runs every year.
If weights for some CPI items were updated on a more frequent basis, consideration
could be given to reducing the frequency of regular CPI reviews. Administrative data
could be used to annually update the weights for parts of the basket that are subject to
strong commodity substitution. The remaining CPI items could be updated on a four- or
five-year cycle and remain within ILO guidelines. This would mean the HES expenditure
module would need to run only every four or five years, rather than every three years.
While fully reweighting the CPI less frequently would be cheaper, and reduce respondent
load for people completing the HES, the quality of the CPI expenditure weights would be
reduced, as there would be more commodity substitution. This could potentially increase
costs for central government – for every 0.1 of a percentage point that the CPI overstates
true annual inflation, there is a potential adverse impact of about $20 million on central
government fiscal net transfers.
Firstly, administrative data is not available for all CPI items that could benefit from morefrequent CPI updates. For these items, the current three-yearly HES expenditure data is
the only available data source. Updating the weights of these items on a less-frequent
basis would result in a decline in the relevance of the CPI.
If subpopulation indexes were compiled, the additional demographic information collected
in the HES would be required to calculate fit-for-purpose subpopulation weights. This
information is not available from administrative data or retail transaction data, which
usually relates only to expenditures, price, quantity, and in some instances, the quality
104
Frequency of weight updates
characteristics of individual products. If HES data was available less frequently, the
quality of subpopulation weights would be lower.
Similarly, some expenditure estimates based mainly on administrative data need to
incorporate lower-level HES information (eg the type of store an item is purchased from).
The quality of the annual expenditure estimates would be lower if the HES data was
available less frequently.
Coherence of data could also be more of a problem. If parts of the basket were updated
annually (or monthly/quarterly) and the remainder only every four or five years, then the
issue of basing the relative importance of CPI basket items on quantity weights from
different periods is exaggerated even further than if the HES remains three-yearly.
For a reduced frequency of full CPI reviews to be a viable option, there would need to be
enough data to update the weights of all CPI items that are subject to commodity
substitution – between review periods. At this stage, administrative and retail transaction
data is estimated to be available for about half the CPI basket, by weight.
Alternatively, the frequency of full CPI reviews (and thus the HES) could be increased to
an annual or two-yearly frequency. This would reduce the effect of commodity
substitution in the CPI, in turn making the CPI more relevant. This could potentially
decrease costs for central government – for every 0.1 of a percentage point that the CPI
overstates true annual inflation, there is a potential adverse impact of about $20 million
on central government fiscal net transfers.
However, it would be more expensive to fully reweight the CPI and run the HES
expenditure module more frequently. It is estimated that the cost of each three-yearly CPI
review is about $650,000, including overheads, and running both the income and
expenditure modules of the HES is around $1.9 million, including overheads. The HES
income and housing expenditure module is run each year and is estimated to cost around
$900,000.
4.7 Issues with updating expenditure weights on a
more frequent basis
4.7.1 Deciding what to update
Deciding what to update would be limited to areas where data is available. The effect of
reweighting parts of the basket would be greater for higher-weighted items and items
where significant quantity/quality shifts have occurred. However, appropriate information
may not be available to update some items that could benefit from more-frequent
updates.
A good case exists for updating the audio-visual equipment class because of the
systematic quality change and the relatively high weight and rapid quality-adjusted price
change of the class. Cigarettes and tobacco could also be considered – they are currently
going through a phase of high relative price increases, due to 10 percent annual
increases in the excise tax on these items. Changes in the excise tax charged for these
items aim to reduce the overall quantity of tobacco consumed.
Figure 4.5 shows that the effective weight for cigarettes and tobacco increased between
the 2008 and 2011 reweights, due to high relative price change, which potentially overweights these price increases.
105
Frequency of weight updates
Figure 4.5
CPI effective expenditure share
Cigarettes and tobacco subgroup
3.0
Percent
2006 reweight
2008 reweight
2011 reweight
2.5
2.0
1.5
1.0
0.5
0.0
J S
06
D
M
07
J
S
D
M
08
J
S
D
M
09
J
S
D
M
10
J
S
D
M
11
J
S
D
M
12
J
S
D
Source: Statistics New Zealand
Given the relatively high weight of the cigarettes and tobacco subgroup, this could be a
good candidate for an annual weight update.
The petrol class has a high weight in the index. However, there is little evidence that the
relative quantity of petrol has changed significantly between review periods, illustrated by
only small changes in expenditure shares at reweight periods. Figure 4.6 shows the
effective weight of petrol since the June 2006 quarter.
Figure 4.6
CPI effective expenditure share
Petrol class
7
6
Percent
2006 reweight
2008 reweight
2011 reweight
5
4
3
2
1
0
J S
06
D
M
07
J
S
D
M
08
J
S
D
M
09
J
S
D
M
10
J
S
D
M
11
J
S
D
M
12
J
Source: Statistics New Zealand
Based on this evidence, it is unlikely that updating the expenditure weight for petrol
annually would have much of an effect, as the underlying relative quantities appear to
remain relatively constant between CPI reweights.
106
S
D
Frequency of weight updates
4.7.2 Trade-offs between coherence and relevance
There is potential for CPI expenditure weights to lose coherence if some weights are
updated more frequently than others, however, about 20 percent of the CPI basket (by
weight) already has different weight reference periods to the majority of CPI items. This is
either due to ‘volume adjusting’ the underlying quantities of those items during threeyearly CPI reviews, or using a weight reference period that is longer than one year.
Because of the time delay in producing new expenditure weights, underlying quantities
from the weight reference period may change significantly while the new weights are
being calculated. In cases where this has happened, ‘volume adjustments’ are applied so
the new weights reflect more up-to-date quantities. For example, in the 2011 CPI review,
cigarette and tobacco quantities were updated to reflect a decline in tobacco consumption
between the weight reference period (the year to June 2010) and the price reference
period (June 2011 quarter). The underlying quantities used in the final weight for
cigarettes and tobacco represented the quantities purchased in the year ended June
2011, rather than the year ended June 2010, which was the weight reference period used
for most basket items. Volume adjustments were carried out to about 5 percent of the CPI
basket by weight. A further 15 percent of the CPI basket (by weight) uses a weight
reference period that is an average of three years of expenditure to partially smooth the
impact of cyclical highs or lows in activity. These are:
• owner-occupied housing
• rentals for housing
• insurance.
Updating the weights for some items more frequently would simply increase this
percentage, from about 20 percent of the CPI basket by weight, to anywhere up to half
the basket, based on the amount of data currently available. This could be further
increased in the future if more administrative data becomes available.
Under the status quo, the weight reference period is the same for a majority of CPI items,
but weights for certain items become less relevant over time. Updating the weights of
some CPI items (either the relative importance of items within expenditure classes, or
at/above expenditure classes) could reduce the coherence of the CPI expenditure
weights, but may increase the relevance.
Annual weight updates would not necessarily have to be carried out for the headline CPI.
Instead, a separate analytical series could be constructed using weights that are updated
more frequently. This series would potentially go some way towards a real-time estimate
of the effect of commodity substitution in the CPI and could even use a superlative index
formula such as a Fisher or Törnqvist. There would be an additional cost in producing
another analytical series.
4.7.3 Framework for updating weights
If Statistics NZ undertook more-frequent updates of some CPI items, a framework would
need to be established. Such a framework would outline the decision-making principles,
process, and criteria for weight updates, and outline at what levels these updates were to
take place. Any updates made would need to be transparent and well communicated.
The framework would be developed to inform users of the process, and reduce the risk
that partial weight updates were seen as being subjective, or selective.
4.7.4 Cost estimate
Updating weights more frequently for parts of the CPI basket could be done within the
prices statistical CPI maintenance programme. However, there would be trade-offs with
other prices statistical maintenance, such as possibly scaling back the rolling review of
retail outlets sampled and product specifications.
107
Frequency of weight updates
For areas where retail transaction data could be used to calculate price change and to
update the relative importance of items, this would take place mainly as part of the usual
CPI compilation, but it would also require some prices statistical maintenance resources.
4.8 Options for more frequent weight updates, and
implications
Table 4.2 summarises some of the options discussed in this chapter, and the implications
of implementing each option.
Table 4.2
Options and implications for more frequent weight updates
Option
Implications
•
all CPI items reweighted threeyearly using HES and other
sources (status quo)
•
•
•
•
volume adjustments and threeyear weight reference periods
for some CPI items during full
CPI reviews (status quo)
•
•
•
•
update weights within selected
expenditure classes, where
information from alternative
sources is available between
three-yearly CPI reviews, to
reflect volume-related shifts
(currently done only
occasionally)
•
•
•
•
•
price change measured using
retail transaction price and
quantity data where available;
weights implicitly updated
update of weights at class level
and above for selected
items/categories to reflect
volume-related shifts
•
•
•
•
108
reweighting frequency remains in the middle
of international practice
effect of commodity substitution about 0.1 of a
percentage point per year (broadly in line
with, and at the lower end of, international
studies)
price change for expenditure classes with
high/low relative price change becomes
increasingly over/under weighted (eg audiovisual equipment, cigarettes and tobacco)
between three-yearly reweights.
weights for some CPI items reflect more upto-date household spending patterns
weights for some CPI items based on an
average of three years of expenditure to
partially smooth the impact of cyclical highs or
lows in activity
about 20 percent of CPI basket (by weight)
has a different weight reference period.
lower-level weights remain more relevant
between review periods for classes where
data is available
CPI expenditure weights are less coherent –
weight reference periods could be different for
up to half of the CPI basket (by weight) based
on data currently available, but only at the
lower levels (quantities at class level and
above still based on those calculated at latest
CPI review)
resource trade-off with other prices statistical
maintenance work, such as CPI rolling review
of field outlets.
more accurate price measures for some
areas of the basket
relative quantity shifts between items
accounted for
could be subject to chain drift without
appropriate chaining methodology.
price change and relative importance of some
expenditure classes (eg audio-visual
equipment, cigarettes and tobacco) more
relevant between CPI reviews
Frequency of weight updates
•
•
•
all CPI items reweighted four- or
five-yearly using HES and other
sources (decreased CPI review
and HES frequency)
•
•
•
•
•
•
all CPI items reweighted
annually or two-yearly using
HES and other sources
(increased CPI review and HES
frequency)
•
•
•
•
•
CPI expenditure weights are less coherent –
weight reference periods could be different for
up to half of the CPI basket (by weight) based
on data currently available
resource trade-off with other prices statistical
maintenance work, such as CPI rolling review
of field outlets.
effect of commodity substitution on CPI
increased
lower respondent load for households
completing the HES
lower cost to run HES and update CPI more
frequently
higher net central government fiscal transfers
as a result of indexing welfare benefits and
excise duty
CPI reviews still within international best
practice guidelines of at least every five
years, but at the less frequent end of the
spectrum.
effect of commodity substitution on CPI
reduced
higher respondent load for households
completing the HES
higher cost to run HES and update CPI more
frequently
lower net central government fiscal transfers
as a result of indexing welfare benefits and
excise duty
CPI reviews still within international best
practice guidelines of at least every five
years, and at the more frequent end of the
spectrum.
4.9 Conclusion
For some goods and services, data is available that would allow more-frequent updating
of expenditure weights. This data could be used to update the relative importance of up to
half the CPI basket (by weight), based on the data that is currently available. Updates
could be carried out within an expenditure class (eg the relative importance of large,
medium, and small new cars), which is current practice but could be done more often, or
it could be used to update the expenditure shares at the class level and above (eg audiovisual equipment).
Where administrative data is available, weight updates could be carried out explicitly on
an annual basis. Explicitly updating CPI weights more frequently for some parts of the
basket could be done within the prices statistical maintenance programme, although
some reprioritising within the current programme would need to occur.
Similarly, superlative index formulae could be used to implicitly update the weights for
areas of the basket where retail transaction data is available on a monthly or quarterly
basis. This would be carried out as part of the regular CPI compilation, but would also
require some call prices statistical maintenance resources.
More frequent weight updates for some parts of the basket would help to reduce the
impact of commodity substitution on the CPI, making it more relevant. While this could be
viewed as making CPI weights less coherent, about 20 percent of the current CPI basket
(by weight) has different weight reference periods to the remainder of the basket. This is
109
Frequency of weight updates
due either to a volume adjustment or having a weight reference period that is based on
more than one year of expenditure. Updating the weights of some CPI items would simply
increase this proportion.
The frequency of full CPI reweights could also be considered. If some CPI weights were
updated annually, the frequency of three-yearly CPI reviews could potentially be reduced.
This would reduce the respondent load for households completing the HES, but would
make the weight of CPI items where the HES is the only data source less relevant over
time. Alternatively, the frequency of full CPI reviews could be increased. This would be
more costly and the respondent load for households filling out the HES would increase,
but the CPI would be more relevant.
If the weights of some CPI items were updated more frequently, a framework would need
to be set out in advance. Such a framework would outline the decision making principles,
process, and criteria for weight updates, and outline at what levels these updates were to
take place. This would make the process more transparent and objective.
110
5 Retail transaction data
5.1 Executive summary
This chapter discusses both the potential benefits and challenges of incorporating
scanner data into the production of the New Zealand CPI.
New methodologies are required. Statistics New Zealand has contributed to international
research in this area. In particular, collaborative work with Statistics Netherlands,
developing and testing new methods on a detailed research dataset of New Zealand
consumer electronics scanner data, has given important new insights.
5.2 Issues to consider
The issues Statistics NZ would like the committee and CPI user community to consider
are:
• Should Statistics NZ continue to pursue the use of scanner data – to measure price
change in the CPI and reflect shifts in the relative importance of goods?
• Does the committee agree the tentative conclusions on methodology (ie the use of
ITRYGEKS (TD) for consumer electronics and RYTPD for supermarket data) are
appropriate?
• Should Statistics NZ look to make full use of the price and quantity information
available in scanner data?
• Should Statistics NZ continue with the current treatment of seasonal goods (eg
fresh fruit and vegetables) until the introduction of scanner data, which would result
in improved price measurement for these products?
5.3 Introduction to using scanner data
Many products, for example those purchased at supermarkets, have their barcodes
scanned at the time of purchase. This retail transaction data – or 'scanner’ data – records
prices, quantities sold, and associated information for all transactions (not just a sample)
across the full reference period.
From the 2006 Consumers Price Index Review onwards, aggregated scanner data for
supermarket products and for consumer electronics has been well used to:
• determine the expenditure weights of some goods in the CPI basket
• determine whether expenditure weight adjustments are required to reflect volume
changes since the weight reference period (but before implementation of reviews)
and, if so, by how much
• select representative products to survey when price collectors visit retail outlets
each month or quarter
• ensure the mix of brands in the CPI price samples reflect market shares.
Since 2008, Statistics NZ has been actively researching the further potential of using
more-detailed scanner data for directly estimating price change for products sold through
supermarkets, and for consumer electronics products. The focus has been on
determining appropriate methodologies. Traditional index formulae are problematic when
applied to scanner data for two reasons:
• the volatility of prices and quantities, due to discounting and seasonality
111
Retail transaction data
• the high degree of ‘churn’ – ie new products entering and old products leaving the
market.
5.3.1 Identifying which method to use
Statistics NZ has collaborated with Statistics Netherlands by empirically testing, on New
Zealand consumer electronics scanner data, a new benchmark index method called the
Imputation Törnqvist Rolling Year GEKS (ITRYGEKS). This method extends the rolling
year GEKS (RYGEKS) method proposed by Ivancic, Diewart, and Fox (2011), which had
been seen as the benchmark method for scanner data. The research showed RYGEKS
can be biased – by not accounting for the implicit price change associated with new and
disappearing products. The ITRYGEKS method incorporates hedonic modelling 1 into the
RYGEKS approach in such a way that new and disappearing products are dealt with
appropriately.
The ITRYGEKS method appears to be feasible and appropriate for the consumer
electronics scanner data, which has extensive information on product characteristics
available for the hedonic models that the ITRYGEKS method utilises.
However, for supermarket scanner data, the ITRYGEKS method is unlikely to be able to
be fully implemented, as there is likely to be insufficient information on product
characteristics in the data to implement the hedonic modelling the method requires.
Research is still underway into the most-appropriate methodology to use for
supermarkets. At this stage, a rolling year 'time product dummy' (RYTPD) hedonic
method appears to be an improvement over the RYGEKS method for supermarket
products.
5.3.2 Benefits of using scanner data
The potential benefits of using retail transaction data to measure price change include:
• improved accuracy, due to greater coverage of transactions and availability of realtime quantities
• ability to use existing administrative-type data sources
• improved treatment of seasonal commodities
• ability to account for commodity and product substitution between reweights.
5.4. The advantages of scanner data
5.4.1 It’s more accurate
Currently, the CPI relies on sampling prices across several dimensions – commodities,
products, outlets, and time. Quantities are based on information acquired during the
Household Economic Survey reference period, and are updated only every three years. It
is difficult to accurately estimate the sampling error associated with current practice, as it
is largely based on informed judgement rather than probabilistic sampling.
1
Hedonic modelling is the use of regression modelling to control for compositional shifts in the
characteristics of goods being sold. Price (or, more usually, the log of price) is modelled against time (eg
‘month’) and price-determining characteristics (eg TV screen size), and the index is derived from the
parameters estimated for time. This ensures that the effect, on the average price, of the change in
quality composition of goods being sold (in terms of the characteristics observed and included in the
hedonic regression model), is removed from the price index.
112
Retail transaction data
In contrast, scanner data has the potential to give a more complete picture of both prices
and quantities sold at any point in time. Depending on the scanner data’s source, there
may also be information on the characteristics of each product, which can be utilised for
quality adjustment.
For supermarket scanner data it would be possible to disaggregate the data regionally.
This would potentially mean having more accurate regional disaggregation for these
products. Region is not currently available on the consumer electronics scanner data, but
there is likely to be only moderate regional variation for these products.
5.4.2 It re-uses existing data
Given that scanner data is already collated by or for businesses, it would be good
practice to re-use it to generate official statistics. This would reduce fieldwork, and the
respondent load associated with collection, which involves observing products and prices
in stores, and discussing product changes with store staff, particularly for consumer
electronics. While there is likely to be an increase in the analytical resource required to
introduce and maintain a robust scanner data production system, emerging consensus
among national statistical offices is that this is the direction a modern statistical office
should be moving in. The net result is likely to be a similar level of resources required to
produce more accurate statistics.
New Zealand consumer electronics data
Statistics NZ has been using scanner data for consumer electronics products from market
research company GfK for several years, to inform expenditure weighting in the CPI. This
data is close to full-coverage of New Zealand’s market, and contains sales values and
quantities aggregated to quarterly levels for combinations of brand, and up to six
characteristics.
Recently, Statistics NZ purchased a more detailed dataset for mid-2008 to mid-2011 for
eight products:
• camcorders
• desktop computers
• digital cameras
• DVD players and recorders
• laptop computers
• microwaves2
• televisions
• portable media players.
Monthly sales values and quantities are disaggregated by brand, model, and around 40
characteristics. This data was used in the research on methods described in this chapter.
5.4.3 It can handle seasonal variation in quantities
Price change for some products is currently difficult to measure accurately because of the
seasonality of quantity shares. The current fixed-basket approach, when applied to
seasonal prices and quantities, has the potential to magnify or distort actual price
2
Microwaves are not strictly a ‘consumer electronics’ product but, as a product with less rapid
technological change, they provide a useful comparison of how different price index methods perform.
113
Retail transaction data
movements. In the extreme case, some products have seasonal periods of unavailability
– which poses a particular challenge. Index estimation could be improved by modifying
the traditional index formula / methods used (eg by using seasonal baskets).
However, the data limitations associated with current practice mean that even the best
solution applied to traditional data sources will still be less than optimal. In particular, is
the fact that the quantities underlying expenditure shares are updated only every three
years (although they vary within these periods).
Scanner data, which has real-time information on both prices and quantities, is more
suited to accurately estimating price movements for these seasonal goods. ITRYGEKS
(for consumer electronics) or RYTPD (for supermarkets) would accurately reflect the price
movements, taking the corresponding actual quantity changes into account appropriately.
Note that some products with seasonal expenditure shares / availability, would still
display seasonality in price indexes derived from scanner data – for example, many fresh
fruit and vegetables. Whether seasonal adjustment (to try and identify the underlying
trend in price movement) is necessary and/or feasible, is a separate issue. It may be
more desirable to focus on annual than monthly movements for these products. Fresh
fruit and vegetables used to be seasonally adjusted in the CPI, but the high irregular
component, along with not being able to revise, meant it was not adding a lot of value,
and it was discontinued after the 2006 review. Seasonal adjustment of a revisable
analytical series seems a more plausible option for future consideration.
Figure 5.1, from Krsinich (2012), shows the seasonal patterns in expenditure shares of
the eight consumer electronics products investigated.
Figure 5.1
Expenditure shares
0.5
Camcorders
0.4
Desktops
0.3
Digital cameras
0.2
DVD/Blu-ray players and
recorders
Laptops
0.1
0
Microwaves
J A S O N D J F M A M J J A S O N D J F M A M J J A S O N D J F M A M J
09
10
08
11
Televisions
Portable media players
Source: Statistics New Zealand, GfK
In figure 5.2, 3 looking at digital cameras, seasonality is present in total expenditure,
quantities, and average price (unadjusted for quality change). Obviously, this would be a
challenging product to accurately measure price movements for – using prices sampled
across time and quantities averaged across the year.
3
Note that, for confidentiality reasons, expenditure, quantity and average price values were each
rescaled so the July 2008 value is equal to 1. The relative levels are unaffected by this rescaling.
114
Retail transaction data
Figure 5.2
Digital cameras
3.5
Total expenditures
3.0
Average prices
Quantities
2.5
2.0
1.5
1.0
0.5
0.0
J A S O N D J F M A M J J A S O N D J F M A M J J A S O N D J F M A M J
09
08
10
11
Source: Statistics New Zealand, GfK
By using an appropriate index methodology, a quality-adjusted price index that deals with
the seasonality of expenditure shares and quality composition appropriately can be
produced. In this case it would be ITRYGEKS, which utilises all prices and quantities
appropriately and imputes price movements for new and disappearing goods based on
hedonic models. Figure 5.3 shows this. ITRYGEKS is shown next to the index based on
the unit price, which reflects the seasonality of quality composition within digital cameras
– there is no obvious seasonality remaining in the quality-adjusted price movements.
Figure 5.3
Digital cameras
1.2
Index
1.0
0.8
0.6
0.4
0.2
0.0
ITRYGEKS (TD)
Unit price
J A S O N D J F M A M J J A S O N D J F M A M J J A S O N D J F M A M J
08
09
10
11
Source: Statistics New Zealand, GfK
5.4.4 It accounts for substitution
Because scanner data has the potential to provide prices and quantities for the mostdetailed level of product specification (ie barcode level), price indexes estimated from this
data by using appropriate methodology will reflect consumers’ substitution behaviour
across products with different features. Scanner data can also be used to empirically test
substitution effects for different product categories, to infer the appropriate level at which
to fix expenditure shares.
115
Retail transaction data
5.5 The challenges of scanner data
The previous section emphasised the advantages of fully utilising the rich information content
of scanner data. However, the behaviour of prices and quantities at the barcode level, as
reflected by scanner data, mean that traditional index methodologies do not work well.
There are two main reasons for this – product churn, and the volatile nature of prices and
quantities due to discounting, seasonality, and the life-cycle of products.
'Churn' refers to the turnover in products being sold in the market. For some products, in
particular consumer electronics, this turnover can be significant. Figure 5.4 shows the
percentage of products sold in July 2008 still on sale each month over the subsequent three
years, for the eight consumer electronics products investigated. For laptop computers, only
around 10 percent of the July 2008 products were still being sold a year later.
Note that current CPI practice for consumer electronics is to regularly update products being
priced during the time between the three-yearly expenditure weight updates. Therefore, the
basket will be more representative than is implied by figure 5.4.
Figure 5.4
Percentage of July 2008 models available
July 2008 to June 2011
120
Percentage
Camcorders
Desktop computers
100
Digital cameras
80
60
DVD players and
recorders
Laptop computers
40
Microwaves
20
Televisions
0
Portable media players
J A S O N D J F M A M J J A S O N D J F M A M J J A S O N D J F M A M J
08
09
10
11
Source: Statistics New Zealand, GfK
Figure 5.5 shows a more detailed picture of the product churn for televisions. It shows the
number of July 2008 products still being sold each month over the three-year period, but
also the number of June 2011 products being sold in the three years up to that month,
and the total number of distinct products sold each month.
116
Retail transaction data
Figure 5.5
Televisions
July 2008 to June 2011
250
Number of products
200
150
100
50
0
J A S O N D J F M A M J J A S O N D J F M A M J J A S O N D J F M A M J
08
10
11
09
All items
Matched to July 2008
Matched to June 2011
Source: Statistics New Zealand, GfK
Given this high degree of product churn, the obvious solution is to use a high-frequency
(eg monthly) chained superlative index (eg Törnqvist).This would maximise the number of
matched products included in the index calculation and reflect substitution across
products by incorporating updated quantities each month.
A superlative index is one that utilises both current- and reference-period quantity shares
symmetrically, which results in substitution between products being accounted for in the
index appropriately.
In a monthly chained index, the basket of products and their associated quantities or
expenditure shares are updated every month. The index is calculated between the
previous and current month on the basis of these updated products and weights and
linked onto the previous index value.
It has been shown by researchers (eg Ivancic, Diewart, & Fox, 2011) that high-frequency
chained superlative indexes can result in substantial ‘chain drift’ when applied to
supermarket scanner data, due to a significant amount of price and quantity ‘bouncing’
because of discounting and seasonality.
Chain drift is the bias that occurs when a chained index diverges, or systematically ‘drifts’
away, from its direct (ie unchained) counterpart. A chained index in which the return of
prices and quantities to previous levels does not correspond to the index also returning to
the previous level, is exhibiting chain drift.
Supermarket products tend to be discounted frequently and, as might be expected,
quantities bought increase sharply in response to these discounts.
In 2010, Statistics NZ and Statistics Netherlands were contracted as peer-reviewers of
the Australian Bureau of Statistics research into supermarket scanner data. This provided
access to the research dataset, which was used as a basis for developing a suite of SAS
programs to run several different index methods (see appendix 5a). Krsinich (2011a)
shows results from applying these methods.
For confidentiality reasons, price indexes from this data could not be published directly,
but the indexes in terms of their difference from the then benchmark method (the rolling
year GEKS, or RYGEKS) were published. RYGEKS is explained in the next section.
117
Retail transaction data
Figure 5.6 shows how three different methods differ from RYGEKS. In this example, the
monthly chained Törnqvist has a downward drift in comparison with the RYGEKS
benchmark.
Figure 5.6
Soft drinks – bottles and cans
0.10
Difference from RYGEKS
index
0.08
0.06
Monthly chained
0.04
0.02
Annually chained
0.00
TPD
-0.02
-0.04
-0.06
J A S O N D J F M A M J J A S O N D J F M A M J J A S O N D J F M A M J J
07
08
10
09
Source: Statistics New Zealand, GfK
5.6 The solutions so far
5.6.1 The rolling year GEKS
To address the problems outlined in the previous section, Ivancic, Diewart, and Fox
(2011) proposed a method for producing price indexes from scanner data that uses all
the prices and quantities in the data, and is free of chain drift. This is called the rolling
year GEKS (RYGEKS). RYGEKS is based on the Gini, Eltetö and Köves, and Szulc
(GEKS) method, which is used for multilateral spatial price indexes such as purchasing
power parities (these compare prices in different countries at a point in time).
Within a window of time (usually one year, plus a period to allow for seasonal
unavailability, so five quarters or 13 months) the RYGEKS index between periods t1 and
t2 is the geometric mean of all the superlative bilateral indexes (such as the Törnqvist or,
in Ivancic et al (2011), the Fisher) between:
3. t1 and all the other periods in the window,
4. t2 and all the other periods in the window.
Formulating the monthly RYGEKS with a 13-month rolling window is as follows:
For the first window, ie t=0 to 12, RYGEKS is equal to GEKS:
Where
is any superlative index (eg Törnqvist) between periods i and j.
From t=13 onwards, RYGEKS links on the most-recent movement from the GEKS
calculated on the next window (ie from t=1 to 13, then from t= 2 to 14, and so on) as
follows:
118
Retail transaction data
and so on.
However, a problem with the RYGEKS method is that it relies on the price movements
between matched products only. Any implicit price change associated with new or
disappearing products is effectively 'linked out'. So, for example, if the initial price of the
latest model of a mobile phone is high relative to the features (ie the quality) of the phone
then this implicit price increase is not included in RYGEKS. Conversely, if this new model
is introduced at a price that is low relative to its features then this implicit price decrease
will similarly not be reflected in the RYGEKS index.
5.6.2 The imputation Törnqvist rolling year GEKS
Jan de Haan, of Statistics Netherlands, proposed an extension to the RYGEKS method
that addresses this limitation of RYGEKS. This uses hedonic models to impute for new
and disappearing products, and is called the imputation Törnqvist rolling year GEKS
(ITRYGEKS). The method was empirically tested on the New Zealand consumer
electronics scanner data. This work was outlined in de Haan and Krsinich (2012), and
presented at these conferences and workshops during 2012, from which useful feedback
was gained:
• UNECE/ILO meeting of the group of experts on consumer price indexes, Geneva,
May 2012
• Statistics Sweden’s workshop on scanner data, Stockholm, June 2012
• New Zealand Association of Economists conference, Palmerston North, June 2012
• Economic Measurement Group workshop, University of New South Wales, Sydney,
November 2012.
Unlike RYGEKS, which is based on superlative indexes (eg Fisher or Törnqvist),
ITRYGEKS is based on geometric means of hedonic bilateral indexes. The formulation is
as above for RYGEKS, with the difference that the
are bilateral, weighted, timedummy hedonic indexes. Three versions of ITRYGEKS are proposed in the paper:
• based on explicit imputation – which requires numeric data and/or categorical data
that does not have new or disappearing categories
• based on time product dummy hedonic indexes – which is shown algebraically to
be redundant as it is equivalent to doing no imputation for new and disappearing
specifications
• based on time dummy hedonic indexes – ITRYGEKS(TD), this was applied to the
consumer electronics data.
By taking the mean expenditure shares as weights for the matched items, and half of the
expenditure shares for the unmatched products, the paper shows that ITRYGEKS(TD) is
algebraically equivalent to a Törnqvist for the matched items. For new and disappearing
products, it applies a Törnqvist formula to prices predicted from hedonic models for the
period in which there is no price available.
That is, ITRYGEKS(TD) from period 0 to t can be expressed as follows:
119
Retail transaction data
Where
is the set of matched products with respect to periods 0 and t
is the set of 'disappearing' products with respect to periods 0 and t
is the set of 'new' products with respect to periods 0 and t
Note that this expression can be generalised to ITRYGEKS between any two periods i
and j.
ITRYGEKS(TD) was applied to the New Zealand consumer electronics data for eight
products:
• camcorders
• desktop computers
• digital cameras
• DVD players and recorders
• laptop computers
• microwaves
• televisions, and
• portable media players.
In addition to applying the ITRYGEKS(TD) method, indexes were produced from the
scanner data using other methods, including two different approaches to hedonic indexes
based on rolling 13-month windows of data:
• a rolling year ‘time dummy’ hedonic index (RYTD), which is derived from regression
models where the log of price is modelled against time and the characteristics of
the products
• a rolling year ‘time product dummy’ index (RYTPD), which includes the product
identifiers as the ‘characteristics’ being controlled for by the regression model.
The following conclusions were reached:
1. The monthly chained Törnqvist is not a viable method for consumer electronics,
as it appears to have downward chain drift for most products examined, except
microwaves and portable media players.
2. RYGEKS shows evidence of bias, due to not accounting for the price movements
of new and disappearing products, particularly computers.
3. The easier-to-implement rolling year time dummy (RYTD) hedonic index (which
explicitly incorporates characteristics into hedonic models) gives similar results to
ITRYGEKS(TD), in particular for computers.
4. In some cases, such as supermarket data, few or no characteristics are likely to
be available and so neither ITRYGEKS nor RYTD, which are both based on time
dummy hedonic models, will be feasible. Results suggest that in this situation a
rolling year time product dummy (RYTPD) – a 'fixed effects' approach, which
controls for the product identifiers rather than characteristics – does some
adjustment for quality change and is therefore preferable to RYGEKS.
Aggregation of the eight products (see figure 5.7), using their relative expenditure weights
from the scanner data, shows that the matched-product RYGEKS is upwards biased
compared with ITRYGEKS(TD), which imputes the implicit price movements of new and
disappearing products. The RYTD hedonic index and the monthly chained Törnqvist both
track the benchmark ITRYGEKS(TD) closely, but for the monthly chained Törnqvist this
appears to be a coincidental cancelling out of biases in two opposite directions – chain
120
Retail transaction data
drift and quality-change bias – for televisions, which have by far the most significant
weight of all the eight products.
Figure 5.7
Products aggregated to 'consumer electronics' for each method
1.2
Index
ITRYGEKS(TD)
1.0
RYGEKS
0.8
RYTD
ITRYGEKS(TPD)
0.6
RYTPD
0.4
Monthly chained Tornqvist
0.2
0.0
Unit value
J A S O N D J F M A M J J A S O N D J F M A M J J A S O N D J F M A M J
09
08
10
11
Source: Statistics New Zealand, GfK
5.6.3 Rolling year time product dummy (RYTPD)
ITRYGEKS(TD) is a feasible method only when there are sufficient characteristics in the
data to estimate the time dummy hedonic indexes. The research data obtained for
consumer electronics has extensive characteristic data, which could be incorporated into
a future production system to estimate price indexes for these products in the CPI.
However, for supermarket data, the only explicit characteristics likely to be available are
weight, volume, or size. This means ITRYGEKS(TD) may not be feasible for supermarket
products.
Because de Haan and Krsinich (2012) showed empirically that the rolling year time
product dummy (RYTPD) hedonic approach tends to sit closer to the ITRYGEKS(TD)
benchmark than the RYGEKS approach does, at this stage RYTPD looks to be the most
promising method to apply for supermarket data.
Theoretical work is underway to determine what the implicit imputations for new and
disappearing products are, under the RYTPD method.
Note that the RYTPD method is now the focus of further collaborative research with
Statistics Netherlands, and is the topic of the paper to be presented by Statistics NZ at
the Ottawa Group meeting of CPI experts in Copenhagen in May 2013 (see appendix
5b).
5.7 Current international practice
A significant body of research exists on scanner data, but actually implementing it in
production in official statistical agencies has been limited.
Statistics Netherlands and Statistics Norway are the only countries to comprehensively
incorporate supermarket scanner data into their CPI production systems. Some other
countries (eg Switzerland) implement scanner data in a partial way by using it as a price
source for the selected products in the traditional fixed-basket approach for
supermarkets. The disadvantage of this approach is that substitution across products will
not be reflected.
121
Retail transaction data
Although RYGEKS has tended to be considered a good benchmark method for scanner
data, it is also relatively new and untested, and has therefore not been adopted into
production. Statistics Netherlands' use of scanner data in production uses unweighted
Jevons 4 indexes at the elementary aggregate level, combined with cut-off sampling
based on expenditure shares. They run RYGEKS separately as a shadow system, and
use it as a benchmark to help inform the thresholds for sampling. (See de Haan & van
der Grient, 2011.)
Statistics Norway use a monthly chained Törnqvist approach to estimating price indexes
for food and non-alcoholic beverages, but this approach has been shown to be
downwardly biased (see Johansen, I & Nygaard, R, 2011).
The official statistical agencies of many other countries are researching the potential of
putting scanner data into production. This includes the Australian Bureau of Statistics,
which aims to implement scanner data into production for supermarkets in the near
future.
5.8 The current research focus
In New Zealand, it appears to be feasible to put consumer electronics scanner data into
production, if access to the data at a disaggregated level with detailed characteristics can
be gained on a timely basis. The ITRYGEKS(TD) method to estimate unbiased price
indexes for these products could be implemented.
The research focus now is on what method will be appropriate for estimating price
indexes for supermarket data, which would not have the detailed characteristics
necessary to apply ITRYGEKS(TD). At this stage the RYTPD method is looking
promising. Its potential will be discussed at the next Ottawa Group meeting. Note that
using a version of this method for rental dwellings has been researched – here there are
limited characteristics, but there is reason to believe there is implicit price change
associated with rental dwellings entering and leaving the rental market (Krsinich, 2011b).
Other options that can be considered for supermarket scanner data are:
• the potential for a cut-down version of the ITRYGEKS(TD) method if the weight,
size, or volume characteristics, in combination with categorical variables derived
from product identifiers, explain enough of the price variation
• manual intervention – such as identifying, linking, and appropriately qualityadjusting between barcodes, where the only change in product is a change in
weight, size, or volume.
See appendix 5b for the abstract of the paper accepted for the next Ottawa Group
meeting in May 2013 – Using the rolling year time product dummy method for quality
adjustment in the case of unobserved characteristics.
5.9 Potential issues associated with putting scanner
data into production
The research so far has focused on which methodology will be most appropriate for
consumer electronics and for supermarket scanner data. Other more practical issues are
likely to arise around putting these methods into production. These will be addressed as
data supply arrangements are clarified.
4
The Jevons index is the unweighted geometric mean of price relatives. A price relative is the ratio of
the price of an individual product in one period to the price of that same product in some other period.
122
Retail transaction data
Timing of supply for supermarket data may mean that figures based on the full month
may not be available in time to incorporate into production. This would require
investigation of the options and implications of partial or lagged data. Note that this kind
of data limitation is dealt with in production in existing Statistics NZ outputs such as the
Retail Trade Survey, which rates up partial periods of data where appropriate.
Data supply arrangements may not enable full coverage of all products. Even so, the
quality of price indexes using scanner data would be an improvement over the current
situation, but investigation of how best to incorporate any partial coverage would be
required.
The following table shows how limitations on the level of detail and coverage available
from the scanner data affect the ability to reflect quality-adjusted price movements
accurately. In the table, ‘product’ means the finest level of specification available – for
example, chocolate biscuits, manufacturer A, brand b, type C, weight 200g. ‘Item’ means
the good – for example, ‘chocolate biscuits’.
Table 5.1
Implications of different levels of detail from scanner data
Level of detail and information available
Implications
1. Full coverage of prices and quantities
across all items and products at all points
in time, with explicit information about
price-determining characteristics at the
product level.
This is the ideal situation where our
benchmark method – the ITRYGEKS(TD) –
can be applied to create price indexes
which are both fully quality adjusted and
fully reflect consumer substitution at the
level of items and products.
Example
This level of detail and information exists in
the GfK consumer electronics scanner data,
although the data is aggregated across time
to the monthly level and across transactions
to the model level, and region and outlet
information is not available.
2. Full coverage of prices and quantities
across all items and products at all points
in time, with limited or no information about
price-determining characteristics at the
product level.
There is less scope for fully quality
adjusting price indexes without explicit
information on characteristics, but the
RYTPD method discussed in the paper is
looking promising. Consumer substitution at
the level of items and products can be fully
reflected.
Example
This level of detail potentially exists in
supermarket scanner data and, unlike
consumer electronics scanner data,
information on region and outlet is
potentially available (which would enable
disaggregation to region and reflection of
consumer substitution across outlets).
3. Full coverage of prices across all
products, without corresponding quantities.
Limited or no information about price123
Without quantities, changes in the
expenditure shares of different products is
not able to be reflected and consumer
Retail transaction data
determining characteristics at the product
level.
substitution across items and products over
time cannot be adequately reflected. Some
external source of weighting, such as from
a household expenditure survey needs to
be incorporated but this will be at discrete
intervals so substitution between these
points in time is not reflected. Also, the
weighting will be at the item level or higher
due to the limitations of the survey data.
4. Sample of items, with full coverage of
product prices and quantities across all
points in time within those items.
Consumer substitution across products
within items is reflected, but consumer
substitution across items (eg if consumers
move towards buying savoury biscuits
rather than chocolate biscuits in response
to their lower price movements) is not
necessarily able to reflected accurately due
to partial coverage of substitutable items.
5. Prices and quantities at all points in time
for a sample of products (for example,
those in the current CPI basket)
There is improved measurement of price
change for the sample of products, as all
transactions of the products are included,
including those at discounted prices.
Consumer substitution across products
cannot be reflected accurately due to partial
coverage of substitutable products, and
there will be subjective judgement required
when replacing products that are
discontinued or at periodic reviews to
maintain representativeness.
Monitoring processes will be very important. Although there are good reasons to set up
scanner data production systems to be as automated as possible, for reasons of
efficiency for example, there will need to be good procedures for checking outliers,
consistency of classifications over time, and stability against past results. These kinds of
processes are already in place for the CPI ‘used cars’ system, which uses a hedonic
regression approach on a rolling window of data, so aspects of that process can be
incorporated.
5.10 Conclusion
For both consumer electronics and supermarket products, implementing scanner data in
production for the New Zealand CPI appears to be feasible. By using scanner data we
could improve accuracy (through improving coverage of transactions and making use of
quantity information) and reduce fieldwork resources by making use of existing privatelyheld administrative data. We would also have the potential to increase the frequency of
price measurement for consumer electronics products from quarterly to monthly.
We have established that the ITRYGEKS(TD) method could be used to estimate indexes
for consumer electronics in such a way that the implicit price changes of new and
disappearing products are incorporated appropriately.
For supermarket data, where characteristics information is lacking, the RYTPD method is
likely to produce better estimates of price change than the RYGEKS method, and
research is on-going to establish the theoretical properties of this method.
124
Retail transaction data
References
de Haan, J, & van der Grient, HA (2011). Eliminating chain drift in price indexes based on
scanner data. Journal of Econometrics, 161(1), 36–46.
de Haan, J, & Krsinich, F (2012, November). Scanner data and the treatment of quality
change in rolling year GEKS price indexes. Paper presented at the Economic Measurement
Group Workshop, Sydney, Australia. Available from www.asb.unsw.edu.au.
Ivancic, L, Diewert, WE, & Fox, KJ (2011). Scanner data, time aggregation and the
construction of price indexes. Journal of Econometrics, 161(1), 24–35.
Johansen, I, & Nygaard, R (2011, May). Dealing with bias in the Norwegian superlative
price index of food and non-alcoholic beverages. Paper presented at the twelfth meeting of
the International Working Group on Price Indices, Wellington, New Zealand. Available
from www.ottawagroup.org.
Krsinich, F (2011a, May). Price indexes from scanner data: A comparison of different
methods. Paper presented at the twelfth meeting of the International Working Group on
Price Indices, Wellington, New Zealand. Available from www.ottawagroup.org.
Krsinich, F (2011b, May). Measuring the price movements of used cars and residential rents
in the New Zealand consumers price index. Paper presented at the twelfth meeting of the
International Working Group on Price Indices, Wellington, New Zealand. Available at
www.ottawagroup.org.
Krsinich, F (2012). A fresh look at patterns in gadget sales. Economic News, Statistics New
Zealand, April 2012. Available from www.stats.govt.nz.
125
Retail transaction data
Appendix 5a: SAS code for scanner data methods
Over the past four years a SAS system to run all the different scanner data index
methods has been developed. This was used to produce results for the research to date,
and will likely form the basis for a future production system for scanner data at Statistics
NZ.
The methods that are currently incorporated are:
• a range of bilateral index methods (Laspeyres, Paasche, Fisher, Walsh, and
Törnqvist) – both direct and chained
• annually chained indexes, with ‘annually smoothed’ expenditure shares that use the
monthly prices in conjunction with the year-to-date’s quantities (or, alternatively,
expenditures) based on the above range of bilateral indexes
• RYGEKS – a rolling year GEKS based on any of the (superlative) bilateral indexes
above (the window length can be specified by the user)
• RYTD – a rolling year time dummy hedonic index (ie a hedonic regression with
characteristics of the products explicitly incorporated). The index is estimated on a
rolling window (whose length can be specified) with the most-recent movement
linked to the index at the previous period
• RYTPD – a rolling year time product dummy index that uses a fixed-effects
'hedonic' regression method where the product identifiers (ie barcodes) are the
'characteristics' controlled for. The index is estimated on a rolling window (length
can be specified) with the most recent movement linked to the index at the previous
period
• ITRYGEKS(TD) – imputation Törnqvist RYGEKS, where the inputs to RYGEKS are
bilateral indexes from a time dummy hedonic regression.
Note that an updated linking approach is being incorporated, which will better combine
the movements of new products in the RYTPD methods.
Versions of this code (with accompanying documentation) were shared with Statistics
Netherlands, the Australian Bureau of Statistics, Statistics Israel, and Statistics Belgium.
126
Retail transaction data
Appendix 5b: Abstract for 2013 Ottawa Group
Abstract accepted for 13th Ottawa Group meeting in Copenhagen, May 2013.
Using the rolling year time product dummy method for quality adjustment in the
case of unobserved characteristics
Frances Krsinich, Statistics New Zealand
This paper will discuss the use of the rolling year time product dummy (RYTPD) method
in situations where characteristics are unavailable for explicitly incorporating into hedonic
regression models. This builds on results from two earlier pieces of work. Krsinich (2011)
used a fixed-effects hedonic model (ie a pooled TPD method) to benchmark the current
matched-model approach to estimating the rental index for New Zealand, where we have
few observed characteristics in the longitudinal rental data. More recently, de Haan and
Krsinich (2012) extended the RYGEKS method to impute price movements for new and
disappearing items using hedonic regression. Results from applying this 'imputation
Törnqvist' (IT) RYGEKS to New Zealand consumer electronics scanner data were
compared with a range of other methods. The RYTPD was the best performing of those
methods that do not explicitly incorporate characteristics. We will argue that the RYTPD
is a viable method in situations where characteristics are unobserved, such as
supermarket scanner data. Unlike matched-model approaches, the RYTPD is doing
some imputation for new and disappearing items, though in production a more
sophisticated linking approach would be required to deal with the lagged capturing of
price movements for new products.
127
6 Frequency of the CPI
6.1 Executive summary
This chapter examines the advantages, disadvantages, and possible options for
producing a monthly consumers price index (CPI). Statistics New Zealand currently
produces a quarterly CPI and a monthly food price index (FPI).
The CPI is a key component in monetary policy setting overseen by the Reserve Bank,
including the official cash rate deliberations that take place every six weeks. Other major
uses of the CPI include indexing government payments such as welfare benefits, and as
a general deflator of, for example, retail sales. The CPI is also viewed as the main
indicator of inflation by the media, commentators, and the general public.
New Zealand and Australia are the only member countries of the OECD that do not
produce a monthly CPI. Producing a monthly CPI would provide more timely snapshots of
inflation – to assist in monetary policy setting and to monitor household inflationary
pressures – and would also improve international comparability with OECD countries that
do produce monthly CPIs. However, a monthly CPI would be costlier to produce.
Prices for many items in the CPI, such as food and non-food groceries, are collected
monthly; fresh fruit and vegetable prices and petrol prices are collected weekly. Different
options exist to produce a monthly CPI. One option could require significant additional
resources to price-survey some items more frequently, and would result in an increased
respondent load, especially if the frequency of postal surveys increases. Another option
would not involve the collection of additional prices, but would spread the price collection
of items currently surveyed quarterly across the three months of each quarter. However,
this would still lead to additional costs.
Some users of official statistics in New Zealand have indicated a preference for a monthly
CPI. The ANZ Monthly Inflation Gauge indicates an interest in more timely inflation data.
This measure provides a directional signal of broad trends in non-tradable CPI inflation.
Other users have generally felt the available range of ‘short-term’ indicator data
(predominantly quarterly) is acceptable and that filling known gaps in official statistics
would have a higher priority than a monthly CPI.
6.2 Issues to consider
This chapter considers two options, and their costs, for producing a monthly CPI and
compares them with the cost of producing the current CPI.
The current CPI costs around $2 million a year (including overheads) to compile, maintain
relevance, and publish. Issues to consider are:
• costs and benefits of producing a monthly CPI
• two options for producing a monthly CPI:
o one where goods and services (items) currently surveyed quarterly are pricesurveyed monthly – estimated annual cost of $3.3 million to compile and
maintain relevance
o a second where the pricing of the items currently surveyed quarterly is spread
over the quarter – estimated annual cost of $2.6 million to compile and maintain
relevance.
Significant parts of the all groups CPI are already surveyed monthly (or weekly in some
cases) but not all this information is published more than quarterly. The food price index
(FPI) is published monthly and contributes almost one-fifth of the weight of the all groups
128
Frequency of the CPI
CPI. Food prices are collected monthly except for fresh fruit and vegetables, which are
surveyed weekly.
Motor fuel prices are surveyed weekly, however, fuel discount information is only
collected quarterly. Other items price-surveyed monthly include alcohol, cigarettes and
tobacco, international air fares, rental car hire, and a variety of non-food grocery items
(eg cleaners, toiletries, over-the-counter pharmaceuticals, paper products, pet food, and
other miscellaneous household consumables). Together these items contribute about a
further 12 percent to the CPI basket. Price movements for these items are currently
published only quarterly. Options exist to publish index series for categories of CPI items
that are collected on a monthly basis. However, these items do not fit together neatly
across the classification. Table 6.4 gives some options for publishing CPI component
series monthly.
In moving towards a monthly CPI, there are potential trade-offs to offset the increased
cost. These include reducing the frequency of some price surveys (eg fresh fruit and
vegetables – currently surveyed weekly), reducing the coverage of urban areas (15
currently), or surveying fewer prices in smaller urban areas (as is done in some other
countries; eg the United States).
The committee and the CPI user community are asked to consider what priority should be
given to producing a monthly CPI or to publishing more of the currently available prices
monthly. If the committee and the CPI user community consider producing a monthly CPI
is a priority, they are asked to suggest which option they prefer and what, if any, tradeoffs should be considered.
6.3 2004 CPI Revision Advisory Committee
The 2004 CPI Revision Advisory Committee made two recommendations that relate to
the production of a monthly CPI.
Recommendation 19: Statistics New Zealand should continue to produce the
Food Price Index on a monthly cycle. To be consistent with the CPI All Groups
index, all prices in the Food Price Index should be seasonally unadjusted.
Recommendation 20: At this point in time, given the balance of users’
requirements and Statistics New Zealand’s resource requirements, the production
of a monthly CPI is a relatively low priority.
6.4 A monthly CPI
New Zealand and Australia are the only OECD countries that do not publish a monthly
CPI. New Zealand does publish a monthly FPI.
6.4.1 Potential benefits of producing a monthly CPI
A more-timely CPI may be useful for monetary policy oversight. Official cash rate (OCR)
announcements are made every six weeks by the Reserve Bank – the CPI is a key
component in deliberations on the OCR and for setting monetary policy objectives.
International comparability would be enhanced, particularly with OECD countries that
publish a monthly CPI. A monthly CPI was considered during a review of the Australian
CPI in 2010. Nearly half of all submissions received in the consultation phase addressed
the issue of how often the CPI should be released. Demand for a monthly CPI was
strongest from organisations with an interest in economic measurement and analysis, or
the financial sector. A clear message from users of CPI data was that the Australian
Bureau of Statistics (ABS) should only consider a monthly CPI if the quality of the index
would not be compromised, and the monthly CPI was of the same quality as the current
129
Frequency of the CPI
quarterly CPI. The review recommended a monthly CPI, subject to additional resources
being gained. To date, no additional resources have been provided for this purpose.
Complying with International Monetary Fund special data dissemination standards
(SDDS), the guidelines for compiling statistics for countries that access international
capital markets, would be easier. The standards recommend member countries publish a
monthly CPI. The SDDS is a global benchmark for disseminating macroeconomic data to
the public. SDDS subscription indicates that the country meets a test of ‘good statistical
citizenship’. The standards are important because they help enhance the availability of
timely and comprehensive statistics, which contribute to sound macroeconomic policies
and efficient functioning of financial markets.
See IMF standards for data dissemination for more information on SDDS.
New Zealand has not subscribed to the SDDS because it does not produce all the
statistics required for membership – the constraints have been the lack of: (a) a monthly
CPI, and (b) a monthly industrial production index (or similar) intended as a ‘tracking
indicator’ of quarterly GDP.
The CPI is viewed as the main indicator of inflation by the media, commentators, and the
general public. Publishing a monthly CPI would allow for more-timely monitoring of
household inflationary pressures.
Several items are already price-surveyed at least monthly, including fresh fruit and
vegetables and petrol (surveyed weekly), food, and non-food groceries, alcoholic
beverages, cigarettes and tobacco, and international air fares.
6.4.2 Benefits of continuing to produce a quarterly CPI
New Zealand produces very few monthly economic statistics – most are quarterly.
Timeliness improvements have focused on bringing forward the release dates of these
publications rather than on increasing the frequency.
For the Retail Trade Survey, a more timely administrative data source is used as a
monthly retail indicator. The monthly Retail Trade Survey was reduced to a quarterly
frequency in November 2010, following user consultation, to reduce survey costs and
respondent load. An alternative monthly retail indicator series is available, through
electronic card transactions. This data is from an administrative source in the banking
industry, is more timely, and has been published since January 2007.
Several official statistics users have suggested the available range of short-term indicator
data (predominantly quarterly) is acceptable. When faced with prioritising statistical
developments, they emphasise that New Zealand has significant gaps in macroeconomic
statistics. For example, balance sheets by institutional sector, or environment accounts
are seen as a higher priority than a monthly CPI.
Potentially there is not much to be gained from price-surveying all items monthly. The
current frequency copes with anticipated price change. Introducing more regular pricing
for quarterly items may introduce noise into the CPI price series rather than clarifying
price trends.
The additional cost of developing, compiling, and maintaining the relevance of a monthly
CPI is considerable (depending on the option chosen).
A full monthly CPI could involve a significant increase in respondent load, particularly if
postal surveys were more frequent. Respondent load is defined by the OECD as “The
effort, in terms of time and cost, required for respondents to provide satisfactory answers
to [a] survey.” Statistics NZ is committed to demonstrating best practice for managing
respondent load across the Official Statistics System.
130
Frequency of the CPI
6.5 International practice
6.5.1 Canada
Canada’s CPI is published every month – three to four weeks after the end of the
reference month.
Prices of most goods and services surveyed for the CPI are collected monthly, usually in
the first three weeks of the reference month. Petrol prices are collected weekly.
Most food items are collected every month. However, for some goods and services prices
are collected less frequently, since they change less often. University tuition fees,
property taxes, and automobile registration fees are collected once a year.
Over 90 percent of the price quotes used in constructing the CPI are collected by
personal visits to selected retail outlets. For other items, prices are collected by
telephone, with personal visits at least once a year.
Data from other Statistics Canada surveys are also used in the CPI. For example, the
new house price index (NHPI) is used in their measure of owner-occupied housing (a
user-cost approach). Some prices are collected from the Internet and other administrative
sources.
For definitions, data sources, and methods, see Consumers Price Index
6.5.2 United Kingdom
The UK’s CPI is published every month – two to three weeks after the end of the
reference month.
The UK price collection is split between 'local' and 'central' collection. Local collectors
collect prices every month, except for seasonal items (when they are not in-season), and
periodically. Normally collectors must visit the outlet, but prices for some items may be
collected by telephone.
Central collection is used for items where all the prices can be collected centrally with no
field work. Where feasible, price data is collected over the Internet. If this is not possible,
it is collected from one central source (eg trade association or government department).
Data may be requested in writing, by telephone, or by email, or may come automatically
because the Office for National Statistics (ONS) is on a provider’s mailing list.
See Consumer Price Indices Technical Manual, 2012
In early 2012, the ONS began Eurostat-funded research on using scanner data for
producing multi-purpose consumer price statistics.
6.5.3 United States of America
The United States CPI is published monthly, and is generally released within three weeks
of the end of the reference month.
All goods and services in the CPI are price-surveyed monthly in the country’s three
largest regions – New York, Los Angeles, and Chicago.
Only a select list of items is price-surveyed monthly, across all regions. Items priced
monthly are typically those with more variable and volatile price movements (eg food,
cigarettes and tobacco, insurance, electricity, telephone services, and motor fuel).
Prices for other items in regions outside the three major cities are collected every second
month, with the collection being spread over odd and even months.
131
Frequency of the CPI
Most item prices are collected by price collectors, by either field visit or telephone.
See Consumer Price Index, (chapter 17 from BLS Handbook of methods) for more
details.
6.6 Cost estimate for the current quarterly CPI
Cost estimates have been developed for running a monthly CPI. The cost of the current
quarterly CPI is provided for comparison.
The cost of producing the current quarterly CPI and monthly FPI was estimated at just
over $2 million for the year ended June 2012 (including overheads). This is an activitybased costing, using the full-time equivalent staff members in each work area who
contribute to producing the CPI. It covers all costs in compiling, maintaining the relevance
(includes rolling and three-yearly reviews), and publishing the CPI.
6.7 Current pricing frequency
The monthly FPI uses about 22,000 individual prices collected from retailers on average
every month. Approximately 3,000 fruit and vegetable prices are collected weekly (about
12,000 monthly). Nearly 10,000 other food prices are collected monthly.
The more volatile an item’s price is, the more frequently its price needs to be surveyed –
to reflect its price behaviour. Fruit and vegetable prices tend to change frequently so
these prices are collected weekly. Food grocery items that exhibit less price volatility are
surveyed once a month.
Approximately 120,000 individual prices are collected to calculate the quarterly CPI.
Prices for non-food items are collected on various frequencies. For example, motor fuel
prices are collected weekly, given their relative price volatility and relative importance in
the index.
Non-food supermarket items are surveyed monthly. Durable items (eg furniture) are
surveyed quarterly, again reflecting the relative volatility of their prices. Most services are
surveyed quarterly (eg real estate agents and solicitors’ fees).
Some other non-food items are surveyed twice a year, to best capture their price
movements during the main periods of demand. Examples are garden and outdoor
furniture, barbeques (November and February), and firewood (May and August).
Some non-food items are surveyed only once a year, according to when they are mainly
available – for example, summer clothing (October or November), winter clothing (April or
May), or school uniforms (February).
Other prices change only once a year, so are surveyed annually – for example, school
and university fees, and sports club subscriptions.
6.8 Monthly option – full monthly CPI collection
A full monthly price collection, where all items currently priced quarterly are surveyed
monthly, would measure monthly price change and improve quarterly and annual
estimates of price change. Substantially more price quotes would be obtained from a
monthly collection and price movements would potentially be identified sooner than under
the current quarterly collection. However, a disadvantage of a monthly collection is a
potential increase in non-response for postal surveys conducted monthly, given a shorter
turn-around time being available. This may require price estimation (imputation) for any
missing prices.
132
Frequency of the CPI
Data from an activity-based costing of the CPI and counts of price-surveyed items were
used to estimate indicative costs of a full monthly CPI. This estimate considers the
running costs of the CPI and includes an estimate for the cost of three-yearly reviews and
the rolling review of field outlets. It does not include any estimate to redevelop processes
and systems to establish a monthly CPI, which were calculated separately.
6.8.1 Assumptions
The following assumptions were made in calculating the cost for a full monthly CPI:
• A monthly CPI would be published by the end of the month following the reference
month.
• The current modes of collection for each type of item will be maintained.
• Items surveyed directly from retail establishments by price collectors on a quarterly
frequency would be price-surveyed every month.
• The current frequency of weekly and monthly pricing remains the same (eg fresh
fruit and vegetables and motor fuel would still be price-surveyed weekly).
Items currently surveyed by postal collection on a quarterly basis would be collected
monthly. Although it takes up to six weeks to send, return, and process an acceptable
number of quarterly postal responses – with a survey date of the 15th of the middle
month of the quarter – many responses are returned and processed more quickly than
this.
To meet the publication deadline under this scenario the survey date would probably
need to move forward – possibly to the 8th of the month (from the 15th). The postal
surveys would need additional resources to turn questionnaires around and process data
within a monthly window. Statistics NZ has operated other monthly postal surveys, such
as the Retail Trade Survey in previous years and the Accommodation Survey (and some
monthly CPI surveys). Both surveys had a publication date six weeks after the reference
month; lower response rates were achieved than for the quarterly CPI postal surveys.
However, these surveys collected data for the whole month and were not sent out until
after the end of the reference month.
One significant drawback with the postal survey is that postal price surveys have
observably higher response loads than price collection carried out by other collection
modes, when price collectors visit retail stores – because the respondent must do all the
work to provide the prices themself. Related to this increase in load is a risk of lower
response rates, which would increase the need to impute for missing prices. The
response load has been estimated to be 54 minutes per respondent per year (2011) or
about 2,700 hours in total. This would treble under the full monthly scenario to about
8,100 hours. By comparison this would be less than the total annual load for the monthly
Accommodation Survey (12,200 hours) and the annual Agricultural Production Survey
(16,400 hours), but significantly more than the quarterly Retail Trade Survey (3,200
hours) and the Quarterly Employment Survey (6,000 hours).
Items currently price-surveyed on an annual or biannual basis would not necessarily need
to be surveyed more frequently under a full monthly scenario. They are surveyed less
frequently for reasons of availability and consumer demand (eg summer and winter
clothing, and firewood) or because their prices change only once a year at a set date (eg
sports club subscriptions and school or university fees). There would be little to gain, in
price measurement terms, from surveying these items more frequently. For estimating
costs, it has been assumed that the biannual and annual price-surveyed items will stay
on the same frequencies as now. Note: More use is being made of the Internet to source
price data (eg e-book readers, international air fares). Just under 3 percent of the CPI
prices collected each quarter are currently collected over the Internet. Research is taking
place into developing an automatic price collection tool to collect prices from the Internet
– referred to as ‘price scraping’.
133
Frequency of the CPI
6.8.2 Estimated costs for a full monthly CPI collection
Moving to a full monthly CPI with the assumptions above would involve significantly
higher field, processing, and analysis costs. The ongoing annual cost of a full monthly
CPI is estimated to rise by about 65 percent, compared with the current CPI, to around
$3.3 million (including overheads). The number of prices surveyed would increase by
around 38 percent under this option.
The current field cost per item for quarterly items surveyed is several times that of weekly
and monthly items. Quarterly priced items are often more complex in nature and require
greater attention to specification details; for example, household appliance items where
details on a number of features/characteristics are collected. In addition, the pricing
outlets are more geographically spread, with fewer items being collected per outlet than
for items collected monthly. Having to price-survey quarterly items on a monthly basis
would add significantly to current field collection wage and travel costs – field collection
wage costs would double and travel costs would rise about 50 percent.
A significantly greater number of price quotes would need to be processed by office staff.
However, 12 releases would be published each year instead of 16 at present (12 FPI and
four CPI), although more time goes into the analysis of results for the quarterly releases,
given they include a much wider range of items. The postal collection would also need to
be turned around more quickly. It is estimated the number of office staff required would
need to increase by nearly 50 percent.
A significant issue with increasing the frequency of postal surveys is increasing
respondent load and the greater possibility of lower response rates, which may affect
data quality.
6.9 Monthly option – spread pricing for items currently
collected quarterly
A cost-saving alternative, relative to the full monthly option, is to not price-survey all the
items currently collected quarterly every month. This could involve collecting the same
total number of price quotes as the current quarterly price collection, but spreading this
collection over the three months of the quarter.
Even though the same number of price quotes would be surveyed under this scenario,
potentially some price changes could be identified sooner and therefore included in
published index numbers sooner. However, some price changes could also be identified
later than is currently the case.
134
Frequency of the CPI
Figure 6.1
Example of price collection under spread pricing option
March quarter
Current quarterly pricing –
15th of middle month of the
quarter
Jan
15 Feb
Mar
Apr
15 May
Outlet 1
Outlet 1
Outlet 2
Outlet 2
Outlet 3
Outlet 3
Item price change 20 Feb
Price survey spread over
quarter. One-third of outlets
price-surveyed – at 15th of
each month
June quarter
Jun
Item price change 10 May
15 Jan
15 Feb
15 Mar
15 Apr
15 May
15 June
Outlet 1
Outlet 2
Outlet 3
Outlet 1
Outlet 2
Outlet 3
Quarterly prices are currently surveyed around the 15th of the middle month of the
quarter (eg 15 February for the March quarter and 15 May for the June quarter). Under
this pricing frequency, if the price of an item surveyed in an outlet changed on the 20th of
February it would not be identified until 15 May, and not included in published numbers
until around mid-July.
Under the spread-pricing option, price collection from outlets for a product category would
be spread over the three months of the quarter. For example, three outlets for a product
category in an urban area would each be allocated to a different month in the quarter –
for the March quarter outlet 1 would be priced in January, outlet 2 in February, and outlet
3 in March. Each would be priced again in April, May, and June, respectively, during the
June quarter. Prices would be surveyed on the 15th of the month.
In some cases, price changes could be identified and published sooner under the spreadpricing option. For example, if the price of an item in outlet 1 changed on 20 February it
would be picked up by the survey in April (the next time outlet 1 is priced) and the price
change would be included in the April publication in May. If the price changed in outlet 2
on 20 February it would be picked up in May and published in the May month results in
June. If the price changed in outlet 3 on 20 February it would be picked up on 15 March
(the next time that outlet is priced). In all instances, the price changes are published
earlier than current publication for the June quarter – mid-July.
Conversely, a price change occurring close to the quarterly pricing date could be
identified more slowly under a spread-pricing option. For example, a price change on 10
May would be picked up on 15 May under a quarterly price survey and published in July.
Under the spread-pricing option, a price change in outlet 1 on 10 May would not be
picked up until that outlet was priced in July. The results would not be published until
August, a month later than now. A price change in outlet 2 on 10 May would be picked up
in May; a price change in outlet 3 on 10 May would be picked up in June.
On balance, price changes would be identified sooner than under the current quarterly
scenario. Under this option, the monthly CPI would be of lower quality (and potentially
more volatile) than the current quarterly CPI, but the quarterly series would be of
comparable or slightly higher quality.
135
Frequency of the CPI
A further justification for the spread-pricing approach is that quarterly items were
surveyed quarterly because their prices changed less frequently than items pricesurveyed weekly or monthly.
6.9.1 Assumptions
The following assumptions were made in calculating the costs for the monthly CPI option
– where the pricing of quarterly items is spread over the months of the quarter:
• A monthly CPI would be published by the end of the month following the reference
month.
• Items currently surveyed quarterly would be price-surveyed in a staggered way,
with a representative selection of one-third of the items being surveyed each
month.
• It should be more feasible to conduct a monthly postal survey, with only one-third of
prices involved each month, than the option where all prices are collected at least
monthly. More resources could go into following-up respondents who have not
returned their surveys. An advantage is that respondent load would be unaffected,
because the same number of prices would be collected as currently.
• An appropriate imputation method would need to be established to treat prices not
surveyed in the current month. Statistics Canada uses the most-recently collected
price in months when prices are not surveyed.
• Overhead costs are generally assumed to have a linear relationship with direct
costs. This may not be accurate as overheads may not increase until direct costs
reach a certain threshold. For example, a new full-time staffing allocation in
Finance or Human Resources would require a substantial increase in direct costs
across the organisation. The IT-related portion of overheads was held constant for
this reason.
6.9.2 Estimated costs – spread pricing for items currently collected
quarterly
The ongoing costs of this spread-pricing monthly CPI option are estimated to be about 30
percent higher than the current CPI – at around $2.6 million, including overheads. The
number of price quotes surveyed would not increase under this option.
Field wage costs would be higher than the current costs, even though the total number of
price quotes collected would remain the same. However, the cost would be significantly
less than under the option in which all prices currently collected on a quarterly basis were
collected monthly. Staggering the collection of quarterly field items is likely to create
inefficiencies compared with the current situation, especially in the time spent travelling to
retail outlets.
Travel costs (mileage) were estimated to increase 30 percent overall, allowing double the
time on travel for the quarterly items. The trips would be less efficient than the current
quarterly field price collection with the stores being similarly spread.
Office staff costs for processing and analysis are likely to be higher than the current
situation, although the number of price quotes remains the same. The full range of CPI
commodities would need to be quality assured and analysed each month under the
spread-pricing monthly option. It is expected that salary costs would increase 30 percent
under this option.
136
Frequency of the CPI
Table 6.1
Summary of the costs of the price-surveying options
Option
Total estimated cost including overheads
$(million)
Current CPI collection, quarterly
2.0
Full monthly collection – option
3.3
Spread current price collection, monthly
– option
2.6
Table 6.2
Summary of price-surveying options – implications
Option
Implications
Current CPI
(quarterly)
Price collection:
• Total number of prices collected each quarter – 120,000
Full monthly
collection – option
Price collection:
• Total number of prices collected each quarter – 165,000
• All items currently price-surveyed quarterly would be surveyed
monthly.
Quality:
• Provides a monthly measure of consumer price inflation.
• Improved quarterly and annual estimates of price change.
Respondent load:
• Postal surveys previously run quarterly would need to be run monthly.
• Respondent load would treble for postal surveys – meaning the survey
would have higher load than quarterly Retail Trade Survey and
Quarterly Employment Survey.
• Possible impact on quality of postal collection if response rates fall.
Spread current
price collection,
monthly – option
Price collection:
• Total number of prices collected each quarter – 120,000
• Price collection of items currently surveyed quarterly would be spread
over the months of the quarter.
137
Frequency of the CPI
Quality:
• The monthly CPI would be of lower quality and potentially more volatile
than the current quarterly CPI.
• Quarterly estimates would be of comparable or slightly higher quality.
Respondent load
• No increase in respondent load compared with current quarterly CPI.
6.10 Implementation costs
Estimates were made for the expected costs to implement each monthly CPI option.
These estimates consider the costs associated with consulting on the change (users and
survey respondents), and redeveloping questionnaires, systems, and processes. The
current systems already cope with monthly prices and publishing the monthly FPI, so are
not expected to require a significant overhaul. It has been assumed that the time to
change to a monthly CPI would include a six-month period where both quarterly and
monthly systems would run in parallel before only a monthly CPI was published.
The full-monthly option would require more resources for the parallel run, because of the
additional data. The spread-pricing monthly option may require more development work
to reduce the possibility of sampling biases being introduced into price measurement.
Table 6.3
Estimated implementation costs for monthly CPI options
Option
Estimated implementation cost, including
overheads $(000)
Full monthly collection – option
450
Spread current price collection, monthly
– option
400
6.11 Other considerations
Several items are already price-surveyed at least monthly but only the FPI is published
monthly. Publishing more of these price series on a monthly frequency could also be
considered.
138
Frequency of the CPI
Table 6.4
Current CPI categories that could potentially be published monthly
By group, subgroup, or class
Group, subgroup, or class
Weight
(percent)(1)
Pricing
Frequency
Weight
of items
priced
monthly
or
weekly(1)
(percent)
Comment
Complete groups, subgroups, or classes that are published monthly
Food group
18.79
Monthly/weekly
18.79
Published monthly
2.48
Weekly/monthly
2.48
Published monthly
Fruit
1.01
Weekly/monthly
1.01
Published monthly
Vegetables
1.46
Weekly/monthly
1.46
Published monthly
3.03
Monthly
3.03
Published monthly
Meat and poultry
2.57
Monthly
2.57
Published monthly
Fish and other seafood
0.46
Monthly
0.46
Published monthly
7.20
Monthly
7.20
Published monthly
Bread and cereals
2.10
Monthly
2.10
Published monthly
Milk, cheese, and eggs
1.91
Monthly
1.91
Published monthly
Oils and fats
0.35
Monthly
0.35
Published monthly
Food additives and
condiments
0.54
Monthly
0.54
Published monthly
Confectionery, nuts, and
snacks
1.68
Monthly
1.68
Published monthly
0.62
Monthly
0.62
Published monthly
2.12
Monthly
2.12
Published monthly
Coffee, tea, and other hot
drinks
0.38
Monthly
0.38
Published monthly
Soft drinks, waters, and
juices
1.74
Monthly
1.74
Published monthly
Restaurant meals and readyto-eat food
3.96
Monthly
3.96
Published monthly
Restaurant meals
1.45
Monthly
1.45
Published monthly
Ready-to-eat food
2.51
Monthly
2.51
Published monthly
Fruit and vegetables
Meat, poultry, and fish
Grocery food
Other grocery food
Non-alcoholic beverages
Complete groups, subgroups, or classes that could be published monthly
Alcoholic beverages and
tobacco group
6.91
139
Monthly
6.91
Published
quarterly
Frequency of the CPI
Alcoholic beverages
4.79
Monthly
4.79
Published
quarterly
Beer
1.98
Monthly
1.98
Published
quarterly
Wine
1.48
Monthly
1.48
Published
quarterly
Spirits and liqueurs
1.32
Monthly
1.32
Published
quarterly
Cigarettes and tobacco
2.13
Monthly
2.13
Published
quarterly
Cleaning products and other
household supplies
0.66
Monthly
0.66
Published
quarterly
International air transport
1.59
Monthly
1.59
Published
quarterly
Pet-related products
0.63
Monthly
0.63
Published
quarterly
Classes that could be published monthly
Gas
0.43
Partial
0.13
Items priced
monthly: bottled
gas.
Small tools and accessories
for house and garden
0.29
Partial
0.16
Items priced
monthly: light
bulbs, batteries.
Pharmaceutical products
0.62
Partial
0.20
Items priced at
supermarkets
monthly:
painkillers,
antacids, cough
liquid, throat
lozenges,
sunscreen,
vitamins.
Other medical products
0.04
Partial
0.02
Items priced at
supermarkets
monthly: plasters,
condoms.
Road passenger transport
0.42
Partial
0.07
Items priced
monthly: car
rental services.
Newspapers and magazines
0.56
Partial
0.35
Items priced
monthly: daily
newspapers,
Sunday
newspapers.
0.27
Partial
0.01
Items priced at
supermarkets
monthly: student
binder refills,
cellulose tape.
Stationery and drawing
materials
140
Frequency of the CPI
Other appliances, articles, and
products for personal care
1.55
Total price-surveyed at least
monthly
32.67
Partial
1.13
Items priced at
supermarkets
monthly: razor
blades,
toothbrushes,
toilet soap,
shower gel,
toothpaste, hair
conditioner,
shampoo, hair
colour,
deodorants,
moisturiser,
disposable babies'
nappies, paper
tissues, toilet
paper, sanitary
pads, tampons.
30.66
1. Weights are expressed as percentages at the June 2011 quarter.
Note: While petrol prices are surveyed weekly information on fuel discounts is received quarterly.
6.12 New developments
Retail transaction data is currently being investigated for use in compiling the CPI. This
may provide an opportunity to reduce field collection costs, especially for the monthly
supermarket items. However, this approach could have offsetting costs; for example,
additional processing and analysis may be required. Also, retail transaction data may not
be more-timely than survey data.
Electronic questionnaires and email may provide more-timely options than the current
postal forms. More use is also being made of the Internet to source price data; for
example, international air fares.
141
7 Seasonality in the CPI
7.1 Executive summary
This chapter outlines the history of the treatment of seasonal items in the CPI and
considers whether there is a need for a fully seasonally adjusted CPI for analytical
purposes.
The consumers price index (CPI) is made up of a range of goods and services (items),
some of which are seasonal in nature. Seasonal items are those that are not available in
the market place for the whole year round, or if available, they have regular fluctuations in
their quantities and/or prices. The appropriate treatment of seasonal commodities in a
CPI is an area where a variety of methods are used by international statistical agencies.
The current CPI headline measure is published without any seasonal adjustment. This
has been since the September 2006 quarter. Before then, price movements for the fresh
fruit and vegetable items within the CPI were explicitly seasonally adjusted. However,
other items known to exhibit seasonality in their price behaviour were not seasonally
adjusted.
The change in 2006 followed a recommendation of the 2004 CPI Revision Advisory
Committee – that Statistics New Zealand adopt a consistent treatment of seasonality in
the CPI and, in particular, that all prices in the all groups CPI index should be seasonally
unadjusted. The committee considered that seasonal adjustment was sound in concept.
However, given such a measure would be subject to revision it could not be used as the
headline CPI measure. For example, because the CPI is used to adjust welfare benefit
rates, it would not be suitable to have a measure that was subject to revision after
publication. The 2004 committee recommended Statistics NZ should consider producing
a seasonally adjusted all groups CPI index as an analytical series. However, the
committee attached a relatively low priority to producing this index.
Since the change in 2006, it is noticeable that seasonal commodities often make a
contribution to the quarterly movements in the CPI (and monthly movements in the food
price index (FPI)).
7.2 Issue to consider
The committee and CPI user community are asked to consider:
• What priority should be given to producing a fully seasonally adjusted CPI for
analytical purposes?
Given a seasonally adjusted CPI would be subject to revision each time it is calculated it
would not be suitable to be used as the headline measure of inflation or for purposes
such as benefit adjustment. However, it would be useful as an analytical series to help
understand the impact of seasonal price changes on the headline unadjusted CPI.
Developing an analytical series would be consistent with the approach used by statistical
agencies in Australia, the United States, and Canada, which produce seasonally adjusted
CPIs for analytical purposes. Conversely, most European countries do not appear to
produce seasonally adjusted CPIs.
The cost of developing such a measure is not estimated to be large and would be
achievable within the existing prices statistical maintenance programme – although it
would need to take priority over other maintenance initiatives.
A seasonally adjusted CPI is likely to be developed indirectly, aggregated from lowerlevel seasonally adjusted and unadjusted series. The cost to develop the overall series is
142
Seasonality in the CPI
largely influenced by the number of seasonally adjusted lower-level series needing to be
developed.
7.3 Introduction
The chapter covers:
• recommendations of the 2004 CPI Revision Advisory Committee
• history of seasonal adjustment in the New Zealand CPI
• the use of seasonal baskets
• a fully seasonally adjusted CPI for analytical purposes.
7.4 Recommendations of the 2004 CPI Revision
Advisory Committee
The 2004 CPI Revision Advisory Committee made three recommendations that
specifically relate to seasonal adjustment.
Recommendation 17: Statistics New Zealand should adopt a consistent treatment
of seasonality in the CPI and, in particular, all prices in the CPI all groups index
should be seasonally unadjusted. The committee recognises that the alternative of
a fully seasonally adjusted index would require subsequent revisions, and that such
revisions are unacceptable given the uses of the CPI. The committee also
recognises there is likely to be some short-term disruption to annual movements
during the year-long transition to a fully unadjusted CPI.
Recommendation 18: Statistics New Zealand should consider producing a
seasonally adjusted CPI all groups index as an analytical series. However, the
committee attaches a relatively low priority to the production of such an index.
Recommendation 19: Statistics New Zealand should continue to produce the FPI
on a monthly cycle. To be consistent with the CPI all groups index, all prices in the
FPI should be seasonally unadjusted.
The 2004 CPI Revision Advisory Committee recommended that Statistics NZ adopt a
consistent treatment of seasonality in the CPI and, in particular, that all prices in the CPI
should be seasonally unadjusted.
The committee was concerned about the inconsistency of the previous treatment of
seasonal items in the CPI – only fresh fruit and vegetables were seasonally adjusted
while other seasonal items in the index were not. Furthermore, an unadjusted measure
would have the advantage of being more transparent, and would reflect the actual price
movements experienced by consumers. It was also noted that most users would focus on
annual movements in the CPI, for which a seasonally adjusted measure would be similar
to an unadjusted measure. It was noted that this would bring Statistics NZ’s practices in
line with the approach of the Australian Bureau of Statistics (at the time). The committee
also recommended that the monthly FPI should be a seasonally unadjusted measure, to
be consistent with the rest of the CPI.
The committee considered the concept of seasonal adjustment was sound. However,
given such a measure would be subject to revision it could not be used as the headline
CPI. Because the CPI is used to adjust welfare benefit rates, it would not be appropriate
to have a measure that was subject to revision after publication. The committee
recommended that Statistics NZ should consider producing a seasonally adjusted all
groups CPI index as an analytical series. However, the committee attached a relatively
low priority to producing such an index. It ranked the recommendation fifth of five ‘nonessential’ recommendations.
143
Seasonality in the CPI
The CPI became a fully seasonally unadjusted series from the September 2006 quarter,
with the FPI becoming fully seasonally unadjusted from the July 2006 month.
Since the change in 2006 it is noticeable that seasonal commodities often make a
contribution to the quarterly movements in the CPI (and monthly movements in the FPI).
For example, in the September 2012 quarter seasonally higher vegetable prices
(tomatoes, lettuce, and capsicums) were notable contributors to the 0.3 percent increase
in the CPI. The reverse was true for the December 2012 quarter, with seasonally lower
vegetable prices for these same commodities being significant contributors to a 0.2
percent fall in the CPI. Vegetable prices normally rise in the June quarter and peak in the
September quarter, before falling in the December and March quarters.
Statistics NZ would like the committee and the CPI user community to consider whether
Statistics NZ should produce an analytical, fully seasonally adjusted CPI, and, if so, what
priority should be given to such an initiative.
7.5 History of the treatment of seasonality in CPI
The CPI is made up of a range of goods and services (items), some of which are
seasonal in nature. Seasonal items are those unavailable in the market place for the
whole year, or if available, they have regular fluctuations in their quantities and/or prices.
The appropriate treatment of seasonal commodities in a CPI is an area where overseas
statistical agencies use a variety of methods.
No explicit seasonal adjustment is currently undertaken in New Zealand’s CPI. However,
before the September 2006 quarter, fresh fruit and vegetables were seasonally adjusted,
the history of which is discussed below.
7.5.1 Seasonal quantities
Seasonal variations in the quantity purchased may sometimes be accompanied by
seasonal fluctuations in price, but this is not always the case. The seasonality in
quantities purchased should be distinguished from the seasonality of price series. For
example, when quantities of seasonal items increase the prices often decreases.
A number of items in New Zealand’s CPI are subject to seasonal fluctuations, whether
due to supply or demand. Fresh fruit and vegetables are prime examples. Other
examples include summer and winter clothing and footwear, electricity, firewood,
international air fares, and accommodation services.
Two broad methods are used to calculate price indexes for goods and services where the
quantities purchased follow a seasonal pattern. These are to use fixed or variable
baskets (weights).
7.5.2 Fixed expenditure weights
CPI expenditure weights for commodities are based on average household expenditure in
the weight reference period, typically a year. No account is taken of the fact that
quantities of many commodities vary during the year.
The total expenditure in the reference year is used to derive an expenditure weight,
where the underlying quantity weights are fixed throughout the year. The treatment of
seasonal commodities is therefore consistent with the treatment of all other commodities.
A possible disadvantage of using this method for seasonal commodities is that fixed
weights for seasonal commodities do not necessarily reflect households’ actual monthly
buying patterns. A commodity may have a smaller weight than implied by actual
purchases in months when it is in-season and a larger weight in months when it is out of
season.
144
Seasonality in the CPI
From the 1993 CPI review, a system of fixed quantity weights for each fresh fruit and
vegetable item was introduced to replace the former variable weights for these items.
In recent decades the availability of most fruit and vegetables during the year has
improved. Increased imports of fresh produce, more produce grown indoors in controlled
environments, and improved storage techniques have extended the availability period for
many fruit and vegetables. Accordingly, the fixed-weight model for fruit and vegetables
has become more relevant. However, as discussed earlier, seasonal commodities still
affect price movements in the CPI.
7.5.3 Variable weighted expenditure basket
Variable weighted baskets attempt to reflect the relative quantities of particular seasonal
items acquired each month (eg the relative amount of tomatoes, lettuces, capsicums, and
potatoes purchased each month).
A potential advantage of using variable weights is that they can better reflect the quantity
of items purchased by households in any given month. The influence of price changes
will be in direct proportion to their usual effect on household budgets at a particular time
of year. Using variable weights attempts to more closely reflect the actual experience of
households. Because the data used to derive variable expenditure weights can be
distorted by atypical events, more than one year’s data is generally required when
determining the variable weights – to reduce this possibility.
A weight of zero is assigned to commodities for the months in which they are not
available in significant quantities.
A disadvantage of variable expenditure weights is that the influence of a particular
commodity on the CPI may be difficult to interpret or explain. For example, the price of a
commodity may increase in a certain month, but this increase may be more than offset by
a decrease in its variable monthly weight and therefore it will contribute to a decrease in
the index. This may be difficult to explain to users.
Another disadvantage is that seasonal patterns may change and the variable baskets
become dated. Data used is based on information from a past period, which could be
affected by irregular events during that period. To lessen the effect data is usually based
on more than one year’s results, if available.
The CPI used variable expenditure weights for fresh fruit and vegetables from 1949 to
1993. Eggs were also included in the variable basket from 1949 until 1965.
The method of weighting involved specifying 12 individual monthly baskets, each
containing the mix of seasonal fresh fruit and vegetables that represented household
purchasing patterns in that particular month. The overall basket of fresh fruit and
vegetables had the same weight or relative importance in the CPI each month, while the
contents of each monthly basket could vary. The total weight of fresh fruit and vegetables
in each month was equal to the average expenditure on fresh fruit and vegetables for the
year.
The variable relative expenditure weights for individual fruit and vegetables were derived
from the Household Expenditure and Income Survey.
See appendix 7a for an example of the variable basket, from 1988.
7.5.4 Seasonal adjustment used for prices in the fixed and variable
baskets
Prices for seasonal fresh fruit and vegetable commodities used in the fixed-weighted
basket (1993–2006) were adjusted by factors, to remove regular seasonal price
fluctuations, before being used in the index. The factors used to remove seasonality from
145
Seasonality in the CPI
prices were calculated annually, to be used for the months of the following year. Each
item had a different factor for each month. The choice of which fresh fruit and vegetable
items to seasonally adjust was also reviewed annually. Some items had irregular
seasonal patterns and it was difficult for the seasonal adjustment program to remove
these (eg mandarins). For a small number of items (eg cauliflower and broccoli), there
was much irregularity in the price data and it was marginal as to whether the seasonal
adjustment would produce better results.
Prices were collected for all fresh fruit and vegetable items throughout the year, or for as
much of the year as the items were available in sufficient quantity and quality to be
purchased. The method used to impute missing prices was to continue to use the last
seasonally adjusted regional average price until the season started again (or the last
actual regional average price if the item could not be reliably seasonally adjusted). This
minimised any unusual movements in the index that were due to prices changing for
items going into and out of season.
The seasonal adjustment process ensured that regular seasonal fluctuations in prices
were eliminated from the index. Accordingly, the index was influenced only by price
changes that differed from the estimated normal seasonal pattern. From the September
2006 quarter, the method of seasonally adjusting fresh fruit and vegetable item prices
was discontinued, following a 2004 CPI Revision Advisory Committee recommendation.
With the variable expenditure baskets used before 1993, the base prices for seasonal
fruit and vegetable items used adjusted monthly prices. The result was called a ‘base
normal price’, which allowed for normal seasonal variation but excluded any irregular
variations that may have occurred in the source data used to calculate it. The exercise
intended to normalise the price, so a fair comparison could be made. For example, in the
index calculation, the currently surveyed price for tomatoes in January would be
compared with the seasonal January base normal price for tomatoes, which excluded any
irregular variations. A significant disadvantage with this method was that the seasonal
factors used to calculate the base normal prices were updated infrequently – at the time
of a periodic review. Seasonal factors could change significantly in the time between
these reviews. In addition, the seasonal pattern in a particular year being priced could be
quite different from the pattern reflected in the base prices, detrimentally affecting the
price comparison between the current and base (or reference) price in the index
calculation.
Analysis carried out by Statistics NZ at the time of the 1993 review showed that an index
calculated with fixed weights for fresh fruit and vegetables produced similar results to, but
had less volatility than, the previous index in which variable weights were used. This was
considered to be due to the stabilising effect of carrying prices forward while items were
out of season and having the same basket and weights each month.
7.5.5 Other commodities
No commodities in New Zealand’s CPI other than fresh fruit and vegetables have had
explicit seasonal treatment. However, Statistics NZ does collect prices of basket items to
reflect the timing of usual price changes (eg university fees are priced in the March
quarter). The total expenditure of all items in the weight reference period (year) is used to
derive an expenditure weight, where the underlying quantity weights are fixed throughout
the year.
Some items in the CPI are priced only when in-season (or in periods when most of
expenditure on them occurs). The price for such items is carried forward when the items
are out of season. Winter (April/May) and summer (October/November) clothing are
examples of this.
146
Seasonality in the CPI
7.6 Seasonal or variable baskets
Expenditure weights for the CPI are updated as part of the three-yearly CPI reviews,
using results from the Household Economic Survey (HES) and other sources. Annual
expenditure estimates, from the weight reference period (the period the HES relates to),
are used to calculate the CPI weights. Weights that represent the quantities of goods and
services acquired in the weight reference period generally remain fixed in the index until
the index is next reviewed.
Using weights that maintain fixed underlying quantities throughout a year can lead to
results that do not reflect households’ experience. This is because, using tomatoes as an
example, households tend to acquire relatively more tomatoes when they are in-season
and their prices are lower, than when they are out-of-season and prices are higher.
The fixed weighting approach can lead to price changes of particular goods and services
in the index being over-weighted when they are out-of-season, and under-weighted when
they are in-season.
One approach used in the past (1949–93) was to use variable (or seasonal)
baskets/weights, where items have different expenditure weights each quarter/month,
that reflect the actual quantities of the goods or services that households acquired in the
weight reference period. This approach has the potential to produce results that are
closer to the price change that households actually face.
However, when applied practically, this approach can also lead to inappropriate results.
This is because seasonal expenditure patterns in the weight reference (HES) period may
not match those in the years following. This would mean the seasonal weighting patterns
would not align with the seasonal patterns that households actually experience. It could
even accentuate price change, due to changing seasonal patterns or seasonal influences
that do not align with those experienced in the weight reference period. For example, if in
a year after new seasonal weights are introduced, summer arrives earlier than it did in the
weight reference period, the associated fall in tomato prices might be under-weighted
relative to the quantities that households are actually purchasing.
Using retail transaction data for price measurement could provide a better solution. The
methods being developed to use retail transaction data for price measurement take the
actual quantities of goods purchased into account in ‘real time’, avoiding the undesirable
situation where seasonal price increases may be over- or under-weighted relative to the
price change of other goods within the same expenditure category.
See the ‘Retail transaction data’ chapter for more details.
7.7 A fully seasonally adjusted CPI
7.7.1 Introduction
From the United States’ Fact sheet on seasonal adjustment in the CPI:
Seasonal adjustment removes the effects of recurring seasonal influences from
many economic series, including consumer prices. The adjustment process
quantifies seasonal patterns and then factors them out of the series to permit
analysis of non-seasonal price movements. Changing climatic conditions,
production cycles, model changeovers, holidays, and sales can cause seasonal
variation in prices. For example, tomatoes can be purchased year-round, but prices
are significantly higher in the winter months when the major sources of supply are
between harvests.
Time series can be split into trend, seasonal, and irregular components.
147
Seasonality in the CPI
The seasonal component of a time series is the repetitive and predictable movement of
the series around the trend line in one year or less. The seasonal component should be
reasonably stable in terms of annual timing, direction, and magnitude. Possible causes
include:
• natural factors (eg seasonal weather patterns)
• administrative measures (eg start and end dates of the school year)
• social/cultural/religious traditions (eg fixed holidays such as Christmas)
• length of the months (28, 29, 30, or 31 days) or quarters (90, 91, or 92 days).
Seasonal adjustment removes the seasonal component, permitting analysis of the nonseasonal price movements. The resulting seasonally adjusted series enables data for
adjacent months/quarters to be compared, free from regular seasonal influences.
Trend estimation removes the seasonal and irregular components. The irregular
component is unpredictable as regards timing, impact, and duration. It can arise, for
example, as a result of unseasonal weather, natural disasters, or strikes. Trend estimates
reveal the underlying direction of movement in a series and are used to identify turning
points.
Seasonally adjusted and trend figures are recalculated each period, for all time periods in
a series. If the actual observed data itself is revised, this results in changes to the trend
and seasonally adjusted series. Even if the source data is not revised, which is generally
the case with indexes such as the CPI, trend estimates towards the end of a series are
subject to revision as new data periods are added to a series. This is because the moving
average used in the body of the series is symmetric, using the same number of points
before and after the time point. But at the end of the series, where there is no future data,
the moving average used is asymmetric. As new data becomes available, data points not
yet in the body of the series have their trend re-estimated, using different and
decreasingly asymmetric moving averages. Revisions can be particularly large if an
observation is treated as an outlier in one quarter, but is found to be part of the underlying
trend when further observations are added to the series.
The seasonal component is estimated with moving averages, and is subject to revision as
new points are added to the series in the same way as the trend component. As with the
trend series, when new data comes to hand, the assessment of what is an outlier can
change. This occurs even for points earlier in the series. This causes revisions in the
seasonally adjusted series.
The next section discusses the results of an investgation into seasonally adjusting data
for the CPI expenditure groups and all groups CPI.
7.7.2 Methodology
Statistics NZ examined the seasonality of the CPI time series using the X-12-ARIMA
package and data from the June 2006 quarter to the September 2012 quarter. This
analysis was generally done at the group level with the exception of transport. This is the
direct method of seasonal adjustment – it involves seasonally adjusting each group series
directly. However, for the transport group the seasonal adjustment was performed at a
finer level. The international air fares component of this group was adjusted separately,
because while the air fares are known to be seasonal, the remainder of this group does
not have seasonality. The transport group as a whole tends to be dominated by changes
in motor fuel prices, which are not seasonal.
The X-12-ARIMA package was developed by the US Census Bureau. This package is
commonly used to seasonally adjust sub-annual time series published by statistical
agencies around the world, including most sub-annual outputs published by Statistics NZ.
The United States, Canada, and Australia use this package to seasonally adjust their
CPIs, to provide seasonally adjusted analytical series for users.
148
Seasonality in the CPI
Note, if Statistics NZ were to produce a seasonally adjusted analytical CPI series, further
investigation would be needed to determine the level of classification at which the
adjustments would take place. It would almost certainly be at a greater level of detail than
the group level.
7.7.3 Summary of results
The quarterly CPI has 11 commodity groups. The presence of seasonality was tested in
these groups as well as the all groups CPI and the monthly FPI.
The all groups CPI, directly adjusted, did not show seasonality. This may result from
seasonal fluctuations being offset when the unadjusted data are aggregated to the all
groups CPI.
Going below the all groups level, six of the 12 tested indexes exhibited some degree of
seasonality (see table 7.1). The quarterly CPI groups showing seasonality were food,
clothing and foodwear, housing and household utilities, recreation and culture, and
education.
The monthly FPI also showed identifiable seasonality.
The transport group did not display seasonality. However, within this group international
air fares (the international air transport class) are known to have a strong seasonal
pattern, so this class was separated from the rest of the transport group and seasonally
adjusted alone. Because price movements for the transport group are often dominated by
changes in motor fuel prices, which occur at irregular times, international air fares were
added to the analysis and adjusted separately.
The household and household utilities group exhibits the presence of seasonality
because it includes local authority rates, which appear to have a seasonal pattern. This is
due to local authority rates changing at a similar time each year. However, following
further analysis the housing and household utilities group was left unadjusted because
local authority rates change as a result of administrative decisions – change is not due to
regular seasonal effects. The education group, which is discussed later, was also left
unadjusted for a similar reason – it is affected by regular changes in fees, which imply
seasonality but are also the result of administrative decisions.
149
Seasonality in the CPI
Table 7.1
Seasonality and weight proportion in CPI groups
Weight
Group/Class
Seasonality
All groups CPI
Not present
100.0
Food (quarterly)
Present
18.8
Alcoholic beverages and tobacco
Not present
6.9
Clothing and footwear
Present
4.4
Housing and household utilities
Present(1)
23.6
Household contents and services
Not present
4.4
Health
Not present
5.4
Transport(2)
Not present
15.1
International air transport(2)
Present
1.59
Communication
Not Present
3.5
Recreation and culture
Present
9.1
Education
Present(1)
1.8
Miscellaneous goods and services
Not present
6.8
Monthly food price index
Present
June 2011 quarter (%)
1
The housing and household utilities group and the education group were left unadjusted as their
seasonality is due to administrative changes and not normal seasonal effects.
2
The index series for international air transport is known to have noticeable seasonality in its price
movements, so it was seasonally adjusted for the purpose of this investigation.
The following sections present more detailed results for the seasonal adjustment of the
quarterly all groups CPI, the monthly food price index, and the quarterly education group.
7.7.4 Quarterly all groups CPI
Directly seasonally adjusting the all groups CPI through X-12-ARIMA did not show any
identifiable seasonality. The seasonally adjusted series for the all groups CPI was
therefore also produced using the indirect method, which was aggregated from the
seasonally adjusted series at the group level and included the international air transport
class. For group series that had no identifiable seasonality, the original, unadjusted data
was used in the aggregation process. For the housing and household utilities, and
education, groups the values were left unadjusted.
7.7.4.1 Results
The seasonally adjusted series for the all groups CPI (aggregated from the groups) does
not appear markedly different from the unadjusted series from the June 2006 quarter to
the September 2012 quarter (see figure 7.1).
150
Seasonality in the CPI
Figure 7.1
Comparison between unadjusted and indicative seasonally adjusted CPI
Base: June 2006 quarter (=1000)
June 2006–September 2012 quarters
Index
1200
Adjusted
Unadjusted
1150
1100
1050
1000
950
J
06
S
D
M
07
J
D
S
M
08
J
S
D
M
09
J
S
D
M
10
J
S
D
M
11
J
S
D
M
12
J
Source: Statistics New Zealand
However, some differences are seen by reviewing differences in the quarterly movements
of the two series. Of the 25 quarterly percentage movements (figure 7.2) over the period
reviewed, nine were higher, 12 were lower and four were identical to the original series.
The greatest difference in the movements of the two series is 0.4 of a percentage point –
for the March 2008 and 2009 quarters. In each of these cases the movement in the
adjusted series is higher. In quarters where the movements in the adjusted series are
lower the greatest difference is 0.3 of a percentage point in the September quarters – for
2006, 2008, 2009, 2010, and 2011. In each of these cases the lower movement in the
adjusted series indicates that seasonal price rises for some items are being removed
from these September quarters.
The December 2010 quarter included the change in the rate of goods and services tax
(GST) from 12.5 to 15 percent.
Figure 7.2
Quarterly percentage changes
CPI All groups – unadjusted and seasonally adjusted
September 2006–September 2012 quarters
2.5
Percent
Unadjusted
2.0
Adjusted
1.5
1.0
0.5
0.0
-0.5
-1.0
S
06
D
M
07
J
S
D
M
08
J
S
D
M
09
Source: Statistics New Zealand
151
J
S
D
M
10
J
S
D
M
11
J
S
D
M
12
J
S
S
Seasonality in the CPI
7.7.5 The quarterly food group and monthly FPI
More differences between the unadjusted and seasonally adjusted data are noticeable
when drilling down into the groups showing seasonality.
Within the food group, fresh fruit and vegetables in particular are a source of seasonal
price movements within the CPI.
The quarterly food group is generally affected by rising seasonal prices in the September
quarter followed by seasonal falls in the December quarter.
This section discusses results for the monthly FPI – to demonstrate the seasonality within
food. The prices collected for the FPI also feed into the quarterly food group calculations.
7.7.5.1 Results
Between June 2006 and October 2012, movements in the seasonally adjusted monthly
FPI are generally different from the unadjusted series (see figure 7.3).
Figure 7.3
Comparison between unadjusted and indicative seasonally
adjusted FPI
Base: June Month (=1000)
1300
June 2006–October 2012
Index
Unadjusted
Adjusted
1200
1100
1000
900
J
06
S
D
M
07
J
S
D
M
08
J
S
D
M
09
J
S
D
M
10
J
S
D
M
11
J
S
D
M
12
J
Source: Statistics New Zealand
Of the 76 monthly percentage movements over this period only seven are identical, 48 of
the adjusted series movements are higher, and 21 are lower.
The greatest difference in the monthly movements of the two series is 1.2 percentage
points, which occurs in July 2007 and 2009, and in June 2011 and 2012. In each instance
the adjusted series shows lower movements than the unadjusted series. Movements for
the June and July months are generally lower for the adjusted series, indicating that
seasonal price rises for some items are being removed for these months when the series
is adjusted, which is expected. In months where the movements in the adjusted series
are higher, the greatest difference is 1.0 percentage point in October 2010 (when GST
increased from 12.5 to 15 percent).
The two series move in opposite directions 17 times over the period. The largest
differences are 1.1 percentage points in July 2008 and 1.2 percentage points in July
2009, with the unadjusted series showing a positive movement and the adjusted series a
negative movement in each case. This indicates that without the seasonal price increases
in these months the index would have gone down. For about half the occasions in which
the two series go in different directions, the unadjusted series is negative and the
adjusted positive. In most instances the unadjusted and adjusted series are relatively
152
S
Seasonality in the CPI
close to zero (ie no more than 0.5 of a percentage point apart), with the largest
separation being 0.8 of a percentage point in October 2012.
Figure 7.4
Monthly percentage changes
FPI – unadjusted and seasonally adjusted
September 2006–October 2012
4
Percent
Unadjusted
Adjusted
3
2
1
0
-1
-2
S
06
D
M
07
J
S
D
M
08
J
S
D
M
09
J
S
D
M
10
J
S
D
M
11
J
S
D
M
12
J
S
Source: Statistics New Zealand
The average, absolute monthly percentage change for the unadjusted series is 0.7
percent and slightly less for the seasonally adjusted series at 0.6 percent (table 7.2).
Negative movements have been treated as positive when calculating the absolute. Table
7.2 again demonstrates that the movements in seasonally adjusted series are
substantially lower during June and July.
153
Seasonality in the CPI
Table 7.2
Average absolute percentage changes
Monthly food price index (negative movements were converted to positive to calculate
absolutes)
Original
Adjusted
Month
Percent change
Monthly food price index
All
0.7
0.6
January
1.1
0.8
February
0.5
0.6
March
0.5
0.5
April
0.3
0.4
May
0.5
0.6
June
1.4
0.6
July
1.0
0.5
August
0.8
0.9
September
0.7
0.6
October
1.0
1.0
November
0.4
0.5
December
0.5
0.5
7.7.6 Education group and seasonality
The education group at first glance appears to be seasonal. Education fees tend to
change only once a year, near the start of the calendar year, and prices are therefore
surveyed once in the March quarter (early childhood education is surveyed quarterly).
The increase coincides with the start of the school and university academic year. This
causes a stepped pattern in the time series for the education group for the March quarter
as fees rise; with much smaller changes in other quarters that are due to price changes in
early childhood education services, which tend to change more frequently.
For price changes in the education group, the series has increased each March quarter
since the June 2006 quarter. Since then, the magnitude of the rises in the March quarters
has been similar – between 3.1 and 5.8 percent.
An issue with the education series is that the magnitude of change varies in each March
quarter, a consequence of the price changes being subject to administrative decisions
and not regular seasonal fluctuations. The series is therefore not considered to be
seasonal.
A similar situation occurs with local authority rates, items that include excise tax (alcohol,
and cigarettes and tobacco), vehicle registration, and sports club fees. Prices for these
items are also subject to administrative decisions and not regular seasonal fluctuations.
154
Seasonality in the CPI
7.7.7 Issues for seasonaly adjusted series developments
7.7.7.1 Length of time series
X-12-ARIMA requires at least three complete years of time-series data to calculate a
seasonal adjustment. Seasonality was tested at the group level, using data from the June
2006 quarter to the September 2012 quarter. The degree of seasonality can differ
depending on the length of the time series used, as seasonal patterns tend to evolve over
time. Longer time series provide better stability for testing seasonality.
7.7.7.2 The level of adjustment used in an indirectly adjusted series
The CPI has many components, which are aggregated through several levels until the
final index is produced at the all groups CPI level. The seasonal adjustment investigation
started from the group level to determine which series should be adjusted. There are
various options for the starting level used but statistical integrity should be the main
consideration, balanced with the resources required relative to other possible initiatives.
The more sub-series used in an indirect adjustment, the more lower-level seasonality will
be uncovered, if it exists. The indirectly adjusted series aggregate may exhibit more
volatility than a direct measure, but this depends on the presence of different seasonal
patterns and levels of irregular movements in the source data.
7.7.7.3 Development costs
An indirect method, where a seasonally adjusted CPI would be calculated from
aggregating seasonally adjusted lower-level indexes, could be developed. The cost to
develop the overall series is influenced by the number of lower-level series to be
adjusted/developed. Developing a seasonally adjusted CPI could be done within the
existing prices statistical maintenance budget. However, doing this would come at the
cost of other prices statistical maintenance initiatives.
7.8 International practice
Several countries publish seasonally adjusted CPIs including Australia, Canada, the
United States, and Japan. European countries do not generally publish seasonally
adjusted CPIs.
The seasonally adjusted series published are not typically headline measures but are
published to assist analysis of the headline unadjusted series.
7.8.1 Australia
The Australian Bureau of Statistics (ABS) publishes the following analytical series:
• All groups CPI, seasonally adjusted – comprises all components included in the all
groups CPI, seasonally adjusted where seasonality is identified at the ‘weighted
average of eight capital cities’ level.
• CPI expenditure classes seasonally adjusted – comprises the subset of seasonally
adjusted expenditure classes (62 classes) at the weighted average of eight capital
cities level.
• Trimmed mean and weighted median – two analytical measures of underlying trend
inflation.
The impetus to proceed with seasonally adjusted series at the ABS was part of
improvements made to the trimmed mean and weighted median series. These series
remove volatility observed in the quarterly price change of the CPI caused by large,
irregular price movements, to estimate the underlying trend inflation. In calculating
underlying trend inflation measures, previous analysis by the Reserve Bank of Australia
found seasonal adjustment “reduces the chance that a highly seasonal item will be
trimmed from the distribution of price changes, providing that inflation over the year in
that item is not significantly greater than overall CPI inflation” (Roberts, 2005).
155
Seasonality in the CPI
The trimmed mean and weighted median are calculated using a distribution of
expenditure classes and are derived as follows:
• The CPI expenditure classes are ranked from lowest to highest according to the
seasonally adjusted percentage change from the previous quarter.
• The seasonally adjusted relative weight of each expenditure class is calculated
based on its previous quarter’s contribution to the all groups CPI.
• The trimmed mean is calculated using a weighted average of percentage change
from the previous quarter (seasonally adjusted), from the middle 70 percent of the
distribution.
• The weighted median is calculated using the percentage change from the previous
quarter (seasonally adjusted) expenditure class at the 50th percentile of the
distribution.
Statistics New Zealand also publishes trimmed mean and weighted median measures.
The trimmed mean measure involves calculating the weighted mean of price movements
in the central core of the ranked distribution, in effect cutting off the tail at each end of the
distribution of price movements. The items that are excluded from the trimmed mean from
period to period depend on the distribution of price changes.
The weighted median measure is the movement for the item or group of items that is in
the middle of the ranked distribution (also known as the 50th percentile). That is, half of
the weighted price changes are below the median, and half are above. As Statistics NZ
does not currently seasonally adjust the CPI data, highly seasonal movements are
removed from the quarterly trimmed mean measure. The ABS calculates its annual
trimmed mean measure using the quarterly trimmed series. However, Statistics NZ
calculates annual trimmed means from annual price movements, which are free of
regular seasonal price movements.
If Statistics NZ were to produce a seasonally adjusted series there would be the potential
to enhance or expand the range of trimmed mean measures by trimming the seasonally
adjusted series, rather than the actual series.
7.8.2 Canada
Currently for the Canadian CPI, 13 series are produced on a seasonally adjusted basis at
the national level. These are:
• the all-items CPI;
• the eight major component indexes; and
• four special aggregates: (1) the Bank of Canada's core CPI, (2) all-items CPI
excluding eight of the most volatile components identified by the Bank of Canada,
(3) all-items CPI excluding food, (4) all-items CPI excluding food and energy.
The current seasonal adjustment process in place is such that each series is adjusted
directly, and thus, is not the result of aggregating seasonally adjusted components.
7.8.3 United Kingdom
As discussed previously, the ONS in the United Kingdom produces a seasonally adjusted
measure of consumer price inflation, the seasonally adjusted retail prices index (SARPIY)
at a level equivalent to section level in the New Zealand CPI.
7.8.4 United States of America
The uses of a seasonally adjusted series can be summed up by the notes for the
seasonally adjusted CPI published by the Bureau of Labor Statistics (BLS). Fact sheet on
seasonal adjustment in the CPI:
156
Seasonality in the CPI
The Consumer Price Index (CPI) is a measure of the change in prices of goods and
services purchased by urban consumers. The CPI publishes unadjusted price
indexes at the national, metropolitan area, and regional levels, and seasonally
adjusted indexes for selected groups and subgroups of the CPI at the national level
where there is a significant pattern of seasonal price change.
What is seasonal adjustment?
Seasonal adjustment removes the effects of recurring seasonal influences from
many economic series, including consumer prices. The adjustment process
quantifies seasonal patterns and then factors them out of the series to permit
analysis of non-seasonal price movements. Changing climatic conditions,
production cycles, model changeovers, holidays, and sales can cause seasonal
variation in prices. For example, oranges can be purchased year-round, but prices
are significantly higher in the summer months when the major sources of supply
are between harvests.
Who should use CPI seasonally adjusted indexes?
Data users who are interested in analysing general price trends in the economy
should use seasonally adjusted indexes. Seasonally adjusted data are usually
preferred in the formulation of economic policy and for economic research, because
they eliminate the effects of changes that normally occur at the same time and in
about the same magnitude every year.
Who should not use CPI seasonally adjusted indexes?
Those who use the CPI in escalation agreements, to adjust payments for changes
in prices, should typically not use seasonally adjusted indexes. Unadjusted indexes
are used extensively for escalation purposes because they measure the change in
actual prices consumers pay for goods and services. Many collective bargaining
contract agreements and pension plans, for example, tie compensation changes to
the Consumer Price Index unadjusted for seasonal variation.
157
Seasonality in the CPI
7.9 Seasonal adjustment options for the New Zealand
CPI
Table 7.3 outlines the options and implications for seasonal adjustment for the New
Zealand CPI.
Table 7.3
Seasonal adjustment options for the New Zealand CPI
Option
Implications
Status quo
•
•
An analytical fully
(indirectly)
seasonally
adjusted series.
•
•
•
•
The CPI and FPI would continue to be released as currently
without seasonally adjusted series to help users understand
the effect of seasonality on period-on-period movements.
Annual movements are not influenced by regular seasonal
effects.
A thorough investigation would be required to determine the
optimal level to indirectly seasonally adjust each CPI group.
This would include investigating groups that currently exhibit
no seasonality at the group level when adjusted directly,
such as the transport group discussed previously. This
investigation may need to go down to basket level in some
cases. This could be done within the existing prices
statistical maintenance budget. However, doing this would
come at the cost of other prices statistical maintenance
initiatives.
Analytical seasonally adjusted series would be subject to
revision every quarter with the inclusion of new CPI data.
There would be the potential to use the seasonally adjusted
series in the calculation of additional or enhanced trimmed
mean series.
Seasonally adjusted series would help improve users’
understanding of the effect of seasonality on period-onperiod CPI and FPI movements.
158
Seasonality in the CPI
Appendix 7a: Additional seasonality tables
Appendix table 1
Fresh fruit and vegetable variable weighted basket – December 1988 quarter
Commodity percentage expenditure at base period
Apples
Jan
Feb
Mar
Apr
May
Jun
Jul
Aug
Sep
Oct
Nov
Dec
0.41
0.39
0.40
0.38
0.34
0.32
0.30
0.30
0.25
0.17
0.28
0.26
0.28
0.27
0.25
0.27
0.28
0.29
0.31
0.02
0.02
0.02
0.03
0.03
0.10
0.10
0.14
0.19
0.18
0.14
0.11
0.13
0.11
0.21
1
0.14
0.12
Bananas
0.26
0.27
0.28
Nectarines
0.15
0.14
0.04
Oranges
0.09
0.08
0.10
0.11
0.14
0.12
0.04
0.05
0.07
0.02
Apricots
Kiwifruit
Peaches
1
Pears
1
0.06
Strawberries
0.04
Plums
0.12
0.05
0.05
0.05
0.05
0.07
0.04
0.08
0.08
0.04
0.03
0.02
0.12
0.15
0.04
0.04
1
1
Asparagus
0.04
0.04
0.04
0.05
0.02
Beans
Brussels
1
sprouts
0.04
0.04
0.02
0.03
0.03
0.04
0.04
0.04
Cabbage
0.03
0.03
0.05
0.07
0.09
0.09
0.10
0.10
0.09
0.08
0.06
0.05
Carrots
0.08
0.08
0.07
0.08
0.09
0.09
0.09
0.08
0.10
0.09
0.10
0.08
Cauliflower
0.03
0.03
0.05
0.08
0.10
0.09
0.10
0.09
0.07
0.08
0.06
0.05
0.04
0.03
0.04
0.04
0.04
0.04
0.03
0.03
0.06
0.06
Celery
1
0.03
0.03
Cucumber
0.05
0.03
Corn
Kumara
0.03
0.04
0.04
0.05
0.06
0.06
0.06
0.07
0.06
0.05
0.04
0.03
Lettuces
0.09
0.09
0.08
0.08
0.04
0.05
0.05
0.06
0.08
0.10
0.09
0.08
Mushrooms
0.03
0.04
0.04
0.05
0.06
0.06
0.05
0.06
0.06
0.05
0.03
0.03
Onions
0.06
0.06
0.07
0.07
0.07
0.06
0.06
0.07
0.06
0.05
0.05
0.06
Potatoes
0.23
0.22
0.29
0.30
0.33
0.29
0.27
0.27
0.27
0.28
0.32
0.34
Pumpkin
0.04
0.04
0.04
0.05
0.04
0.04
0.04
0.05
0.05
0.04
0.04
0.05
Tomatoes
0.36
0.24
0.20
0.22
0.18
0.15
0.15
0.15
0.20
0.22
0.31
0.31
0.05
0.04
0.04
0.04
0.04
0.05
0.03
2.01
2.01
2.01
2.01
2.01
2.01
2.01
2.01
2.01
Broccoli
Total
2
2.01
2.01
2.01
Fresh fruit and vegetable items no longer in the CPI basket.
2
Totals are calculated from unrounded data.
1
0.07
0.02
Mandarins
Tangeloes
0.05
The above weightings demonstrate that the availability of seasonal fruit and vegetables
has improved since 1988. For example, kiwifruit was price-surveyed for only four months
of the year in 1988. Since then the season has extended, with improved storage
technology. Kiwifruit is now imported in significant quantities in the off-season, allowing
kiwifruit of good quality to now be available in retail outlets throughout the year. Likewise,
mandarins, beans, celery, cucumber, kumara, and broccoli are generally available to be
price-surveyed in the CPI in all months, barring irregular fluctuations in supply. On
occasion, it has been difficult to obtain sufficient prices for strawberries and nectarines in
winter months. Prices for nectarines were historically not included in the April and May
159
Seasonality in the CPI
FPI. Similarly, prices for strawberries were not included in the May and June FPI,
because not enough prices can be collected from stores during these months. With these
exceptions, most fresh fruit and vegetables items in the CPI basket can now be pricesurveyed in all months of the year.
Appendix table 2
Range of percentage changes from previous period
Original
Group/Class
Min
Max
Adjusted
Range
Min
Max
Range
Percent
All groups CPI (aggregated)
-0.5
2.3
2.8
-0.7
2.2
2.9
Food
-2.4
3.7
6.1
-1.7
2.8
4.5
Clothing and footwear
-1.2
1.8
3.0
-0.6
1.3
1.9
-16.5
13.9
30.4
-13.6
7.7
21.3
Recreation and culture
-2.4
2.9
5.3
-1.3
1.7
3.0
Monthly food
-1.5
2.8
4.3
-1.3
3.2
4.5
International air transport
160
Seasonality in the CPI
Appendix table 3
Average absolute percentage changes from previous period
Original
Adjusted
Group/Class
Quarter
Percent
Food
Clothing and footwear
International air transport
Recreation and culture
161
All
1.4
1.3
March
1.1
1.2
June
1.0
1.2
September
2.0
0.9
December
1.7
1.9
All
0.7
0.3
March
0.8
0.3
June
0.9
0.4
September
0.4
0.2
December
0.7
0.5
All
6.6
3.1
March
9.9
2.8
June
4.5
4.4
September
3.6
2.7
December
9.0
2.4
All
0.9
0.5
March
1.3
0.5
June
0.7
0.4
September
0.4
0.4
December
1.2
0.7
8 Dissemination of CPI and FPI information
8.1 Executive summary
This chapter outlines the range of CPI and FPI information that is published. Information
is disseminated through different mediums, including the Statistics New Zealand
information releases, Statistics NZ’s online time series database (Infoshare), and
quarterly Price Index News newsletters.
Statistics NZ currently publishes national CPI indexes down to the class level of the New
Zealand Household Expenditure Classification (NZHEC). This is the level of the CPI at
which expenditure weights are fixed. Beyond this, indexes are also published for 17
selected ‘sections’ (the level below class) in the food group.
Analytical CPI series are also published. These include measures of ‘core’ inflation, such
as the trimmed means and weighted percentile series; and other measures, such as the
tradable and non-tradable series, and CPI less/plus certain items or groupings.
Increasing use is being made of visualisation techniques to convey CPI information. This
includes infographics and an upcoming ‘price kaleidoscope’, which will offer a dynamic
visualisation of relative price change and the relative importance of expenditure classes
in the index.
8.2 Issue to consider
• The range of CPI and FPI information that is published.
8.3 Purpose of this chapter
This chapter outlines the range of information that is published for the New Zealand CPI
and FPI. The committee and CPI user community are asked to comment on whether
there is a need to modify the range of published information to better meet user needs,
and if so, in what areas this could be done.
8.4. Publishing mediums
8.4.1 Information releases
A key publication medium for Statistics NZ is its information release.
CPI and FPI results are published in quarterly (CPI) and monthly (FPI) information
releases. These releases include quarterly/monthly and annual movements, and include
key points and a commentary that covers contextual information. This commentary
includes graphical information outlining price movements, and contribution information
that shows which areas of the CPI basket influenced price change for the period.
CPI information releases are the most-viewed releases produced by Statistics NZ on their
first day of publication; the releases/tables are some of the most downloaded. Table 8.1
compares this information for the CPI and other high-profile releases.
162
Dissemination of CPI and FPI information
Table 8.1
Unique visitors on first day of publication and unique downloads for selected
Statistics NZ information releases
2012 calendar year
Average unique
Total number of
web page visitors
unique downloads
Release title
– first day of
(information release
release
and tables)
Consumers price index
485
29,000
Gross domestic product
410
19,000(1)
Household labour force
survey
335
10,400
Balance of payments
120
4,000
Labour cost index
110
7,800
Producers price index
100
6,300
50
3,100
Food price index
1. includes downloads for annual national accounts releases.
Information releases contain ‘definitions’ and ‘data quality’ sections that outline
background information on the FPI and CPI. These help users understand compilation
methods and interpret the results.
CPI and FPI information releases are accompanied by two sets of tables, titled ‘tables’
and ‘supplementary tables’. These tables show annual and quarterly/monthly percentage
changes, and index numbers for a range of sub-indexes and analytical series.
Information in the CPI supplementary tables provides longer-term time series, dating
back to 1914 for the all groups CPI, and to 1983 for the other published series. FPI
supplementary tables go back to 1960 for the food group, and to 2003 for the other
published series. The ‘tables’ for both CPI and FPI releases provide more recent
information.
Other information within the CPI ‘tables’ file includes:
• international comparisons
• expenditure and regional populations weights
• contribution information – details how each CPI component influenced the total
movement
• UN COICOP – provides a breakdown of CPI movements for the 12 divisions of the
international Classification of Individual Consumption according to Purpose
• distribution of price change.
For a detailed list of what is currently published in the tables and supplementary tables,
see appendix 8a.
8.4.2 Infoshare
The most comprehensive source of publicly available CPI information is Statistics NZ’s
online time-series database, Infoshare. This service provides users with a wide range of
time series free of charge. Infoshare includes quarterly (CPI) and monthly (FPI) index
numbers for all indexes and analytical series published in the tables and supplementary
163
Dissemination of CPI and FPI information
tables, as well as additional time series. A list of categories for the CPI and FPI time
series that are published on Infoshare is in appendix 8b.
Table 8.2 compares the number of views and unique visitors to the CPI subject area on
Infoshare (which includes FPI data) with other similar subject areas.
Table 8.2
Infoshare – number of unique visitors and total views by subject area
Total for calendar year 2012
Number of unique
visitors
Subject area
Number of views
Consumers price index(1)
4,000
14,500
National accounts
2,300
29,700
Household labour force survey
1,500
14,600
Producers price index
600
4,800
Balance of payments
450
4,000
Labour cost index
450
3,600
1. Includes FPI series.
8.4.3 Price Index News
CPI data is also published through Statistics NZ’s quarterly Price Index News electronic
newsletters. The newsletters provide analysis and storytelling outside the regular CPI
releases, using longer time-series data. They contain articles and infographics that
display CPI information in a more visual form.
Each issue of Price Index News offers short articles aimed at different types of price
index users. The articles generally fit the following categories:
• Short stories – which use CPI data to tell stories about price change, changes to
the CPI basket, and the general history of CPI items
• Working with others – updates on collaboration between the Prices team and
outside partners
• Virtual papers – technical papers on topics related to price measurement
• Nuts and bolts – sources and methods articles that detail how certain components
are calculated within the wider suite of price indexes
• On the horizon – information about events that will influence future price changes
• Development updates – initiatives to maintain or improve the relevance of the wider
suite of price indexes.
Nuts and bolts articles contribute to a ‘CPI sources and methods’ section on the Statistics
NZ website, which gives an overview of the methods used to both calculate weights and
measure price change within the CPI. Sources and methods articles are discussed
below.
164
Dissemination of CPI and FPI information
8.5 What is published
8.5.1 Classification
The CPI has been published using the New Zealand Household Expenditure
Classification (NZHEC) since the September 2006 quarter. The classification was
adopted following a recommendation from the 2004 CPI Revision Advisory Committee.
Statistics NZ consulted the public following the 2004 recommendation, and made the
decision to develop and adopt NZHEC, which is a modified version of COICOP. The
consultation found that users were generally in favour of a COICOP-based classification,
as long as modifications were made to suit New Zealand’s context.
Statistics NZ currently publishes national CPI indexes down to the class level of NZHEC.
The underlying quantities of CPI expenditure weights are held fixed at this level between
the three-yearly CPI reviews. Beyond this, indexes are also published for 17 selected
‘sections’ (the level below class) within the food group. These sections align with series
published under an old classification system, before Statistics NZ adopted NZHEC.
These sections are useful to users, and so they continue to be published. Figure 8.1
illustrates the different levels of the classification and the number of indexes published at
each level.
Figure 8.1
Consumers price index – NZHEC hierarchy
All groups CPI
Group (11 published)
Subgroup (45 published)
Class (108 published)
Published level
Section (17 published, 181 in total)
Below published level
Subsection (220 in total)
Item (499 in total)
Subitem (82 published, 710 in total)
As figure 8.1 shows, four additional levels of NZHEC are below the published (class)
level. Index numbers for these levels are calculated, but not published, with the exception
of 17 section-level sub-indexes, and weighted average prices for 82 subitems. A list of
CPI and FPI average-price subitems is in appendixes 8c and 8d, respectively.
Indexes are published down to the class level for two main reasons. Firstly, CPI data is
more accurate at the more aggregated levels of the index. For example, the expenditure
weight of the ‘fruit’ class accurately reflects expenditure on fruit, while the expenditure
weights for ‘oranges’ and ‘mandarins’ subitems incorporate expenditure on other citrus
fruit, that are not directly tracked in the CPI (eg lemons).
165
Dissemination of CPI and FPI information
Secondly, fixing the weight at the class level allows flexibility around updating the relative
importance of items below the class level between three-yearly CPI reviews. Currently,
relative quantities below this level are monitored. They may be adjusted where necessary
to reflect volume-related shifts in the relative importance of goods or services within the
expenditure classes, although in practice this has been done only occasionally.
Information below the published level is sometimes available through the quarterly Price
Index News newsletters (see Price Index News above), or on request, subject to quality
and confidentiality checks.
8.5.2 Regional breakdown
Regional CPI indexes are published down to the group level (excluding the
communication and education groups) for the following broad regions:
• Auckland
• Wellington
• Rest of North Island
• North Island
• Canterbury
• Rest of South Island
• South Island.
CPI prices are currently collected from retail outlets in the following 15 urban areas:
• Whangarei
• Auckland
• Hamilton
• Tauranga
• Rotorua
• Napier-Hastings
• New Plymouth
• Wanganui
• Palmerston North
• Wellington
• Nelson
• Christchurch
• Timaru
• Dunedin
• Invercargill.
Regional FPI indexes at the highest level are published for all 15 regions.
Some price measures, such as the purchase of newly built houses and housing rentals,
are designed to be representative only to the broad-region level. This means the current
sample designs of key components of the CPI do not support the publication of fit-forpurpose regional series for all 15 regions. However, they are available on request (with
caveats).
For more information on regional CPI price collection see the ‘Sampling framework’
chapter.
166
Dissemination of CPI and FPI information
8.5.3 Analytical series
Analytical series are published for:
• CPI less/plus certain items or groupings
• tradables/non-tradables
• UN COICOP
• ‘core’ inflation measures.
A comprehensive list of analytical series published in Infoshare is in appendix 8e.
The ‘Seasonality in the CPI’ chapter discusses options for seasonally adjusted time
series, which could also be produced as analytical series.
8.5.3.1 ‘Core’ inflation analytical series
Statistics NZ publishes two sets of analytical series aimed at measuring ‘core’ inflation.
These series are targeted at the underlying and more persistent drivers of price change.
The measures are the trimmed means and weighted percentiles.
The trimmed means measure excludes the influence of the largest price increases and
decreases in the CPI. This is currently done at the subitem level of the classification. The
trimmed means progressively remove the influence of the subitems displaying the largest
increases and decreases, ranging from 5 percent to 30 percent of the weight.
Weighted percentiles highlight the movement of subitem level indexes at points in the
distribution of price changes (eg 25th percentile, median) for a particular time period.
The Reserve Bank of New Zealand publishes additional series aimed at measuring core
inflation, such as the factor model and sectoral factor model measures.
8.5.4 Sources and methods
Statistics NZ publishes a collection of CPI sources and methods articles, sourced mainly
from articles originally included in Price Index News. These articles detail the sources and
methods used to calculate individual parts of the CPI. Articles consider how, for example:
• price change is measured
• expenditure weights are calculated
• samples are selected
• quality adjustments are carried out.
These articles provide transparency around how the CPI is calculated, and inform users
of the current practice.
8.5.5 Historical CPI data
CPI information releases are available online back to the March 2005 quarter. Older
releases can be requested.
Work is underway to collate and make available historical average price and weighting
information. This will be for various goods and services tracked in the CPI back to the
start of last century, and past advisory committee and CPI review reports. This
information is expected to be available later in 2013. This information has been used for
Price Index News articles, but it may also be of use to researchers and other CPI users.
167
Dissemination of CPI and FPI information
8.6 Data visualisation
Visualisation techniques are being used more often to convey CPI information. This
includes pictorial techniques such as infographics, and more dynamic approaches such
as a ‘price kaleidoscope’.
Infographics are generally included in Price Index News. They offer a visual
representation of CPI information, and often portray changes over historical periods.
Infographics present CPI information in a way that appeals to a wider audience, and help
to tell the underlying stories of CPI information. Appendix 8f shows three infographics that
use CPI information.
A price kaleidoscope being developed creates a visual representation of CPI price
change and the relative importance of different parts of the CPI.
Figure 8.2 shows a screenshot of how the price kaleidoscope might look (the prototype
was developed by the Auckland University of Technology).
Figure 8.2
New Zealand price kaleidoscope
The size of each CPI category on the circle represents its weight in the index. Colours
illustrate the magnitude of price change; for example, dark blue for strong price
decreases and dark red for strong price increases.
Users will be able to ‘drill down’ to view the relative importance of subgroups and classes
by clicking on a group (eg food). The kaleidoscope will also show the weight of each
group/subgroup/class at different time periods. The size of each category will change
over time, to reflect relative change in prices. The price kaleidoscope is expected to be
available on the Statistics NZ website in 2013.
Users will be able to select periods of change, or toggles backwards or forwards through
time.
168
Dissemination of CPI and FPI information
8.7 Personal inflation calculator
Another idea being investigated is an online personal inflation calculator. Similar
calculators operate in the United Kingdom and Germany.
The online calculator would allow people to estimate their own personal rate of inflation,
based on their individual spending patterns. Users enter their expenditure on different
groups of goods and services (eg what they spend on food, housing, utilities) and the
calculator uses each expenditure as weights to calculate personal inflation.
Because the CPI is designed to measure price change for households as a whole, a
personal inflation calculator could be a good supplement to the all groups CPI, or even
subpopulation indexes. It would provide a more targeted measure of inflation.
There are some potential issues to resolve before a calculator is produced. For example,
it may be appropriate for the online calculator to be based on a payment conceptual
framework, rather than the acquisition framework used in the CPI.
Similarly, the level at which users enter expenditure needs to be considered. For
example, users could enter a single total for ‘food’, or they could enter expenditure on
separate food types.
8.8 Index reference period
Index numbers are published to the nearest whole number using an index reference of
1000. This is consistent with international practice – most CPIs are expressed either on a
reference of 1000 or 100.0. The current index reference period is the June 2006 quarter
(=1000) for the CPI and the June 2006 month (=1000) for the FPI.
It may be necessary to ‘re-reference’ the CPI and FPI as part of the 2014 CPI review,
given how long the current index reference period has been used. Because rereferencing the index time series can cause a temporary disruption to users, it is only
done when necessary. Typically, the CPI is re-referenced when significant classification
or method changes are made, or index numbers for some series become very low
relative to the base of 1000. This is because the smaller an index number gets, the less
precise the measurement for that particular component is. Examples of sub-indexes with
low index numbers at the December 2012 quarter are:
• telecommunication equipment – 238
• audio-visual equipment – 270.
Re-referencing the index would involve ‘rescaling’ the CPI, so the June 2014 quarter
(CPI) and June 2014 month (FPI) index numbers are set to 1000. To achieve this, all
index numbers before the index reference period would also be rescaled. Rescaled index
numbers would be published with decimal places for periods before the new index
reference period, in order to preserve originally published quarterly/monthly and annual
movements. There is a risk that CPI users could calculate slightly different
quarterly/monthly or annual movements if they use rounded values rather than
unrounded values.
169
Dissemination of CPI and FPI information
Appendix 8a: Tables and supplementary tables
published as part of information releases
Appendix table 1
Consumers price index tables
Table no.
Description
1
Consumers price index, tradables, non-tradables, and all groups – index numbers
and percentage changes
2.01
Consumers price index, groups and subgroups – index numbers
2.02
Consumers price index, groups and subgroups, percentage change from
previous quarter
2.03
Consumers price index, groups and subgroups, percentage change from same
quarter of previous year
3.01
Consumers price index, selected groupings – index numbers
3.02
Consumers price index, selected groupings, percentage change from previous
quarter
3.03
Consumers price index, selected groupings, percentage change from same
quarter of previous year
4
International comparisons of consumer price indexes, excluding housing and
household utilities group and credit services class – index numbers and
percentage changes
5
Weighted average retail prices of selected items
6
Consumers price index, expenditure weights, by group
7
Consumers price index, population weights, by region/pricing centre
8.01
Contribution to all groups and percentage change from previous quarter, by
group, subgroup, or class
8.02
Contribution to all groups and percentage change from same quarter of previous
year, by group, subgroup, or class
9
Consumers price index, expenditure weights, by group, subgroup, or class
10
Consumers price index, COICOP divisions – index numbers and percentage
changes
11
Consumers price index, trimmed means and all groups – percentage changes
12
Consumers price index, weighted percentiles and all groups – percentage
changes
13
Distribution of national item-level index movements from the previous quarter
170
Dissemination of CPI and FPI information
Appendix table 2
Consumers price index supplementary tables
Table no.
Description
Earliest published
quarter
1
Consumers price index, tradables, non-tradables, and all
groups – index numbers and percentage changes
June 1914
2.01
Consumers price index, groups and subgroups – index
numbers
March 1983
2.02
Consumers price index, groups and subgroups,
percentage change from previous quarter
March 1983
2.03
Consumers price index, groups and subgroups,
percentage change from same quarter of previous year
March 1983
3.01
Consumers price index, selected groupings – index
numbers
March 1983
3.02
Consumers price index, selected groupings, percentage
change from previous quarter
March 1983
3.03
Consumers price index, selected groupings, percentage
change from same quarter of previous year
March 1983
4.01
Purchase of housing class, selected regions – index
numbers and percentage changes
June 2006
4.02
Actual rentals for housing subgroup, selected regions –
index numbers and percentage changes
June 2006
171
Dissemination of CPI and FPI information
Appendix table 3
Food price index tables
Table no.
Description
1
Food price index, subgroups
2.01
Food price index, subgroups, classes, and selected sections – index numbers
2.02
Food price index, subgroups, classes, and selected sections, percentage change
from previous month
2.03
Food price index, subgroups, classes, and selected sections, percentage change
from same month of previous year
3
Weighted average retail prices of selected food items
4
Contribution to food price index and percentage change, by subgroup, class, or
selected section
5
Distribution of national item-level index movements from previous month
6
Food expenditure weights, by subgroup, class, or selected section
7
Population weights, by region/pricing centre
Appendix table 4
Food price index supplementary tables
Table no.
Description
Earliest published
month
1
Food price index, subgroups
January 1960
2.01
Food price index, subgroups, classes, and selected
sections – index numbers
March 2003
2.02
Food price index, subgroups, classes, and selected
sections, percentage change from previous month
March 2003
2.03
Food price index, subgroups, classes, and selected
sections, percentage change from same month of
previous year
March 2003
172
Dissemination of CPI and FPI information
Appendix 8b: Infoshare – CPI and FPI time-series
categories
• CPI all Groups for New Zealand (Qrtly-Mar/Jun/Sep/Dec)
• CPI Level 1 Groups for New Zealand (Qrtly-Mar/Jun/Sep/Dec)
• CPI Level 2 Subgroups for New Zealand (Qrtly-Mar/Jun/Sep/Dec)
• CPI Level 3 Classes for New Zealand (Qrtly-Mar/Jun/Sep/Dec)
• CPI Level 4 Sections for New Zealand (Qrtly-Mar/Jun/Sep/Dec)
• CPI Non-standard all Groups Less/Plus Selected Groupings for New Zealand
(Qrtly-Mar/Jun/Sep/Dec)
• CPI Non-standard Selected Quarterly Groupings for New Zealand (QrtlyMar/Jun/Sep/Dec)
• CPI Non-standard Tradable & Non-tradable Component Series (QrtlyMar/Jun/Sep/Dec)
• CPI Non-standard Trimmed Means and Weighted Percentiles for New Zealand
(Qrtly-Mar/Jun/Sep/Dec)
• CPI Non-standard UN COICOP (Qrtly-Mar/Jun/Sep/Dec)
• CPI Regional all Groups (Broad Regions) (Qrtly-Mar/Jun/Sep/Dec)
• CPI Regional Groups (Broad Regions) (Qrtly-Mar/Jun/Sep/Dec)
• CPI Selected Quarterly Weighted Average Prices for New Zealand (QrtlyMar/Jun/Sep/Dec)
• Food Price Index for New Zealand (Monthly)
• Food Price Index Level 2 Subgroups for New Zealand (Monthly)
• Food Price Index Level 3 Classes for New Zealand (Monthly)
• Food Price Index Level 4 Sections for New Zealand (Monthly)
• Food Price Index Regional (Broad Regions) (Monthly)
• Food Price Index Regional (Separate Regions) (Monthly)
• Food Price Index Selected Monthly Groupings for New Zealand (Monthly)
• Food Price Index Selected Monthly Weighted Average Prices for New Zealand
(Monthly)
173
Dissemination of CPI and FPI information
Appendix 8c: Infoshare – published CPI average prices
(dating back to June 2006 quarter)
• Beer – bottles (supermarket & liquor store), 1 dozen
• Beer – glass (licensed premises), 400ml
• Wine – cask, white (supermarket & liquor store), 3 litres
• Whisky (liquor store), 1000ml
• Cigarettes (supermarket & convenience store), pk of 25
• Socks – men's (clothing store & department store), pair
• Panty-hose – 15 denier, average size (supermarket), pair
• Dry cleaning – men's 2-piece, woollen suit
• Spouting/guttering – plastic, per 3m
• Concrete blocks – 390 mm x 190 mm x 190 mm, per 100
• House paint – acrylic, white, 10 litres
• Plasterboard – 2400 mm x 1200 mm, per sheet
• Wallpaper – roll
• Carpet – wool, heavy duty, cut pile, width 3.66 m, per m
• Electric light bulb – standard, 100 watt (supermarket)
• Bleach (supermarket), 2.5 litres
• Clothes washing powder – concentrate (supermarket), 500g(1)
• Detergent – dishwashing liquid (supermarket), 900ml
• Cling food wrap – refill roll (supermarket), 45m
• General Practitioner – consultation, adult without community services card
• Optometrist – examination
• Dental examination, 2 X-rays, scale and polish
• Car battery – 12 volts
• Petrol – 91 octane, 10 litres
• Petrol – 95/98 octane, 10 litres
• Diesel, 10 litres
• Warrant of fitness – private car
• Postage – standard, medium-size envelope
• Compact disc – current top 10 album (record store & department store)
• Pet food – canned (supermarket), 700g
• DVD hire – overnight Friday, new release, 1 movie
• Envelopes – medium-size (supermarket), pk of 20
• Computer printer paper – 1 ream, 500 sheets
• Hairdressing – women's, shampoo, cut and blow wave
• Bathroom soap – cake, 100 g (supermarket), pk of 4
174
Dissemination of CPI and FPI information
• Shampoo (supermarket), 400ml
• Tissues – facial (supermarket), box of 180
• Toilet paper (supermarket), 12 rolls(2)
1. 1kg until June 2011 quarter.
2. 4 rolls until June 2011 quarter.
175
Dissemination of CPI and FPI information
Appendix 8d: Infoshare – published FPI average prices
(dating back to June 2006 month)
• Oranges, 1kg
• Bananas, 1kg
• Apples, 1kg
• Kiwifruit, 1kg
• Sultanas (supermarket only), 375g
• Peaches – canned (supermarket only), 410g
• Lettuce, 1kg
• Broccoli, 1kg
• Cabbage, 1kg
• Tomatoes, 1kg
• Carrots, 1kg
• Mushrooms, 1kg
• Potatoes, 1kg
• Peas – frozen (supermarket only), 1kg
• Beef steak – blade, 1kg
• Beef steak – porterhouse/sirloin, 1kg
• Beef – mince, 1kg
• Pork – loin chops, 1kg
• Lamb – chops, 1kg
• Bacon – middle rashers (supermarket only), 1kg
• Sausages, 1kg
• Tuna – canned (supermarket only), 185g
• Bread – white sliced loaf, 700g
• Biscuits – chocolate, 200g
• Breakfast biscuits, 1kg
• Flour – white (supermarket only), 1.5kg
• Rice – long grain, white (supermarket only), 1kg
• Milk – standard homogenised, 2 litres
• Yoghurt – flavoured, 150g pottle (supermarket only), pk of 6
• Cheese – mild cheddar (supermarket only), 1kg
• Eggs, dozen
• Butter – salted, 500g
• Sugar – white, 1.5kg
• Chocolate – block (supermarket only), 250g
• Spaghetti – canned, 420g
176
Dissemination of CPI and FPI information
• Coffee – instant, 100g
• Tea bags (supermarket only), box of 100
• Soft drink, 1.5 litres
• Bottled water, 750ml
• Fish and chips, One fish/chips
• Meat pie – hot, each
• Fruit juice – apple based (supermarket only), 3 litre(1)
• Potato crisps, 150g(2)
• Tomato sauce – canned, 560g(3)
1. 1 litre until June 2011 month.
2. 190g until June 2011 month.
3. 575g until June 2011 month.
177
Dissemination of CPI and FPI information
Appendix 8e: Infoshare – analytical series
• All groups less credit services subgroup
• All groups less central and local government charges
• All groups plus interest
• All groups less food group
• All groups less alcoholic beverages and tobacco group
• All groups less clothing and footwear group
• All groups less housing and household utilities group
• All groups less household contents and services group
• All groups less health group
• All groups less transport group
• All groups less communication group
• All groups less recreation and culture group
• All groups less education group
• All groups less miscellaneous goods and services group
• All groups less housing and household utilities group and credit services subgroup
• All groups less purchasing of new housing class
• All groups less household energy subgroup and vehicles fuels
• All groups less vehicle fuels
• All groups less food group and vehicle fuels
• All groups less petrol class
• All groups less food group, household energy subgroup and vehicle fuels
• All groups less International travel
• All groups less alcoholic beverages subgroup
• All groups less cigarettes and tobacco subgroup
• Tradable all groups
• All groups tradables component less vehicle fuels
• Non-tradable all groups
• All groups non-tradables less housing and household utilities group
• All groups non-tradables less purchase of new housing class
• Quarterly 5, 10, 15, 20, 25, and 30 percent trim
• Annual 5, 10, 15, 20, 25, and 30 percent trim
• Quarterly weighted 10th, 25th, median, 75th, and 90th percentile
• Annual weighted 10th, 25th, median, 75th, and 90th percentile
• Annual 5, 10, 15, 20, 25, and 30 percent trim – June 2006 qtr weights
• Annual weighted 10th, 25th, median, 75th, and 90th percentile –- June 2006 qtr
weights
• Annual 5, 10, 15, 20, 25, and 30 percent trim – June 2008 qtr weights
178
Dissemination of CPI and FPI information
• Annual weighted 10th, 25th, median, 75th, and 90th percentile – June 2008 qtr
weights
• Annual 5, 10, 15, 20, 25, and 30 percent trim – June 2011 qtr weights
• Annual weighted 10th, 25th, median, 75th, and 90th percentile – June 2011 qtr
weights
• UN COICOP – Food and non-alcoholic beverages
• UN COICOP – Alcoholic beverages, tobacco and narcotics
• UN COICOP – Clothing and footwear
• UN COICOP – Housing, water, electricity, gas and other fuels
• UN COICOP – Furnishings, household equipment and routine household
maintenance
• UN COICOP – Health
• UN COICOP – Transport
• UN COICOP – Communications
• UN COICOP – Recreation and culture
• UN COICOP – Education
• UN COICOP – Restaurants and hotels
• UN COICOP – Miscellaneous goods and services
• UN COICOP – Fuels and lubricants for personal transport
179
Dissemination of CPI and FPI information
Appendix 8f: Recent infographics
Not so cut and dried: 60 years of tracking haircut prices in the CPI (published April 2012)
180
Dissemination of CPI and FPI information
Electronic gadgets in the CPI (published January 2013)
181
Dissemination of CPI and FPI information
Bigger, safer, better tracking retail and quality adjusted new car prices in the CPI
(published October 2011)
182