Comments on Improving Consumption Measurement
Chris Carroll
Johns Hopkins University
Remarks At CRIW-NBER Conference On Improving Consumption Measurement
Cambridge, MA
July 13, 2009
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
Why Are Some Surveys/Countries Better Than Others?
My Best Guess
We Have A Serious Problem
Nondurable Consumption Per−capita
Food Consumption Per−capita
8.2
9.5
9.4
8.1
Log − 2003 $
Log − 2003 $
9.6
NIPA
CEX
PSID
8.3
8
7.9
7.8
9.3
9.2
NIPA
CEX
9.1
9
7.7
8.9
7.6
8.8
7.5
1980
1985
1990 1995
Year
2000
2005
1980
1985
1990 1995
Year
2000
2005
Housing Services Per−capita
Durable Consumption Per−capita
8.2
8.1
7.8
8
7.7
7.9
Log − 2003 $
Log − 2003 $
7.9
7.6
7.5
7.4
7.3
7.7
7.6
7.5
NIPA
CEX
7.2
7.1
1980
NIPA
CEX
7.8
1985
1990 1995
Year
2000
2005
Carroll
7.4
1980
1985
1990 1995
Year
Improving the CEX
2000
2005
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
Why Are Some Surveys/Countries Better Than Others?
My Best Guess
CE vs NIPA for Canada
0
Consumption
5000
10000
15000
Figure 2: Mean Nominal Non-Durable Consumption
1975
1980
1985
SNA
1990
Year
1995
2000
2005
FAMEX/SHS
the SCF had unusually small samples
1979, the
1981
Carrollin 1977,
Improving
CEXand 1984, which explains
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
Why Are Some Surveys/Countries Better Than Others?
My Best Guess
CE vs NIPA for Germany
Per Capita Income in Euros, Prices of 2000
1.8
1.7
x 10
4
Per Capita Disposable Income from NIPA, EVS & GSOEP
EVS
GSOEP
NIPA
1.6
1.5
1.4
1.3
1.2
1.1
1
1985
1990
Carroll
1995
Year
2000
Improving the CEX
2005
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
Why Are Some Surveys/Countries Better Than Others?
My Best Guess
1
1.2
1.4
1.6
1.8
CE vs NIPA for Italy
1980
1985
1990
1995
National Accounts
2000
2005
SHIW
Figure 1: Average per-capita total consumption in the SHIW
and in the National Accounts
Carroll
Improving the CEX
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
Why Are Some Surveys/Countries Better Than Others?
My Best Guess
NIPA for
Russia
sonCE
ofvsRLMS
with
Official Statistics
1000 2000 3000 4000 5000
B.Consumption per capita
2004
1994
member
1998
2000
2002
RLMS
0
IPA
1996
2004
NIPA
D.Non-food consumption per household member
Carroll
Improving the CEX
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
Why Are Some Surveys/Countries Better Than Others?
My Best Guess
CE vs NIPA
Sweden
 for
 
      
130
125
120
NIPA
SEK 1000
115
110
105
100
95
HUT
90
1984
1986
1988
1990
1992
1994
1996
1998
2000
              
         
       
Carroll
Improving the CEX
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
Why Are Some Surveys/Countries Better Than Others?
My Best Guess
CE vs NIPA for UK
Figure 2.6: Income and Consumption Coverage
1
0.95
Coverag
0.9
0.85
0.8
0.75
0.7
Income Coverage
2004
2002
2000
1998
1996
1994
1992
1990
1988
1986
1984
1982
1978
0.6
1980
Consumption Coverage
0.65
Year
In order to try to understand what may be behind the declining performance of
the expenditure data, it is worth looking
of thethecomponents
behind the
Carrollat some
Improving
CEX
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
Why Are Some Surveys/Countries Better Than Others?
My Best Guess
Commonalities
In Many Countries, Coverage Has Declined Seriously
In Some, Deterioration Is Accelerating
Carroll
Improving the CEX
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
Why Are Some Surveys/Countries Better Than Others?
My Best Guess
Commonalities
In Many Countries, Coverage Has Declined Seriously
In Some, Deterioration Is Accelerating
Carroll
Improving the CEX
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
Why Are Some Surveys/Countries Better Than Others?
My Best Guess
Differences:
In Person Vs Phone Interviews
Diary Vs Recall
Incentives To Participate
Quality of Interviewers
Quality of Survey (Focus Groups!)
Use Of Budget Constraint
Carroll
Improving the CEX
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
Why Are Some Surveys/Countries Better Than Others?
My Best Guess
Differences:
In Person Vs Phone Interviews
Diary Vs Recall
Incentives To Participate
Quality of Interviewers
Quality of Survey (Focus Groups!)
Use Of Budget Constraint
Carroll
Improving the CEX
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
Why Are Some Surveys/Countries Better Than Others?
My Best Guess
Differences:
In Person Vs Phone Interviews
Diary Vs Recall
Incentives To Participate
Quality of Interviewers
Quality of Survey (Focus Groups!)
Use Of Budget Constraint
Carroll
Improving the CEX
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
Why Are Some Surveys/Countries Better Than Others?
My Best Guess
Differences:
In Person Vs Phone Interviews
Diary Vs Recall
Incentives To Participate
Quality of Interviewers
Quality of Survey (Focus Groups!)
Use Of Budget Constraint
Carroll
Improving the CEX
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
Why Are Some Surveys/Countries Better Than Others?
My Best Guess
Differences:
In Person Vs Phone Interviews
Diary Vs Recall
Incentives To Participate
Quality of Interviewers
Quality of Survey (Focus Groups!)
Use Of Budget Constraint
Carroll
Improving the CEX
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
Why Are Some Surveys/Countries Better Than Others?
My Best Guess
Differences:
In Person Vs Phone Interviews
Diary Vs Recall
Incentives To Participate
Quality of Interviewers
Quality of Survey (Focus Groups!)
Use Of Budget Constraint
Carroll
Improving the CEX
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
Why Are Some Surveys/Countries Better Than Others?
My Best Guess
Differences:
In Person Vs Phone Interviews
Diary Vs Recall
Incentives To Participate
Quality of Interviewers
Quality of Survey (Focus Groups!)
Use Of Budget Constraint
Carroll
Improving the CEX
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
Why Are Some Surveys/Countries Better Than Others?
My Best Guess
Biggest Problem: Increased Reluctance to Participate
Differential Sample Selection: Richer ⇒ Less Willing
Second Biggest: Changes In Technology (Automated Payment, Etc)
If You Asked Me These Questions, I’d Have No Idea!
I’m Concerned You’re Drilling A Drying Up Hole
Carroll
Improving the CEX
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
Why Are Some Surveys/Countries Better Than Others?
My Best Guess
Biggest Problem: Increased Reluctance to Participate
Differential Sample Selection: Richer ⇒ Less Willing
Second Biggest: Changes In Technology (Automated Payment, Etc)
If You Asked Me These Questions, I’d Have No Idea!
I’m Concerned You’re Drilling A Drying Up Hole
Carroll
Improving the CEX
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
Why Are Some Surveys/Countries Better Than Others?
My Best Guess
Biggest Problem: Increased Reluctance to Participate
Differential Sample Selection: Richer ⇒ Less Willing
Second Biggest: Changes In Technology (Automated Payment, Etc)
If You Asked Me These Questions, I’d Have No Idea!
I’m Concerned You’re Drilling A Drying Up Hole
Carroll
Improving the CEX
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
Why Are Some Surveys/Countries Better Than Others?
My Best Guess
Biggest Problem: Increased Reluctance to Participate
Differential Sample Selection: Richer ⇒ Less Willing
Second Biggest: Changes In Technology (Automated Payment, Etc)
If You Asked Me These Questions, I’d Have No Idea!
I’m Concerned You’re Drilling A Drying Up Hole
Carroll
Improving the CEX
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
Why Are Some Surveys/Countries Better Than Others?
My Best Guess
Biggest Problem: Increased Reluctance to Participate
Differential Sample Selection: Richer ⇒ Less Willing
Second Biggest: Changes In Technology (Automated Payment, Etc)
If You Asked Me These Questions, I’d Have No Idea!
I’m Concerned You’re Drilling A Drying Up Hole
Carroll
Improving the CEX
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
Nielsen Data
Credit Card Data
Personal Finance Online
QuickenOnline
iBank
http://www.mint.com/product/videos/mint-demo/
Big Advantage: People Get Something Valuable At the End
50 percent chance that current structure should be ditched altogether
Carroll
Improving the CEX
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
Nielsen Data
Credit Card Data
Personal Finance Online
QuickenOnline
iBank
http://www.mint.com/product/videos/mint-demo/
Big Advantage: People Get Something Valuable At the End
50 percent chance that current structure should be ditched altogether
Carroll
Improving the CEX
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
Nielsen Data
Credit Card Data
Personal Finance Online
QuickenOnline
iBank
http://www.mint.com/product/videos/mint-demo/
Big Advantage: People Get Something Valuable At the End
50 percent chance that current structure should be ditched altogether
Carroll
Improving the CEX
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
Nielsen Data
Credit Card Data
Personal Finance Online
QuickenOnline
iBank
http://www.mint.com/product/videos/mint-demo/
Big Advantage: People Get Something Valuable At the End
50 percent chance that current structure should be ditched altogether
Carroll
Improving the CEX
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
Nielsen Data
Credit Card Data
Personal Finance Online
QuickenOnline
iBank
http://www.mint.com/product/videos/mint-demo/
Big Advantage: People Get Something Valuable At the End
50 percent chance that current structure should be ditched altogether
Carroll
Improving the CEX
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
Nielsen Data
Credit Card Data
Personal Finance Online
QuickenOnline
iBank
http://www.mint.com/product/videos/mint-demo/
Big Advantage: People Get Something Valuable At the End
50 percent chance that current structure should be ditched altogether
Carroll
Improving the CEX
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
Nielsen Data
Credit Card Data
Personal Finance Online
QuickenOnline
iBank
http://www.mint.com/product/videos/mint-demo/
Big Advantage: People Get Something Valuable At the End
50 percent chance that current structure should be ditched altogether
Carroll
Improving the CEX
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
Nielsen Data
Credit Card Data
Personal Finance Online
QuickenOnline
iBank
http://www.mint.com/product/videos/mint-demo/
Big Advantage: People Get Something Valuable At the End
50 percent chance that current structure should be ditched altogether
Carroll
Improving the CEX
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
Nielsen Data
Credit Card Data
Personal Finance Online
QuickenOnline
iBank
http://www.mint.com/product/videos/mint-demo/
Big Advantage: People Get Something Valuable At the End
50 percent chance that current structure should be ditched altogether
Carroll
Improving the CEX
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
Nielsen Data
Credit Card Data
Personal Finance Online
QuickenOnline
iBank
http://www.mint.com/product/videos/mint-demo/
Big Advantage: People Get Something Valuable At the End
50 percent chance that current structure should be ditched altogether
Carroll
Improving the CEX
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
Comparison of CEX Data With Other Sources Should Be Routinized
Every CE Report Should Show Comparisons With Other Sources, Say:
NIPA
Nielsen
CPS
Biannual Review Report Should Contain Broader Comparisons
SCF, Labor Market, SOI Tax Income, and “special topics”
Explain What It Should/Should Not Be Used For
Occasional External Reviews, Not Just Internal
Carroll
Improving the CEX
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
Comparison of CEX Data With Other Sources Should Be Routinized
Every CE Report Should Show Comparisons With Other Sources, Say:
NIPA
Nielsen
CPS
Biannual Review Report Should Contain Broader Comparisons
SCF, Labor Market, SOI Tax Income, and “special topics”
Explain What It Should/Should Not Be Used For
Occasional External Reviews, Not Just Internal
Carroll
Improving the CEX
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
Comparison of CEX Data With Other Sources Should Be Routinized
Every CE Report Should Show Comparisons With Other Sources, Say:
NIPA
Nielsen
CPS
Biannual Review Report Should Contain Broader Comparisons
SCF, Labor Market, SOI Tax Income, and “special topics”
Explain What It Should/Should Not Be Used For
Occasional External Reviews, Not Just Internal
Carroll
Improving the CEX
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
Comparison of CEX Data With Other Sources Should Be Routinized
Every CE Report Should Show Comparisons With Other Sources, Say:
NIPA
Nielsen
CPS
Biannual Review Report Should Contain Broader Comparisons
SCF, Labor Market, SOI Tax Income, and “special topics”
Explain What It Should/Should Not Be Used For
Occasional External Reviews, Not Just Internal
Carroll
Improving the CEX
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
Comparison of CEX Data With Other Sources Should Be Routinized
Every CE Report Should Show Comparisons With Other Sources, Say:
NIPA
Nielsen
CPS
Biannual Review Report Should Contain Broader Comparisons
SCF, Labor Market, SOI Tax Income, and “special topics”
Explain What It Should/Should Not Be Used For
Occasional External Reviews, Not Just Internal
Carroll
Improving the CEX
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
Comparison of CEX Data With Other Sources Should Be Routinized
Every CE Report Should Show Comparisons With Other Sources, Say:
NIPA
Nielsen
CPS
Biannual Review Report Should Contain Broader Comparisons
SCF, Labor Market, SOI Tax Income, and “special topics”
Explain What It Should/Should Not Be Used For
Occasional External Reviews, Not Just Internal
Carroll
Improving the CEX
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
Comparison of CEX Data With Other Sources Should Be Routinized
Every CE Report Should Show Comparisons With Other Sources, Say:
NIPA
Nielsen
CPS
Biannual Review Report Should Contain Broader Comparisons
SCF, Labor Market, SOI Tax Income, and “special topics”
Explain What It Should/Should Not Be Used For
Occasional External Reviews, Not Just Internal
Carroll
Improving the CEX
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
Comparison of CEX Data With Other Sources Should Be Routinized
Every CE Report Should Show Comparisons With Other Sources, Say:
NIPA
Nielsen
CPS
Biannual Review Report Should Contain Broader Comparisons
SCF, Labor Market, SOI Tax Income, and “special topics”
Explain What It Should/Should Not Be Used For
Occasional External Reviews, Not Just Internal
Carroll
Improving the CEX
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
Comparison of CEX Data With Other Sources Should Be Routinized
Every CE Report Should Show Comparisons With Other Sources, Say:
NIPA
Nielsen
CPS
Biannual Review Report Should Contain Broader Comparisons
SCF, Labor Market, SOI Tax Income, and “special topics”
Explain What It Should/Should Not Be Used For
Occasional External Reviews, Not Just Internal
Carroll
Improving the CEX
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
Comparison of CEX Data With Other Sources Should Be Routinized
Every CE Report Should Show Comparisons With Other Sources, Say:
NIPA
Nielsen
CPS
Biannual Review Report Should Contain Broader Comparisons
SCF, Labor Market, SOI Tax Income, and “special topics”
Explain What It Should/Should Not Be Used For
Occasional External Reviews, Not Just Internal
Carroll
Improving the CEX
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
Comparison of CEX Data With Other Sources Should Be Routinized
Every CE Report Should Show Comparisons With Other Sources, Say:
NIPA
Nielsen
CPS
Biannual Review Report Should Contain Broader Comparisons
SCF, Labor Market, SOI Tax Income, and “special topics”
Explain What It Should/Should Not Be Used For
Occasional External Reviews, Not Just Internal
Carroll
Improving the CEX
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
And Collaboration
Openness, Transparency, Accountability
How Do These Buzzwords Apply?
Unfolding Brackets of Complexity Of Data Use
Very Easy To Get Top Level Data
Clear Path to “Drill Down” to Deeper and Deeper Levels
A Lot Of Researchers Have Been Defeated By Complexity
Experiment, e.g., With HRS or PSID Instrument On CE Sample
Carroll
Improving the CEX
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
And Collaboration
Openness, Transparency, Accountability
How Do These Buzzwords Apply?
Unfolding Brackets of Complexity Of Data Use
Very Easy To Get Top Level Data
Clear Path to “Drill Down” to Deeper and Deeper Levels
A Lot Of Researchers Have Been Defeated By Complexity
Experiment, e.g., With HRS or PSID Instrument On CE Sample
Carroll
Improving the CEX
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
And Collaboration
Openness, Transparency, Accountability
How Do These Buzzwords Apply?
Unfolding Brackets of Complexity Of Data Use
Very Easy To Get Top Level Data
Clear Path to “Drill Down” to Deeper and Deeper Levels
A Lot Of Researchers Have Been Defeated By Complexity
Experiment, e.g., With HRS or PSID Instrument On CE Sample
Carroll
Improving the CEX
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
And Collaboration
Openness, Transparency, Accountability
How Do These Buzzwords Apply?
Unfolding Brackets of Complexity Of Data Use
Very Easy To Get Top Level Data
Clear Path to “Drill Down” to Deeper and Deeper Levels
A Lot Of Researchers Have Been Defeated By Complexity
Experiment, e.g., With HRS or PSID Instrument On CE Sample
Carroll
Improving the CEX
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
And Collaboration
Openness, Transparency, Accountability
How Do These Buzzwords Apply?
Unfolding Brackets of Complexity Of Data Use
Very Easy To Get Top Level Data
Clear Path to “Drill Down” to Deeper and Deeper Levels
A Lot Of Researchers Have Been Defeated By Complexity
Experiment, e.g., With HRS or PSID Instrument On CE Sample
Carroll
Improving the CEX
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
And Collaboration
Openness, Transparency, Accountability
How Do These Buzzwords Apply?
Unfolding Brackets of Complexity Of Data Use
Very Easy To Get Top Level Data
Clear Path to “Drill Down” to Deeper and Deeper Levels
A Lot Of Researchers Have Been Defeated By Complexity
Experiment, e.g., With HRS or PSID Instrument On CE Sample
Carroll
Improving the CEX
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
And Collaboration
Encourage Researchers With Setup Like Census Data Centers
Carroll
Improving the CEX
It’s A Global Problem
New Technologies
Checks and Balances
Openness, Transparency, Accountability
Conclusion
A Golden Age!
CE Should Be A Part Of It, Not A Drowning Polar Bear
Carroll
Improving the CEX