National WAP
Evaluation:
Single Family
and Mobile
Home Energy
Impacts
Michael Blasnik
M Blasnik & Associates
Greg Dalhoff
Dalhoff Associates, LLC
David Carroll
APPRISE
1
Managed by UT-Battelle
for the Department of Energy
Presentation Overview
 Purpose
 Measurement and Analysis Procedures
 Findings for Homes with Natural Gas Main Heat
 Findings for Homes with Electric Main Heat
 Analysis – Next Steps
 Findings for Homes with Fuel Oil Main Heat
2
Managed by UT-Battelle
for the Department of Energy
Purpose
 Energy Performance – Document energy savings and
cost-effectiveness.
 Program Performance – Foundation for documenting
all program benefits and costs.
 Diagnostic – Assessment of what works best under
what conditions.
3
Managed by UT-Battelle
for the Department of Energy
MEASUREMENT AND
ANALYSIS PROCEDURES
4
Managed by UT-Battelle
for the Department of Energy
Primary Objective
 What was the usage of the home prior to
weatherization?
 What services were delivered to the targeted housing
unit and household?
 What is the usage of the home after weatherization?
5
Managed by UT-Battelle
for the Department of Energy
Study Scope
 Primary Focus
– Detailed analysis of Program Year 2008
 WX Program Year – 4/2008 to 3/2009
 State Program Year – 7/2008 to 6/2009
 Supplemental Information
– Usage analysis only for Program Year 2007
 Preliminary Information
– Usage analysis for clients served in the first half of Program
Year 2009
6
Managed by UT-Battelle
for the Department of Energy
Logistical Challenges
 What clients were served by the program?
– Collection of client account information from 51 grantees
and 400 subgrantees for PY 2007, 2008, and 2009 clients
 What services did those clients receive?
– Collection of detailed information on service delivery for
program year 2008 for about 19,000 clients
 What is the energy usage of the home before and after
weatherization?
– Collection of usage data for 57,000 clients from 4/1/2006
through 3/31/2011.
7
Managed by UT-Battelle
for the Department of Energy
Usage Data Requirements
 PY 2008 Clients
– Weatherized between 4/2008 and 6/2009
– Pre-weatherization usage = 12 months prior to
weatherization (as early as 4/2007 through 3/2008)
– Post-weatherization usage = 12 months after weatherization
(as late as 7/2009 through 6/2010)
– Data required for analysis of PY 2008 from April 2007
through June 2010 = 39 Months of Usage Data
 PY 2007 Clients – Need data from 4/06 through 6/09
 PY 2009 Clients – Need data from 4/08 through 6/11
8
Managed by UT-Battelle
for the Department of Energy
Usage Data Collection
 Natural Gas Main Heat
–
–
–
–
Sample of 15,000 clients per program year
Total sample of 45,000 clients for PY 07, PY 08, and PY 09
Requested data from 368 gas utilities for 45,000 clients
Received data from 71% of utilities for 30,000 clients (67%)
 Natural Gas and Electric Main Heat
– Sample of 19,000 clients per program year
– Total sample of 57,000 clients for PY 07, PY 08, and PY 09
– Requested data from 984 electric suppliers for 57,000 clients
– Received data from 74% of utilities for 37,000 clients (67%)
9
Managed by UT-Battelle
for the Department of Energy
Analysis Challenges
 Differences in Weather from Pre-Program Year to PostProgram Year
– Use of PRISM to compare “Weather Normalized”
consumption for the two periods
 Other factors affecting low income households
– Use of a Comparison Group
 PY 2008 clients serve as a comparison group for PY 2007 analysis
 PY 2009 clients serve as a comparison group for PY 2009 analysis
 Attrition from incomplete data or inconsistent data
– Use of ORNL model
– Use of Fixed Effects regression model
10
Managed by UT-Battelle
for the Department of Energy
Analysis Completeness
 How does Weatherization affect the quality of the
housing unit?
– Indoor Air Quality Field Study
 How does Weatherization affect clients?
– Indoor Air Quality Field Study Occupant Survey
– Program-Wide Occupant Survey
 What is the overall benefit of the Program?
– Estimation of NonEnergy Benefits
11
Managed by UT-Battelle
for the Department of Energy
FINDINGS FOR HOMES
WITH NATURAL GAS MAIN
HEATING FUEL
12
Managed by UT-Battelle
for the Department of Energy
National WAP Energy Impacts
Gas Heated Single Family
Clients with
Good Data
Natural Gas
Gross Impact
4,113
PreWX
Usage
Savings
therms
therms
980
Net Impact
Electric
Gross Impact
Net Impact
3,321
Percent
savings
First Year
$$
170
17.3%
$206
155
15.8%
$188
kWh
kWh
9,513
748
7.9%
$74
527
5.5%
$52
Total First Year $$
13
Gross
$280
Net
$240
Managed by UT-Battelle
for the Department of Energy
Climate Zones
14
Managed by UT-Battelle
for the Department of Energy
Climate Zone Energy Impacts
Gas Heated Single Family
PreWX
Usage
Natural Gas
Percent
Savings
therms
Net
Savings
Percent
Savings
therms
Very Cold
1,038
183
17.6%
163
15.7%
Cold
1,063
194
18.3%
177
16.7%
Moderate
815
122
15.0%
121
14.8%
Hot/Wet
627
89
14.2%
77
12.3%
Electric
15
Gross
Savings
kWh
kWh
Very Cold
9,347
898
9.6%
740
7.9%
Cold
9,125
654
7.2%
589
6.5%
Moderate
11,177
880
7.9%
490
4.4%
Hot/Wet
12,448
649
5.2%
592
4.8%
Managed by UT-Battelle
for the Department of Energy
Comparing Energy Impacts
Gas Heated Single Family
PreWX
Usage
Natural Gas
Percent
Savings
therms
Net
Savings
Percent
Savings
therms
2008
980
170
17.3%
155
15.8%
1989
1,340
135
10.1%
170
13.0%
1981
1,502
150
10.0%
N/A
N/A
Electric
16
Gross
Savings
kWh
2008
9,513
1989
N/A
1981
N/A
Managed by UT-Battelle
for the Department of Energy
748
kWh
7.8%
527
5.5%
Impacts for Top 25% / Agency
Gas Heated Single Family
Natural Gas
Net Impact
Electric
Net Impact
PreWX
Usage
Savings
therms
therms
1,164
277
kWh
kWh
11,030
1,787
Percent
savings
First Year
$$
23.8%
$336
16.2%
$176
Total First Year $$
Net
17
Managed by UT-Battelle
for the Department of Energy
$512
Policy Note
 Homes with the highest preWX usage save the most
 In 1981, the AVERAGE preWX gas usage was 1,500
therms
 Statistics from the 2005 RECS
– Low-income households in gas single family = 6.5 million
– Use 1200 or more therms = 820,000 (12%)
– Use 1600 or more therms = 240,000 (3%)
 Projected savings for 2008 on preWX usage of 1,340
therms = 250 therms; on 1,500 therms = 300 therms
18
Managed by UT-Battelle
for the Department of Energy
FINDINGS FOR HOMES
WITH ELECTRIC MAIN
HEATING FUEL
19
Managed by UT-Battelle
for the Department of Energy
National WAP Energy Impacts
Electric Heat Single Family
Clients with
Good Data
Electric
Gross Impact
Net Impact
20
Managed by UT-Battelle
for the Department of Energy
702
PreWX
Usage
Savings
Percent
savings
First Year
$$
kWh
kWh
19,551
1,987
10.2%
$172
1,706
8.7%
$148
Climate Zone Energy Impacts
Electric Heat Single Family
PreWX
Usage
Electric
21
Gross
Savings
Percent
Savings
kWh
Net
Savings
Percent
Savings
kWh
Very Cold
20,769
1,992
9.6%
1,518
7.3%
Cold
22,680
2,771
12.2%
3,028
13.4%
Moderate
18,536
1,742
9.4%
908
4.9%
Hot/Wet
18,240
1,877
10.3%
2,579
14.1%
Managed by UT-Battelle
for the Department of Energy
Comparing Energy Impacts
Electric Heat Single Family
PreWX
Usage
Electric
22
Gross
Savings
Percent
Savings
kWh
Net
Savings
Percent
Savings
kWh
2008
19,551
1,987
10.1%
1,706
8.7%
1989
14,972
867
5.8%
1,830
12.2%
1981
N/A
N/A
N/A
N/A
N/A
Managed by UT-Battelle
for the Department of Energy
WHAT’S NEXT?
23
Managed by UT-Battelle
for the Department of Energy
Next Steps – Impact Estimates
 Other Periods - PY 2007 and PY 2009
 Other Models
– ORNL – Reduces sample attrition
– Fixed Effects Regression – Different analytic framework
 Longer Term Analysis
– PY 2007 – 3 years post program analysis (2008, 2009, 2010)
– PY 2008 – 2 years post program analysis (2009, 2010)
 PY 2009 – 2 years pre program analysis
– 2008 to 2009 change (reported gross to net adjustment)
– 2007 to 2008 change (potential gross to net adjustment)
24
Managed by UT-Battelle
for the Department of Energy
Next Steps – Diagnostics
 Factors Associated with Higher Savings
–
–
–
–
Pre-Program usage
Pre-Program housing unit conditions
Installed measures
Program factors
 Audit procedures
 Training investment
 Quality control procedures
25
Managed by UT-Battelle
for the Department of Energy
Next Steps – Cost Effectiveness
 Document first year savings
 Project savings over time based on measure life and
price projections
 Estimate net present value of savings
 Compare to installation costs
 Compare to total program costs
26
Managed by UT-Battelle
for the Department of Energy
FINDINGS FOR HOMES
WITH FUEL OIL MAIN
HEATING FUEL
27
Managed by UT-Battelle
for the Department of Energy
Fuel Oil Homes – Data Collection
and Analysis Strategy
 Sample agencies serving clients with fuel oil main heat
 Select a sample of 76 treatment and 52 control clients
 October 2010 – PreWX tests, meter homes
 January 2011 – Weatherize homes
 April 2011 – PostWX tests, retrieve equipment
 Analysis – Estimate savings based on metered data
28
Managed by UT-Battelle
for the Department of Energy
National WAP Energy Impacts
Fuel Oil Heat Single Family
Winter 2010/2011 Treatment
Fuel
Oil
Gross Impact
Net Impact
29
Managed by UT-Battelle
for the Department of Energy
PreWX
Usage
Savings
Percent
savings
First Year
$$
therm
equivalent
therm
equivalent
1,050
221
21.0%
$560
266
23.1%
$674
@$3.50 per
gallon