Electric / Gas / Water
2003 Statewide Nonresidential Audit
Program Evaluation
Summary of Final Evaluation Report
Prepared by:
Rafael Friedmann, PG&E
Kris Bradley & Christie Torok, Quantum Consulting
Evaluation Objectives
• Tracking System Assessment
– Verify tracking system accomplishments versus goals and
evaluate tracking system content
• Small Company Impact and Satisfaction Assessment
– Measure first year program impact and customer
satisfaction with program components
• Large Company Impact Assessment
– Measure program-related energy efficiency actions taken
over the first two years following the Audit
• Best Practices Assessment
– Provide an up-to-date, prospective recommendation for the
best approach to on-site audits for different sized
customers.
Knowledge to Shape Your Future
2
Program Targets and Accomplishments
• Statewide the Audit program delivered audit services to
over 29,000 utility customers, with over ½ of the
completed audits provided to customers defined as hardto-reach (HTR)
PG&E
SCE
SDG&E
SCG
Total
Knowledge to Shape Your Future
Total Participation
Q4 Report
Goals
11,865
6,500
8,533
7,100
5,063
5,000
4,283
4,000
29,744
22,600
Hard-to-Reach Participation
Q4 Report
Goals
9,059
3,500
3,963
2,840
2,621
800
475
400
16,118
7,540
3
Data Collection Summary
• Smaller Customer Surveys (Less Than 100 kW)
–
–
–
–
259 Participant / 261 Nonparticipant
Customers with Less Than 100 kW
PY 2003 Participants
Phone, Mail, On-Site, CD-ROM, On-Line
• Larger Customer Surveys (More Than 100 kW)
–
–
–
–
84 Participant / 86 Nonparticipant
Customers with More Than 100 kW
PY 2002 Participants
On-Site Only
• Program Manager and Implementation Staff Interviews
Knowledge to Shape Your Future
4
Rates of Audit Program Awareness in the General
Population
• 25% of small and very small nonparticipating customers
are aware of the audit program, versus about 40% of
medium/large customers
45%
40%
35%
30%
25%
42%
20%
37%
15%
25%
25%
Very Small
(N=129)
Small
(N=132)
10%
5%
0%
Medium
(N=37)
Large
(N=49)
Knowledge to Shape Your Future
*Medium and large participants are PY2002 and onsite; very small and small participants are PY2003 and include all delivery mechanisms
5
Utility Marketing Sources of Program Awareness in the
General Population
• Awareness of the Audit Program in the general
population is driven by the IOUs, who account for 70
percent of overall awareness
100%
10%
90%
3%
23%
29%
--1%
--2%
28%
80%
7%
3%
--2%
70%
--1%
Other
12%
60%
18%
22%
50%
24%
Bill Inserts
84%
40%
30%
Respondent
Contacted Utility
38%
50%
Utility Brochure in
Mail
20%
34%
Contacted by Utility
Representative
10%
9%
0%
Very Small and Small
Participants
(N=259)
Knowledge to Shape Your Future
Very Small and Small
Nonparticipants
(N=59)
Medium and Large
Participants
(N=84)
Medium and Large
Nonparticipants
(N=33)
*Medium and large participants are PY2002 and onsite; very small and small participants are PY2003 and include all delivery mechanisms
6
Satisfaction with Audit Program Elements Among Small and
Very Small Participants
• Satisfaction levels with the Audit program are reasonably
high, but participants desire a more useful audit product
(report)
100%
90%
80%
--2%
5%
19%
--2%
15%
16%
Unsatisfied
(1-3)
Somewhat
Satisfied (4-7)
27%
Very Satisfied
(8-10)
70%
32%
N=259
60%
50%
40%
70%
74%
65%
30%
46%
20%
10%
0%
Overall Audit Program
Knowledge to Shape Your Future
Quality of Report
Credibility of
Recommendations
Usefulness
7
Self-Reported Knowledge of Energy Efficiency
• Participants show a measurable increase in energy
efficiency knowledge resulting from the audit
7
(N=259)
Nonparticipants
(N=84)
(N=86)
6
Participants
- Before Audit
Mean Knowledge Rating (1-10 scale)
(N=255)
(N=261)
(N=84)
5
Participants
- After Audit
4
3
2
1
Small and Very Small
Medium and Large
*Medium and large participants are PY2002 and onsite; very small and small participants are PY2003 and include all delivery mechanisms
Knowledge to Shape Your Future
8
Rebate Program Participation Levels
• Audit participants purchase rebated equipment at much
higher rates than nonparticipants
60%
SPC
Express Efficiency
Unspecified Rebate Program
50%
40%
30%
20%
10%
0%
Small/Very Small Participant
(N=259)
Knowledge to Shape Your Future
Small/Very Small
Nonparticipant (N=261)
Medium/Large Participant
(N=84)
Medium/Large
Nonparticipant (N=86)
9
Participant Suggestions for Program Improvement
• Participants want more customization in the audit reports
(and recommendations), and want measure costs and
rebates for use in ROI calculations
100%
Less information
90%
More communication &
information
80%
70%
More personal contact
60%
Provide results faster
50%
40%
Email info on rebates, EE
30%
20%
Give costs to replace
equipment
10%
More customization
0%
Very Small and Small Participants
(N=85)
Knowledge to Shape Your Future
Medium and Large Participants
(N=15)
*Medium and large participants are PY2002 and onsite; very small and small participants are PY2003 and include all delivery mechanisms
10
Small and Very Small Customer Equipment Adoption Rates
• Among smaller customers, participant adoption rates for
lighting and cooling, as well as ‘other’ miscellaneous
equipment, are considerably higher than for a
nonparticipant control group
35%
32%
Participant (N=259)
Nonparticipant (N=261)
30%
24%
Adoption Rate
25%
20%
20%
17%
15%
12%
10%
12%
9%
9%
5%
0%
Knowledge to Shape Your Future
Lighting
Cooling
Gas
Other
11
Medium and Large Customer Equipment Adoption Rates
• Among larger customers, when comparing participants
and a nonparticipant control group, marked differences
in adoption rates exist for industrial process, lighting,
cooling, and gas equipment
60%
Participant
Nonparticipant
N=51
50%
N=84
N=84
N=39
Adoption Rate
40%
30%
N=86
N=86
N=33
20%
N=47
N=86
N=84
10%
0%
Knowledge to Shape Your Future
Lighting
Cooling
Gas
Other
Industrial Process
12
Percent of Equipment Adoptions Recommended in the Audit Report
• Recommendations in the audit reports clearly leads to
equipment adoptions, especially for lighting measures
100%
90%
Percent of Equipment Adoptions
80%
70%
60%
Don't know
50%
No
Yes
40%
30%
20%
10%
Lighting
Cooling
Gas Appliances
Other Equipment
Medium and
Large
(N=24)
Medium and
Large
(N=8)
Very Small
and Small
(N=44)
Medium and
Large
(N=6)
Very Small
and Small
(N=17)
Medium and
Large
(N=29)
Very Small
and Small
(N=52)
Medium and
Large
(N=31)
Very Small
and Small
(N=66)
0%
Process
Equipment
*Medium and large participants are PY2002 and onsite; very small and small participants are PY2003 and include all delivery mechanisms
Knowledge to Shape Your Future
13
Implementation of Conservation Measures by Customer Size
• Counter to PY2002 evaluation results, Audit influence on
conservation measure adoptions is substantial
90%
Participant
Nonparticipant
N=32
80%
Percent Implementing Conservation Measures
N=190
N=69
70%
N=129
N=52
N=132
60%
N=37
N=49
50%
40%
30%
20%
10%
0%
Very Small
Knowledge to Shape Your Future
Small
Medium
Large
14
Percent of Conservation Measures Begun as a Result of the Audit
• Self-reported conservation practices begun as a result of
the audit ranges from 46 percent among the very small
to about 25 percent for the largest customers
50%
Percent of Conservation Measures Begun as a Result of the Audit
N=139
45%
N=51
40%
N=38
35%
30%
N=35
25%
20%
15%
10%
5%
0%
Very Small
Knowledge to Shape Your Future
Small
Medium
Large
*Medium and large participants are PY2002 and onsite; very small and small participants are PY2003 and include all delivery mechanisms
15
Tailor On-Site Audit Delivery by Customer Size
• Program outreach – program goals should drive the
focus of the outreach effort to obtain the desired mix of
small, medium and large customers
Customer Size
On-Site Audit Approach
Set goals based on number of
completed audits
Implement audit outreach to Express
Efficiency participants
Set goals based on energy saving
targets
Knowledge to Shape Your Future
Very Small
(<20 kW)
Small
(20 - <100 kW)
•
•
•
Medium
(100 - <500 kW)
Large
(>500 kW)
•
•
•
16
Tailor On-Site Audit Delivery by Customer Size (Cont’d)
• Site Visit – for smaller customers all audit services
should seek to maximize energy efficient equipment
adoptions and Express Efficiency participation
Customer Size
On-Site Audit Approach
Deliver the audit report during the site
visit
Provide an Express Efficiency
application; filled-in where possible
Furnish a list of qualified service
providers
Knowledge to Shape Your Future
Very Small
(<20 kW)
Small
(20 - <100 kW)
Medium
(100 - <500 kW)
•
•
•
•
•
•
•
Large
(>500 kW)
17
Tailor On-Site Audit Delivery by Customer Size (Cont’d)
• Audit Report and Recommendations – tailor audit
products to customer wants and needs by customer
size
Customer Size
On-Site Audit Approach
Smaller customers need educational
information on relevant measures
Smaller customers need measures
ranked by the cost to the bottom line
Larger customers need customized
information relevant to their building
Larger customers need measures
ranked by ROI to justify investment
Knowledge to Shape Your Future
Very Small
(<20 kW)
Small
(20 - <100 kW)
•
•
•
•
Medium
(100 - <500 kW)
Large
(>500 kW)
•
•
•
•
18
Tailor On-Site Audit Delivery by Customer Size (Cont’d)
• Audit Follow-up – motivate account representatives to
follow-up using job performance incentives for achieving
program savings and equitable spending targets
Customer Size
On-Site Audit Approach
Track account rep. performance using
completed Express applications
In-depth follow-up efforts with larger
customers to maximize savings
Knowledge to Shape Your Future
Very Small
(<20 kW)
Small
(20 - <100 kW)
•
•
Medium
(100 - <500 kW)
Large
(>500 kW)
•
•
19
Program Implementation for Smaller Customers
• Overall, a direct install program may be a preferable
delivery channel for bringing energy efficiency to the
under 20 kW market
• Alternatively, providing a turnkey program that combines
Express Efficiency and Audit services may improve
program results for very small customers
• Outreach efforts to small customers should emphasize
remote Audits
• Emphasize low-cost and no-cost recommendations in
Audits directed at smaller customers
Knowledge to Shape Your Future
20
Program Implementation for Larger Customers
• Continue to direct on-site audits to larger customers
• Provide in-depth follow-up services to maximize
recommended measure uptake and support for the
Express Efficiency and SPC rebate programs
• Provide well-documented, detailed return-on-investment
(ROI) calculations in the Audit report
• Include significant analytic customization and sitespecific research in Audit reports for large customers
Knowledge to Shape Your Future
21
Other Program Implementation Recommendations
• Provide Audit program outreach services using previous
years’ Express Efficiency participants
• With the exception of SoCalGas, all other IOUs should
increase Audit emphasis on the gas appliance end-use
category
• Use outreach or follow-up strategies to improve
customer utilization of the CD-ROM Audit tool
• Assess program marketing effectiveness and emphasize
best practices
• Consider tracking goals based on downstream energy
efficiency uptake
Knowledge to Shape Your Future
22