Residential Program Studies



RLPNC 17-6 Market Adoption Model Findings Memo
RLPNC_17-6 Reflector Market Adoption Model
RLPNC 17-6 GSL Market Adoption Model
RLPNC 17-6 Specialty Market Adoption Model

 

Reported Leave-Behind APS Net of Free-riderhsip and In-Service Rates Memo

 

CLC and National Grid Education Kits Deemed Savings Review Memo

 

National Grid 2017 Residential Wi-Fi Thermostat DR Evaluation

 

National 2017 Seasonal Savings Evaluation

 

RES 28: Task 4 – Ductless Mini-Split Heat Pump Cost Study

 

RES 19: Task 5 – Water Heating, Boiler, and Furnace Cost Study

 

RES 44: Multi-family Program Impact and Net-to-Gross Evaluation

 

RES 35: Home Energy Services Process Evaluation Comprehensive Report

 

RLPNC 16-11: Lighting Market Scan Memo

 

RLPNC 17-9: Residential Lighting Market Assessment Study

 

RLPNC 16-2: Task 8A-2017 Lighting Supplier Interviews Final Report

 

RLPNC 16-6: Web Scraping Results June 2017-November 2017 Memo

 

RES 40: Moderate Income Market Characterization Survey Findings Final Report

 

TXC 46: Analysis of Immediate Code Compliance Support Initiative Residential Training Surveys on 2015 IECC Memo

 

RLPNC 18-1: Appliance Recycling Database Review and Savings Update Memo

 

RLPNC 17-4 and 17-5: Products Impact Evaluation of In-Service and Short-Term Retention Rates Study

 

RLPNC 17-12: Lighting Decision Making Report

 

RLPNC 16-10: What Is Next for Products – Market Scan

 

RLPNC 17-2 – Single-Family New Construction Mini-Baseline Study

 

2017

RLPNC 16-5 and 17-10 Sales Data Analysis and Modeling Report

 

TCX 46 – Code Compliance Support Initiative Interview Follow-Ups

 

In 2017, as part of the ongoing evaluation of the Massachusetts Code Compliance Support Initiative (CCSI), evaluators conducted follow-up in-depth interviews (IDIs) with 40 individuals who had attended one or more residential classroom training approximately six months earlier. This report summarizes the findings from these interviews.

Key findings include the following: Most respondents made some changes to their work because of the trainings. These changes were predominantly focused on insulation and envelope areas, builder and contractor education, and ventilation. The most useful aspects of the trainings for all respondents were the updates and summaries on code changes. Almost all respondents had shared some of the information from the trainings with other parties; close to one-half also shared the handouts and other materials provided at the trainings. More than one-half of all respondents have recommended the trainings to others previously, and the vast majority said they would recommend the trainings to others in the future. Respondents’ suggestions for improving the trainings include having a stronger focus on specific areas and focusing on best practices for code compliance. Both municipal building code employees and building professionals suggested that offering continuing education credits could increasing training awareness and attendance among builders and contractors.

RES 17-Connected Thermostats and Technology Literature Review Memo

 

This memo summarizes findings from a comprehensive review of literature pertaining to smart thermostats and other connected devices in residential settings. There were several goals for this study:Provide an understanding of the state of existing technologies and their effects on energy consumption, describe the likely direction of emerging and future technologies, and outline potential future effects on home energy use and savings. The research team synthesized its findings from a wide range of sources and distilled key findings to inform the program administrators’ (PA) decision whether to invest in deeper, targeted research on savings attributable to these connected devices.

The review of existing literature on smart thermostats and other connected devices revealed these products are gaining traction in the consumer market, with thermostat market share increasing most rapidly. For this reason and others outlined in this memo, the research team recommends performing a primary research study of smart thermostat energy savings specific to Massachusetts. Energy savings have been identified for smart thermostats in different utility programs across North America, but reports from recent studies are not consistent in their findings across multiple years, climate zones, or thermostat types. Therefore, a primary research study will provide current, geographically and programmatically relevant data to further inform future decisions regarding incentivizing smart thermostat purchases by Massachusetts utility customers. Furthermore, due to the lack of consistent data regarding other connected devices and the relatively sparse market for these products, evaluators recommends not performing a primary research study for such devices at this time. However, as the market for other connected household devices grows and the technology develops, such an updated environment could justify a primary research study in the future.

RLPNC 16-11 Fourth Quarterly Lighting Market Scan Memo

 

This fourth quarterly market scan memo for the RLPNC 16-11 study presents information published in the second quarter of 2017 on the U.S. residential lighting market. This memo is organized by topic, then by chronology, with the newest information presented first, followed by earlier information from previous memos. All sources are from documents published in 2015 or later and, with a few exceptions, based on research conducted in 2014 or later. The market scans focus on information gained from areas outside of Massachusetts.

This Q1 2017 market scan presents one new discussion of the effects of EISA 2020 (Effects of EISA) and three new sets of NTG estimates. Suppliers interviewed for a CT LED NTG evaluation expect that EISA 2020 will cause LED sales to rise. (This is an expectation we have reported in previous market scans.) These suppliers also expect that ENERGY STAR 2.0 will stimulate LED sales while causing CFL sales to gradually decline, and that EISA 2020 will cause halogen sales to decline. The CT LED NTG evaluation also included market share predictions (Sales). With continued program support, suppliers expect that in 2021, LEDs will account for 61% of standard residential bulb sales and 53% of reflector residential bulb sales in CT. However, without continued program support, the suppliers expect LEDs would account for only 45% and 38% of standard and reflector residential bulb sales, respectively. The three new sets of NTG estimates are for 2015 and 2016. The 2015 NTG estimates range from 0.21 to 0.46 for standard CFLs (CA), from 0.31 to 1.1 for specialty CFLs (CA), and from 0.30 to 0.83 for LEDs (CA and CT). The 2016 NTG estimates range from 0.57 to 0.77 for LEDs (CT and WI).

Lighting Insights Final Report of 2016 Activities

 

This report summarizes findings from three primary research tasks conducted in 2016 with lighting manufacturers, high-level retail buyers and store managers (“lighting market actors”) to support the continued assessment and monitoring of the Massachusetts lighting market and the Massachusetts ENERGY STAR® Lighting Program (“the program”). Primary data collection tasks included: In-depth interviews conducted with 14 participating manufacturers and four high-level retail buyers as follow-up to interviews initially conducted in April 2015, CATI surveys completed with 243 store managers participating in the Massachusetts lighting program, and Meetings and informal discussions at the ENERGY STAR Partners Meeting.

The key findings from the three research tasks address market share and sales predictions, price predictions, EISA 2020 implementation scenarios, and how to improve program effectiveness for low-income/renter customers. Complete findings for each research task appear in the main body of this report including in-depth interviews (Section 2), CATI survey (Section 3) and the ENERGY STAR Partners Meeting (Section 4). These research tasks form part of a larger study, Lighting Supplier Interviews and Store Manager Surveys (RLPNC 16-2). This study also includes one additional outstanding task, a second round of lighting supplier in-depth interviews. Considerations based on the research findings are also summarized in this report.

Single-Family Stretch Code Update Compliance and Potential

 

Beginning in January of 2017, the Massachusetts stretch code references the 2015 International Energy Conservation Code (IECC) rather than the 2009 IECC that it was previously based on. The new stretch code requirements are being referred to as the 2015 Stretch Code Update. This study calculates projected statewide compliance estimates for stretch code homes and assesses the projected potential savings from compliance enhancement from single family homes that will be built under the 2015 Stretch Code Update. The Program Administrators (PAs) and Energy Efficiency Advisory Council (EEAC) determined that this study was an opportunity to conduct a simplified analysis, based on pre-existing data collection, to determine if projected stretch code compliance and the associated compliance enhancement savings potential justify further evaluation research.

The 2015 Stretch Code Update resulted in a significant increase in efficiency requirements for stretch code homes. While this report is based on homes that were built under the previous stretch code requirements, we project that the 2015 Stretch Code Update will result in decreased compliance statewide for stretch code homes and an increase in the potential for compliance enhancement. Assuming building practices remain the same, statewide compliance using the MA-REC (Massachusetts Residential Evaluation Contract) methodology is projected to be 90% under the 2015 Stretch Code Update requirements. This compares to a statewide compliance rate of 96% under the previous stretch code requirements. Maximum projected potential energy savings from compliance enhancement are 14% statewide using the MA-REC methodology. This compares to 5% under the previous stretch code requirements. Maximum projected potential energy savings from compliance enhancement range from 13% to 34% using methods that compare HERS scores to energy usage. Evaluators recommend that the PAs continue to provide trainings and support for compliance enhancement in stretch code municipalities and that they measure compliance with homes that were permitted and built under the 2015 Stretch Code Update.

Lighting Market Assessment Consumer Survey and On-Site Saturation Study

 

This report presents results of the 2016-17 Massachusetts Residential Lighting Market Assessment. The study was designed to update estimates of lighting saturation and other critical market indicators, and assess consumers’ knowledge of and interactions with the lighting market in Massachusetts. The data for this study came from randomly sampled web and telephone surreys with residents and on-site lighting inventories of homes in Massachusetts and a comparison area (portions of New York, namely a 40-mile radius around the cities of Albany, Buffalo, Rochester, and Syracuse, as well as all of Westchester County, referred to in this report ass New York) completed between October of 2016 and January of 2017.

Evidence from this study suggests that the Massachusetts programs have had a strong impact on saturation and penetration of LEDs. While consumers in New York are adopting LEDs, LED saturation and penetration rates continue to lag the rates measured in Massachusetts. Further, ENERGY STAR® LEDs (the only type of LEDs supported by program efforts) accounted for seven of the eight-percentage-point difference in LED saturation between the two states. In 2013, Massachusetts and New York had similar levels of CFL saturation (28% and 26%, respectively), but began to diverge between 2013 and 2017; by 2017, CFL saturation in Massachusetts was 25% higher than New York’s saturation 29% vs. 22%). This divergence aligned closely with the New York State Energy Research & Development Authority’s (NYSERDA) decision to cease spiral CFL incentives in 2012 and essentially all upstream lighting incentives in 2014. Over the past three years, we have observed a steady (though not statistically significant) decline in CFL saturation in Massachusetts, down from 33% in 2014 to 29% in 2017. Moving forward, we expect to see this trend accelerate. Additional findings related to socket saturation trends, comparison area trends, ENERGY STAR LEDs, penetrations, and panel trends, as well as recommendations are also included.

RLPNC Sales Data Analysis Task 1f and Task 2c Memo

 

This memo presents trends in light bulb sales and market share in Massachusetts, New York, the nation, and program and non-program areas. The purpose of the effort is to understand the dynamics and direction of the residential lighting market. Evaluators obtained national- and state-level light bulb sales data from IRI and Nielsen via the CREED Initiative. IRI and Nielsen each offer a point-of-sale (POS) and a panel dataset. The POS datasets are composed of sales data reported by selected retailers, while the panel datasets are composed of purchases scanned by consumer panelists. The analyses in this memo include standard and specialty CFL, halogen, incandescent, and LED bulbs, and exclude other bulb types (e.g., linear fluorescents). The memo also includes a brief update on Second and Third Quarter 2016 Bulb Shipments, as reported by the Association of Electrical Equipment and Medical Imaging Manufacturers.

Key findings included the following. 2015 LED market share in Massachusetts was approximately 19%, comparable to all program states (20%) but higher than New York (15%) and non-program states (9%). Longitudinal data suggest that lighting programs successfully boost energy-efficient bulb share, nationally and in MA. ENERGY STAR LED bulb sales are increasing nationally and in MA and NY, although NY lags behind MA. Point estimates of market share vary by data source, but they generally point to the same national lighting trends. General bulb sales and market share trends are consistent across study areas. Total bulb sales per household have been declining across the nation. LED market share of NEMA shipments lost ground to halogens and incandescents in the second quarter of 2016, then rebounded in the third quarter. Comparison of the NEMA shipment and retailer market shares shows a lag for incandescents. The evaluation team suggests that the PAs purchase the same datasets for the 2016 calendar year, if Nielsen can provide greater detail on the ENERGY STAR qualification of the panel data.

RES 16 – HEHE Condensing Equipment Barriers Report
RES 16 -HEHE Condensing Equipment Barriers Summary

 

This study is a follow-on study to the 2014 HEHE Impact Evaluation. The  Massachusetts High-Efficiency Heating Equipment (HEHE) Rebate Program offers prescriptive rebates for the installation of new high-efficiency natural gas heating and water heating equipment. The primary objective of this follow-on study was to evaluate the energy savings and costs associated with installing outdoor reset controls to determine the extent to which these and other controls could mitigate the non-condensing problem identified in the earlier study. The evaluation team chose to focus the field portion of this study on outdoor reset controls because they are the most common control strategy for modulating boiler supply temperature in the U.S.

See summary document above.

RLPNC 16-6 – Lighting Shelf Stocking Report

 

This study included a shelf-stocking and price survey to evaluate the impact of the Mass Save® residential lighting program on consumer retail lighting in Massachusetts. The objectives were to: determine the amount of shelf area dedicated to LED, CFL, halogen, and incandescent lamps in retail stores in Massachusetts and New York; examine the characteristics of these lamps, their prices, and how they are displayed by retailers; gauge the impact of program support on shelf-stocking practices and pricing in Massachusetts compared to a non-program area, New York; show how shelf-stocking practices have changed over time in Massachusetts; and explore web scraping as a complementary method to store visits for shelfstocking studies.

The overall story that emerges from the shelf-stocking data is that LEDs seem poised to dominate lighting aisles in both states, especially in those channels that sell the most lighting. LEDs have reached their highest shelf share thus far, 30%. Incandescent lamps, however, are the most common type on shelves in both Massachusetts and New York. Among participating Massachusetts stores, the shares of incandescent and halogen lamps have increased (up to 40% and 18%, respectively) to fill some of the gap left by the rapid decline of CFLs from their 2012 peak of 69% to 12% in 2016. Additional key findings related to the proportion of lamps on shelves in Massachusetts and New York are also presented in the study.

RLPNC 16-11 Task 4b Market Progress Assessment Memo

 

This Market Progress Assessment summarizes current and historical values for key indicators of progress toward market transformation in the Massachusetts residential lighting market. The indicators are based on outcomes expected to accrue in the market over time as a result of Massachusetts Residential ENERGY STAR® Lighting Program activities. Phase 1 of the study resulted in a logic model for the program. This document is the first deliverable for Phase 2 of the Massachusetts Lighting Logic Model and Market Scan (Study RLPNC 16-11). For this study, evaluators conducted a detailed review of past Massachusetts lighting studies in order to identify appropriate indicators of progress toward the outcomes in the logic model and extracted historical values. In the process, we identified outcomes for which no indicators are being tracked, or for which additional or improved indicators may be helpful, and developed recommendations for tracking these in future.

Figure 1 at the conclusion of this memo displays the outcomes from the finalized logic model with their related indicators. The sections preceding this figure explain the various indicators and their priority level,
along with the evaluator’s recommendations for continued tracking.

Multifamily Program Improvement Strategies Preliminary Report

 

The goal of the overall Multifamily Program Improvement Strategies study is to identify actionable strategies and innovations to improve the multifamily program performance, realization rates, and overall program cost-effectiveness for both the residential and commercial sectors. The focus of this report, prepared as the Task 4 Preliminary Results Memo, is to identify implementation changes which will result in rapid improvements in program performance. As lighting accounts for 92% of program savings, the lighting end use has been a primary focus of the evaluator’s research. A summary of activities incorporated into this report include: Task 2b: Technical Resource Manual Multifamily Measure Characterization, Task 3a: Project Verification, Lighting Baseline and Delta Watts, and Task 3b: Project Verification, On-Site Verification.

The strategies suggested in this report are limited to the results of project Tasks 2b and 3. Additional analysis of cost effectiveness and comprehensive impacts will be incorporated as part of Task 7 to inform and formalize the final project recommendations. Suggested improvement strategies include: Adjust In-Unit lighting hours of use, use site-specific HOU for common area lighting retrofits, continue to explore prioritizing lighting retrofits to high hours of use locations, strengthen project reporting and verification procedures, shift customer expectations to incorporate additional measures into projects, reinforce auditor training to identify and complete additional measure types, incorporate lighting controls into projects, and update Technical Resource Manual.

2016 Annual Report Market Adoption Model
2016 Annual Report Market Adoption Model-Simplified Version
2016 Annual Report Market Adoption Model-Interim Reflector

 

Stretch Code Market Effects Study

 

This study conducted historical case studies in six stretch code communities to examine the possible causal mechanisms that can influence adoption and compliance in different types of communities. The study focused on understanding how the programs and initiatives sponsored by the Program Administrators (PAs) could possibly have affected stretch code-related savings both by affecting individual communities’ adoption of the stretch code, and by increasing energy code compliance in stretch code communities. Evaluators reviewed various publicly available documents from these six communities concerning the adoption of the stretch code, including city council or town meeting minutes as well as other relevant documents such as committee agendas or minutes, municipal annual reports, and newspaper articles.

Key findings from this study include the following: The two main reasons for adoption of the stretch code by the communities studied are the desire to be environmentally responsible, and the Green Community designation which provides access to state funding. The new construction programs sponsored by the PAs, particularly for residential homes, also played a role in the adoption of the stretch code by individual communities by offering incentives to partially compensate for increased construction costs. New construction programs have also likely played a role in increasing the base of HERS raters available, thus helping, at least during the early period, some communities to adopt the stretch code, which uses a performance path for compliance, and requires HERS ratings for all newly built homes. Most code officials considered the Code Compliance Support Initiative (CCSI) trainings sponsored by the PAs to be useful in enforcing the stretch code. However, there has been little use of the Mass Save Energy Code Technical Support outside the classroom.

Lighting Distribution Model Report

 

Heat Pump Water Heater Impact Study-Volume 1
Heat Pump Water Heaters Impact Study Tasks 1 and 2 Summary

 

See summary document above.

See summary document above.

RES 25 – Heating and Cooling Contractor Interview Findings
RES 25 Study Summary for 2016 Term Year Report

 

See summary document above.

See summary document above.

MA Multifamily High Rise Baseline Study Report
Addendum to MA Multifamily High Rise Baseline Study

 

This addendum  to the Multifamily High Rise Baseline Study report documents the User Defined Reference Home (UDRH) inputs that will be used when calculating savings retrospectively for 2016 program projects and prospectively for 2017 program projects. The tables in the addendum detail the following information: the current baseline inputs; updated inputs that will be used retrospectively to re-run savings for the 2016 program year; updated inputs that will be used prospectively to calculate savings for the 2017 program year; and sources for new baseline inputs.

Table 1 presents the new baseline inputs for residential and common area lighting. For in-unit residential lighting, the working group agreed to use a value 0.75 watts per square foot; this value will be used both retrospectively and prospectively. For common area lighting, the working group agreed to use different values retrospectively and prospectively. Retrospectively, for use in re-running the 2016 program year savings, the working group agreed to reduce the 2012 IECC lighting requirements by 27%. Prospectively, for use in the 2017 program year, the group agreed to apply a 10% reduction to the 2015 IECC lighting tables. Table 2 presents the new baseline inputs for residential and common area heating systems and  detail the assumptions associated with each heating system measure. Table 3 presents the new baseline inputs for residential and common area cooling systems. Retrospectively, all inputs other than ductless mini-splits, are based on ASHRAE 90.1-2010 and the 2012 IECC requirements. Prospectively, all inputs other than ductless mini-splits are based on ASHRAE 90.1-2013 and the 2015 IECC requirements. The ductless mini-split values, both retrospective and prospective, were agreed upon by the working group and are based on the recent ductless mini-split heat pump impact evaluation that was previously mentioned. Table 4 presents the new baseline inputs for in-unit water heating technologies. Retrospectively, the program will use the energy factors specified in the commercial section of the 2012 IECC. For the 2017 program year, the program will use new federal standards that went into place on April 16, 2015 as the baseline inputs for the program. The Team felt it was reasonable to use the previous federal standards when re-running savings for the 2016 program year.

RLPNC 16-3 Lighting Decision Making Memo

 

This memo describes results of a study interned to determine what motivates consumers to purchase efficient lighting and what factors influence consumers to instead purchase inefficient lighting in Massachusetts and New York. The evaluation team analyzed lighting purchase data and administering surveys to recent shoppers in Massachusetts and New York from a panel of lighting purchasers provided by InfoScout, a research company that has a nationwide panel of customers who upload their receipts from retail and restaurant shopping trips in exchange for various rewards.

Massachusetts lighting consumers are buying more efficient bulbs at a lower cost than their counterparts in New York across nearly all demographic categories and retail channels. Massachusetts consumers are, however, still buying a large share of inefficient bulbs (roughly one-half of all purchases). Efficiency and long life are the most important factors to buyers of efficient lighting. Efficiency is especially valued by Massachusetts consumers. Consumers of inefficient lighting are more likely to be attempting to match their purchases to bulbs that have burned out. Decisions about what type of bulbs to purchase are typically made in advance of the shopping trip, especially for LED bulbs. Retail channels like grocery and discount stores sell a relatively small share of lighting overall, but a disproportionately large share of inefficient bulbs. Low-income consumers, compared to other income groups, are not buying a disproportionate share of lighting at hard-to-reach channels.

RLPNC 16-11 Second Quarterly Lighting Market Scan Memo

 

This is the second of four market scan memos that are part of the RLPNC 16-11 study. The first memo summarized information on the state of the United States (US) residential lighting market as of September 2016 and where this market might be headed. This second market scan memo presents new information on the US residential lighting market published in the fourth quarter of 2016. The market scans focus on information gained from areas outside of Massachusetts.

One-half of 16 lighting suppliers (manufacturers and retailers) interviewed for the 2015-2016 Northwest lighting market tracking study expect EISA 2020 to drive consumers to choose LEDs going forward. The eleven lighting manufacturers interviewed for this study expect their CFL market share to decline and LED market share to rise more quickly than previously reported. On average, these lighting manufacturers estimated their 2015 US market share to comprise 33% LED, 24% CFL, 23% halogen, and 20% incandescent. The proportion of suppliers’ 2015 LED sales that are ENERGY STAR certified ranged from 35% to 100%, with one quarter of suppliers reporting that all the bulbs they manufactured were ENERGY STAR certified. Evaluated identified three new NTG estimates for lighting programs since the first market scan. All three are from 2015 upstream lighting programs in different portions of Pennsylvania, including the service territories of PPL, PECO, and FirstEnergy. The NTG ratio of 0.61 for all LEDs sold through PPL’s program was derived using a demand elasticity model that estimates free-ridership but not spillover. The NTG ratios of 0.40 for CFLs, 0.66 for standard LEDs, and 0.62 for specialty LEDs incentivized through PECO’s lighting program were based on store intercept surveys (n=330) and include both free-ridership and spillover. The NTG ratio of 0.68 for FirstEnergy’s lighting program is based on interviews with corporate retail chain representatives and includes free-ridership only.

2016

Single Family Code Compliance Baseline Study-Volume-1
Single Family Code Compliance Baseline Study-Volume 2
Single-Family Code Compliance Baseline Study-Volume 3
Single Family Code Compliance Baseline Study-Volume-4
Single-Family Code Compliance Baseline Study-Volume 5

 

This study was designed to provide the Program Administrators (PAs) and EEAC Consultants with a code compliance assessment of newly constructed single-family homes permitted at the end of the 2009 International Energy Conservation Code (IECC) cycle, homes permitted during the beginning of the 2012 IECC cycle, and homes permitted under the stretch code. To provide the information needed to update the UDRH for the low-rise component (residential buildings three stories or lower) of the RNC program. Another primary goal was to provide the information needed to update the UDRH for the low-rise component (residential buildings three stories or lower) of the RNC program. To meet these goals and answer the research questions listed below, the PAs and EEAC Consultants decided to conduct on-site inspections with non-program single-family homes.

Given the amount of information covered in this study, results are presented in five separate report volumes:

  • Volume 1: Sampling, Recruitment, and On-site Data Collection This report volume presents the sampling, recruitment, and on-site data collection methods.
  • Volume 2: Data Analysis Findings This report volume presents detailed findings from the on-site inspections.
  • Volume 3: Baseline Findings This report volume provides a high-level summary of the baseline data and analysis findings to support updating the Residential New Construction (RNC) program User Defined Reference Home (UDRH).
  • Volume 4: Compliance Findings This report volume focuses on the code compliance assessment, which is founded on the data presented in the data analysis report volume.
  • Volume 5: UDRH Addendum This report volume presents the updated Residential New Construction (RNC) program UDRH.

CCSI Residential Immediate Training Survey Memo

 

This memo provides analyses of the immediate survey responses collected through paper surveys, registration data, and Audience Response Systems (ARS) from six CCSI residential trainings on the 2012 IECC code held from January through April of 2016. The CCSI contractor held four trainings on Envelope and Building Science (EBS) on January 13, February 4, March 31, and April 5. Two trainings on HVAC and Indoor Air Quality (HVAC-IAQ) were held on  March 10 and April 28. Out of an estimated 144 training attendees, 116 attendees filled out paper survey forms.

Of the survey respondents, 31 were building code officials and the remaining 85 fell into the general category of builders, architects, contractors, equipment suppliers, and others. The memo presents detailed results related to the following topics: Usefulness Ratings for Envelope Building Science Training Components, Whether the Envelope Building Science Training Components Were New, Usefulness Ratings for HVAC Indoor Air Quality Training Components, Whether the HVAC Indoor Air Quality Training Components Were New, Quality of Trainings, ARS Ratings of Training Quality, Housing Units Permitted under 2012 IECC for January through April 2016 Trainees, Housing Units Permitted under 2012 IECC for 2014 through 2016 Trainees, When Expect a Final Inspection on a 2012 IECC Unit, When Expect to First Use Training Information, Most Important New Information Provided by the Trainings (January through April 2016), How Information Provided by the Trainings Will Be Used, Additional Comments and Suggestions to Improve Trainings, Training Attendees, Years in Present Position for January through April 2016 Trainees, Proportion of Retrofit Building Permits, Cities and Towns Represented in the January through April Trainings, Numbers of Unique Training Attendees.

Follow-up Interviews with CCSI Residential Training Attendees

 

As part of the ongoing evaluation of the Massachusetts Code Compliance Support Initiative (CCSI), evaluators conducted follow-up in-depth interviews (IDIs) with 60 individuals who had attended one or more residential classroom trainings approximately six months earlier. Thirty respondents work as municipal building code employees and 30 work as builders, architects, equipment suppliers, or energy efficiency professionals (referred to as ‘builders and others’). The overall goal of the follow-up interviews was to determine if and how the subjects are using what they learned at the trainings in the field; the interviews also explored how information from the trainings is shared, the changing environment for code compliance and enforcement, and suggestions for improving the trainings.

Nearly three-fourths of respondents had shared some of the information from the trainings with other parties. Close to one-half of the municipal building code employees said that the most useful part of the trainings were related to insulation and envelope areas. The most common reasons for not making any changes to fieldwork after attending the trainings were already knowing the information and working in a stretch code community. Nearly three-fourths of respondents had shared some of the information from the trainings with other parties. Just over one-half of the respondents said they had attended one or more trainings or gatherings discussing building codes since attended the CCSI training. Most builders and others reported increased interest in energy efficiency among both code officials (67 percent) and their customers (80 percent) during the past year. The most frequent suggestion offered by all respondents was for the CCSI to try to get more people to attend the trainings, especially builders (from municipal building code employees) and contractors (from builders and others). More general suggestions for improving code compliance include offering field assistance at construction sites and educating homeowners about the new code through information accessible by the public.  All but two of the 60 respondents reported that they would encourage their colleagues to attend the CCSI trainings. More detailed findings are discussed in the report.

Ductless Mini Split Heat Pump Impact Evaluation

 

This report presents results of an in situ evaluation of ductless mini‐split heat pumps (DMSHPs). The evaluation team initially planned to study 132 Massachusetts homes that participated in the COOL SMART Program. The PAs, however, extended the scope of work to include 20 Rhode Island homes that participated in the High Efficiency Heating and Cooling Rebate Program. The evaluation sought to address many utility and consumer questions about DMSHPs, focusing on power and energy consumption, heat output, efficiency, and interactions with existing HVAC equipment.

The report presents findings related to operating hours, equivalent full load hours, savings, and cold climate performance as well as a discussion of cooling, heating, COP/SEER/HSPF, savings values, and controls and zoning. Evaluators also provide the following recommendations: The evaluation team recommends exploring ways to improve the PAs’ existing lost opportunity program for DMSHPs, such as how best to encourage the installation of multiple DMSHP heads to better match existing zones and displace primary system operation. The study still finds that a modest level of savings are achievable by moving from a standard efficiency DMSHP to a higher efficiency DMSHP. Substantially more savings could be achieved (i.e., the top 25% of savings) if newly installed DMSHPs are operated more regularly and continuously by better matching and integrating them zonally with primary heating systems, through better configuration design and installation and contractor and customer education and training. The evaluation team also recommends exploring methods for targeting homes with electric resistance heating for DMSHP retrofits, targeting propane‐heated homes for DMSHP, exploring methods for addressing oil‐heated home, and examining opportunities for a new construction measure to substitute DMSHPs for central air conditioners. Other future studies could explore the use of interfaces between learning thermostats and ductless systems.

Residential Lighting Hours-of-Use Update

 

The 2014 Northeast Residential Lighting Hours-of-Use (HOU) Study was designed to allow sponsors in the Northeast to update HOU estimates based on changes in room-by-room saturation collected as part of regular saturation studies. This memo serves as an update for the Program Administrators in Massachusetts for the upstream lighting programs. This update is not applicable to the direct-install programs which operate using a somewhat different HOU estimate. The sources of data for this memo include: the Northeast Lighting Hours-of-Use Study, results of the Massachusetts On-site Lighting Inventory (2013), and the 2015-16 Lighting Market Assessment Consumer Survey and On-site Saturation Study.

The memo provides an overview of the average number of sockets and snapback-adjusted HOU by room; socket saturation for CFLs, LEDs, and all energy-efficient bulbs (CFLs, LEDs, and linear fluorescents) between 2013 and 2016; and changes in saturation. As the data show, with the exception of kitchens, all room types show net gains in energy-efficient bulbs. For kitchens, the gains in LED saturation were offset by losses in CFL and linear fluorescent (not shown) saturation. This is important because kitchens have the second highest HOU after exteriors. To estimate updated HOU, evaluators calculated the proportion of bulbs in each room by bulb type (separately for CFLs and LEDs). The memo provides the results of these calculations for each bulb category type as well as the snapback-adjusted HOU by room and the resulting household HOU estimate. Conclusions drawn by the evaluators suggest that stimated HOU for CFLs should be revised downward (by about 3%) for any retrospective (such as the 2016 Annual Report) or prospective CFL savings calculations. However, because the PAs plan to end supporting CFLs as part of the upstream program in 2017, there may be no prospective CFL savings calculations. Estimated HOU for LEDs should be revised upward (by about 3%) for any retrospective (such as the 2016 Annual Report) and prospective LED savings calculations. However, there should be consideration of the likelihood that LEDs will continue to replace CFLs and may reduce the HOU over time. LED installations should continue to be monitored for potential future shifts. Evaluators do not recommend any changes for direct-install lighting programs. For these programs, the PAs should continue to use the all-bulb HOU estimates provided in the HOU report. For the household level, this estimate is 2.7. However, whenever possible the PAs should track installed bulbs by room type and apply the room-specific HOU estimates.

Multifamily Program Gas and Electric Impact Study

 

The Mass Save® Multifamily Retrofit Program is a direct installation program that promotes both electric and gas energy efficiency measures. The goal of this study is to provide 2013 program level (Residential Multifamily) realization rates for both electric and gas overall and provide disaggregated results where appropriate. Evaluators used a two-stage, premise-level, difference-of-differences modelling approach for energy consumption analysis using a dataset combining consumption, weather, and participation and other premise and customer-specific characteristics information. This approach estimates gross energy savings and relies on a comparison group consisting of future participants to control for non-program related change. The data in this study relies nearly exclusively on data directly from National Grid and was accompanied by stages of review to ensure the representation and adequacy of the data to support the billing analysis.

The Multifamily Program is producing electric savings, though much less than estimated in the tracking system. The evaluators’ estimate of savings from the 2013 program year is 3,559 MWh with a precision of ±49.3 at the 90% confidence interval. This result provides a realization rate of 24.4%. Evaluators found no substantial difference in results when analyzed at the premise level or when examined by facility size (number of premises in each facility) or housing type (apartment vs condominium), though they found participants with commercial billing rates experienced greater savings in absolute terms as well as percent of consumption than participants with residential rates. The final estimate of program level natural gas impacts is 383,129 Therms, which does not include interactive effects. This result is accompanied by a realization rate of 86.2% and a precision of ±64.1% at 90% confidence interval. The tracking savings estimate for commercial bill rates had a considerably higher realization rate (127%) and savings as a percent of consumption (12.4%) than savings estimated for residential (38% and 1.6%, respectively). Evaluators also found results vary when facilities examined by ownership type, though not as significantly, with apartment and condominium realization rates of 108.8% and 68.5%, respectively.

Massachusetts Behavioral Programs Impact Evaluation

 

This report summarizes the impact analyses conducted in 2015 of National Grid’s and Eversource Energy’s Home Energy Report (HER) programs.  At the end of 2014, there were a total of thirty-one evaluable cohorts in these Program Administrator’s HER programs. This included fifteen electric cohorts, eight gas cohorts, and four dual fuel cohorts. The evaluation team conducted three distinct impact analyses related to these HER programs: cohort-specific impact analysis, mapping analysis, and dual treatment analysis.

The cohort-specific analysis  indicated that total net electric savings from the Massachusetts HER programs were 127,855 MWh, and total net gas savings were 643,157 MMBtu. The evaluation team also presents  recommended savings estimate ratios for the PAs to adopt in future years when third-party impact evaluations are not completed. From the mapping analysis, the evaluation team concluded that the overlap between different program cohorts is generally small and is not expected to impact cohort-specific savings analysis using standard evaluation methods. The dual  analysis suggest that on the electric side, dual-fuel customers saved less than dual-treatment customers at a  statistically significant level. On the gas side, the difference was not statistically significant. Cross-PA customers (a subset of dual treatment customers who receive electric reports from one PA and gas reports from another) were satisfied with the frequency at which they currently receive reports and they found the reports just as useful as dual-fuel customers.  The evaluation team does not believe that coordination across the PAs is warranted and recommends that the PAs continue implementing the HER program in its current form.

MA 2015-16 Lighting Market Assessment Final Report

 

This report presents the results of the Massachusetts 2015-16 Residential Lighting Market Assessment, which was designed to update estimates of lighting saturation and other critical market indicators, and to assess consumers’ knowledge of, and interactions with, the lighting market in Massachusetts. The data for this study came from web and telephone surveys and on-site lighting inventories conducted with random samples of homes in Massachusetts and a comparison area completed between December 2015 and March 2016.

While consumers are adopting LEDs in the non-program comparison area, evidence from this study suggests that the Massachusetts programs have had a strong impact on saturation and penetration of energy-efficient bulbs.  LED saturation and penetration rates in the comparison area (New York) continued to lag significantly behind the rates measured in Massachusetts. Further, ENERGY STAR® LEDs accounted for the entirety of the difference in LED saturation between the two states. Finally, as the potential energy savings analysis in this report demonstrates, there are substantial savings yet to be realized in the Massachusetts residential lighting market. By 2016, CFL saturation in Massachusetts was significantly higher compared to New York, closely aligned with NYSERDA’s decision to cease standard spiral CFL incentives in 2012 and essentially end all upstream lighting incentives in 2014. In contrast, faced with similar questions about the future of residential lighting, Massachusetts elected to phase out support for standard CFLs in a controlled manner. Recent changes to the ENERGY STAR specifications that preclude CFLs, and expectations that the price for ENERGY STAR LEDs will continue to decrease, mean that the role of CFLs in the upstream program is coming to an end. The report presents other key findings as well as recommendations, considerations, and guidance for future study.

MA Lighting Interactive Effects Results Memo

 

This memo details the findings of the Lighting Interactive Effects study conducted  in order to better understand and report the true impact of energy efficient lighting retrofits. The goal of this study was to determine a statewide average for the heating, ventilation, and air conditioning (HVAC) impacts of residential lighting measures, quantified using interactive effects (IE) factors. To accomplish this, the Residential Evaluation Team developed and calibrated hourly building energy simulation models for single family, low-rise multifamily and high-rise multifamily structures. The team weighted the model results using socket saturations for each building type and Massachusetts-specific housing stock data to arrive at one statewide average.

Table 1 in the memo presents average interactive effects for the Residential Upstream Lighting Initiative in Massachusetts. These values may be updated later in 2016 or early in 2017 as improved housing characteristic information becomes available through the forthcoming residential baseline study.

Ductless Mini-Split Heat Pump Draft Cooling Season Results Memorandum

 

The Massachusetts and Rhode Island Program Administrators (PAs) commissioned an evaluation team to conduct an in situ study of ductless mini‐split heat pumps (DMSHPs) in Massachusetts and Rhode Island.The PAs and the evaluation team identified several research questions related to the performance, efficiency, and energy savings resulting from the installation of DMSHPs through Massachusetts’ COOL SMART (and through Rhode Island’s High Efficiency Heating and Cooling) Program. This memo addresses the first two questions, which ask: 1. How much energy is being saved with an average DMSHP installation? 2. How does the in situ performance correlate with DMSHP rated cooling capacity, rated SEER, rated EER, and ambient conditions? The final report will address all research questions.

The design and manufacture of DHSHPs is evolving quickly, and much has changed since these units were installed in 2013 and 2014. More makers are offering DMSHPs. Cooling SEERs as high as 33 are advertised, with most makers advertising units approaching SEER 30. These higher SEER systems will likely offer even greater cooling savings than those delivered by the population in this study (which averaged about 22 SEER). Compared to the metered sample and to previous population averages, the average SEER increased in 2015. Overall savings, however declined slightly because the average Tier 2 DMSHP system size was much smaller than the Tier 1 DMSHPs. The savings found in this study are generally aligned with the TRM‐derived values but are lower because the evaluators’ findings show that the metered DMSHP units operate at fewer full load cooling hours (265 actual hours, 259 hours used for savings) than the deemed TRM value (360 hours). Though the weather differences between MA and RI likely result in runtime differences, the variability due to other factors (e.g. user operation) is much higher.  The final report will include state‐specific results if we find statistically significant differences in runtime after normalizing by population‐weighted weather data. The final report will also include analytical observations of statistical significance which may help decision makers further refine program design and marketing.

Evaluation of Cape Light Compact’s Creating Awareness for Power Efficiency Initiative

 

This report presents the results of a process and impact evaluation of Cape Light Compact’s (CLC) Creating Awareness for Power Efficiency (CAPE) behavior feedback initiative. The evaluation covers the period from program launch in August 2014 through December, 2015. The evaluation team conducted a process evaluation consisting of in-depth interviews with 27 customers of varying participation-levels (never installed the equipment, installed the equipment but lost connectivity, installed the equipment and maintained connectivity). In addition, the evaluation team conducted an impact evaluation using econometric analysis to estimate savings.

Customers’ experiences with the CAPE initiative illustrate many of the common benefits and challenges of energy use feedback programs related to equipment installation, participant engagement, and participant action. Results suggest that equipment installation was the primary barrier to participation in the CAPE initiative. Customers who participated in the CAPE initiative expressed high levels of interest in monitoring energy use and motivation to save energy. However, among customers with access to their energy data, monitoring decreased over time with more than half of participants monitoring their energy use once per week or less. Few CAPE participants committed to energy saving actions as a result of their energy use monitoring. Although the savings estimates for the CAPE initiative were not statistically significant, it is clear from the confidence bounds that the savings are not as high as the savings from the SHEMP Legacy program but they may be similar to the SHEMP Energize program.

Final 2015 Report Lighting Market Adoption Model Weighting Scheme Implications Memorandum
Lighting Market Adoption Model Spreadsheet

 

This memorandum provides a comparison of the weights in the July 30, 2015 Draft Market Adoption Model and in the March 2, 2016 Final Market Adoption Model. It also  summarizes the impact of the weighting change on gross delta watts assumptions.

The most substantial change in the weighting scheme is the introduction of unique weights for CFLs and LEDS. Related to this, under the new scheme LED sales tend to be more heavily weighted towards 40W and 60W bins while CFLs are more heavily weighted to the 100W and 60W bins. In terms of gross delta watts assumptions, CFL delta watts all increase by a very small amount while LEDs exhibit a reduction in delta watts. This is because the new weighting scheme moves more CFLs into the 100W category and LEDs into the 40W and 60W categories.

Reducing the Size of the Control Group in the Home Energy Report Program Memorandum

 

This memorandum addresses the evaluation team’s research and calculations to inform Massachusetts Program Administrators (PAs) and the Energy Efficiency Advisory Council (EEAC) of opportunities to reduce the size of control groups in the Home Energy Report (HER) program. Power analysis results present the PAs and EEAC with optimal numbers of control group customers which can be removed from each HER cohort and assigned to a new cohort, while taking into consideration the statistical confidence of resulting savings estimation, sensitivity analysis around potential deviations from expected savings values, and other key considerations.

Key findings suggest that National Grid has the ability to reduce the control group size in six of the eleven HER program cohorts analyzed, resulting in over 100,000 new treatment customers. All four of the Eversource Energy (formerly NSTAR) HER program cohorts analyzed were found to be reducible, allowing nearly 50,000 new customers to be transitioned into a new treatment group.

Residential Single-Family Building Department Document Review
 

In conjunction with the ongoing single-family compliance/baseline study, evaluators visited 52 building departments throughout Massachusetts to determine what type of documentation is being filed to show compliance with the energy code for single-family homes.

Evaluators determined that energy-related code requirements are not consistently being documented at building departments. Very few homes had documentation of all energy-related code requirements. Much variation exists between and within towns in terms of energy-related code requirement documentation. Duct leakage and air leakage documentation is lacking, particularly in non-stretch code municipalities. Code officials are not using REScheck checklists. There is very little documentation that Manual J calculations are being conducted. Stretch code homes rarely have documentation filed showing that the ENERGY STAR homes thermal enclosure checklist was completed.

Berkshire Gas Home Energy Report Process and Impact Evaluation
 

This report present the results of the first‐year process and impact evaluation of Berkshire Gas’ Home Energy Report (HER) program. The primary objective of the program is to provide residential households with information on their gas consumption and tips on how to save energy to prompt them to take action to reduce their natural gas usage. The HER program, implemented by Opower, was launched in October 2014, and has delivered reports to nearly 12,000 customers during its first program year.

Overall, participants are satisfied with the HER reports. Over half of participants (59%) classify the reports as useful, similar to surveys of other Massachusetts gas HER programs. Participants are most satisfied with the personal usage comparison. Total program savings were 7,603  MMBtu, which is 0.7 MMBtu annually per person or 0.49%. This absolute savings value is relatively low given the high baseline usage compared to the first‐year evaluated savings of other gas cohorts in Massachusetts. In percentage terms, savings are the lowest among the MA cohorts. The lower than expected evaluated savings are supported by Opower’s reported savings, and the customer surveys conducted reveal that fewer Berkshire participants report that the reports led to energy‐saving behaviors.

LED Incremental Cost Study
 

This report presents the methods and findings from a study to understand the incremental bulb purchase cost of light-emitting diodes (LEDs) relative to compact fluorescent lamps (CFLs) and halogen bulbs through 2018 (with additional extrapolations through 2020 included in Appendix A). The three data sources used for various extrapolations were 1) in-depth interviews with suppliers and high-level retail buyers (supplier/retailer interviews), 2) a regression assessment based on historical point-of sale (POS) pricing data, and 3) an exponential regression analysis based on web-scraping data. Preliminary results of the supplier interviews and web-scraping exercises also informed the separate Lighting Net-to-Gross Study, specifically the development of prospective NTG ratios for 2016 to 2018.

All three incremental cost methods predict that LED prices will continue to decline over time. The magnitude of decline varies by method; interviews and POS suggest a 30% decline in estimated prices between 2015 and 2018, and web-scraping estimates a 50% decline in price relative to 2015 by 2018. There was also a large spread in the estimate of starting LED prices ranging from $11.29 to $7.25 (in 2015). The variance in price decline might be attributed to the different distribution channels and associated data that are available within each incremental cost method. The web-scraping approach draws published bulb prices from home improvement and hardware retail channels because grocery stores and similar channels do not offer online retail options. Conversely, the majority of bulbs considered in the POS model were those offered in grocery and drug stores because most big box channels are not includes in the POS data. These factors considered together mean that the data do not provide clear guidance on selecting a single incremental cost for the program years 2016 to 2018. Recommendation and considerations are also included.

2015

Cool Smart DMSHP Heating Results Memo
 

As part of a large impact metering study, the COOL SMART Impact Evaluation Team sought to determine heating season savings and full load hours associated with DMSHPs installed through the COOL SMART program. The team metered approximately 150 participating systems during the winter of 2014-2015 and extrapolated their usage to a more typical weather year.

As shown in the separate DMSHP baseline memo, the most common heating baseline for DMSHPs was a code-minimum DMSHP. Starting in 2016, all DMSHP units sold in the United States must meet a minimum heating season performance factor (HSPF) of 8.2, up from 7.7. The team calculated equivalent full load hours consistent with a HSPF-based savings calculation and savings against an 8.2 HSPF baseline, shown in a table. While the team found 447 equivalent full load hours and 210 kWh savings on average, the group of participants who purchased systems for both heating and cooling (65% of the total) had 20% higher usage (as measured in annual full load hours) and savings. The very small sample of customers who purchased for heating only shows even higher usage (as measured in annual full load hours) and savings.

 Residential Customer Profile Study
 

The Residential Customer Profile Study provides a summary of energy efficiency program participation, gross savings, and incentive spending for the Residential and Low-Income sectors across all Program Administrators (PAs) in Massachusetts. This report presents findings by sector and program for calendar year 2013. Results are based upon an analysis of program records that have been merged with U.S. Census Bureau data and demographic data from the American Community Survey at a U.S. Census Block Group level.

The study present program participation, gross savings, and incentive payments at the sector, program, and PA levels. Maps and other geographic analysis of program participation, savings, and incentive spending are presented at the U.S. Census Block Group level. All savings reported are for the primary fuel savings—the electric and gas savings that are directly targeted by programs—and do not include secondary fuel savings or savings in oil, propane, or wood.

  Saturation Comparison of MA, CA, NY
 

The purpose of this research was to compare the saturation estimates for MA, CA, and NY, with an eye toward the circumstances and strategies that may have been most responsible for boosting efficient bulb saturation by 8% in CA from 2009 to 2012 – a time period during which MA and NY saw a saturation plateau. The Team attempted to achieve this by interviewing individuals heavily involved in the design, implementation, and evaluation of the CA lighting program, as well as reviewing the evaluation and saturation calculation methodologies from CA and NY to examine any differences from MA.

The literature review of the evaluation methodologies and saturation comparisons between CA, NY, and MA revealed only one substantial difference in how saturation was estimated: CA evaluations did not include empty sockets in the saturation calculation, while MA did. However, evaluators determined that the saturation differences between the states were actual differences and did not result from differences in methods, and continued with further research into why saturation rates diverged.  Key findings from the study suggest that earlier implementation of EISA and the CA programmatic move toward grocery, discount, and drug stores were likely to have been the most influential factors in CA’s saturation gains from 2009 to 2012. Furthermore, given the higher initial saturation in MA than CA (i.e., more older CFLs), CFL-to-CFL replacement rates in MA were also likely to have been higher from 2009 to 2012 than in CA, which could explain saturation stagnation over that period. Concurrent research in MA and varying comparison states both with and without programs sheds more light on these relationships. It also makes clear that legislation and programmatic changes are not the only factors influencing increases in efficient bulb saturation.

Residential Lighting Shelf Survey and Pricing Analysis
 

This study presents findings from a shelf stocking and price survey that targeted two types of stores: those that are currently participating in the program (participant stores) and those stores that participated in the program in a previous year but are no longer participating (former‐participant stores).  Evaluators  analyzed lighting inventory data from a shelf survey to assess: CFL prices in the state relative to competing lighting products; the amount of shelf area dedicated to CFLs; and the pricing, number of bulb packages, and shelf location of CFLs and LEDs relative to otherlighting types. Evaluators also compared the results of the current survey to those of similar surveys conducted in 2010 and 2012.

Evaluators identified several key findings from the study. Participant stores continue to provide a wider variety, more inventory, and better prices of CFLs and LEDs (including discounts) than former‐participant stores. Participant stores also offered more packages with multiple CFLs than did the former‐participant stores. The majority of LEDs offered in participant stores are discounted through the program. CFL inventory appeared to be losing ground in 2013, compared to 2012, for both participant and former‐participant stores, although the differences are not statistically significant. The proportion of shelf space devoted to CFLs among participant stores has increased since 2010. Inventory shares of 75W and 100W incandescent packages have dropped from 8% in 2012 to 3% in 2013 for participant stores. 60W shares stayed relatively constant as EISA efficiency standards do not affect these bulbs until January 2014. Former‐participant stores appear to have replaced CFL inventory primarily with incandescent bulbs. Average CFL prices for A‐line bulbs in participant stores are almost double those of corresponding halogen bulbs. Even though the EISA phase‐out date had started for each, both 100W and 75W incandescent bulbs are widely available, with 57% of participating stores still stocking 100W incandescents and 63% stocking 75W incandescents. The participant stores stock a wider variety of CFL bulb types and wattages than the former‐participant stores. Overall, average discounted prices are less than non‐discounted prices; however, the average PA incentive is larger than that difference, indicating that the full PA discount may not be passed through to consumers.

Cool Smart Baseline Memo
 

This memorandum summarizes research regarding the appropriate baselines for estimating gross energy impacts from ductless mini‐split heat pumps (DMSHP) promoted through the electric PA’s COOL SMART program. The challenge with establishing installed baselines for DMSHPs is that there is not a clear standard efficiency technology that is the lone alternative to a DMSHP. The determination of appropriate  baselines is based on information gathered from program participants during on‐site visits to install meters measuring the energy usage of the DMSHPs and other heating and cooling equipment found in each home.

The evaluation team presents results based on two scenarios that reflect two ways of interpreting the baseline that should be used for customers who said they would have left the space unconditioned. There is likely some confusion on the part of respondents around what constitutes a conditioned or unconditioned space.  The evaluation team’s proposed baselines for determining gross DMSHP impacts for each scenario are summarized in tables.

Multistage Lighting Net to Gross Assessment Overall Report
 

The present Massachusetts (MA) program cycle included several studies with different research approaches that estimated net‐to‐gross (NTG) or net‐of‐freeridership. The purpose of this report is to describe those approaches briefly and to explain the consensus building process undertaken to develop and finalize NTG estimates, both retrospectively for 2014, and prospectively for 2016‐2018 (as required by the MA Department of Public Utilities). A group consisting of Program Administrators (PAs), Energy Efficiency Advisory Council (EEAC) consultants, and evaluation consultants reviewed the CFL and LED NTG and net‐of‐freeridership estimates derived for 2013 and/or 2014 and on‐going evaluation research regarding prices for lighting currently and in the near‐term future, ultimately working together to achieve agreement on the recommended NTG estimates.

The report presents the consensus retrospective (2014) and prospective (2016 to 2018) NTG values for the most critical products. The discussions involved in developing these recommendations are described in detail in the main body of the report. In all cases but one, the consensus values ended up being the average of the NTG values recommended by the group members for each bulb type and program category. The exception was non‐ hard‐to‐reach (HTR) LEDs, in which the final consensus values of 90% for 2016, 80% for 2017 and 70% for 2018—agreed upon after considerable discussion over multiple meetings and emails— were slightly lower than the simple average of group members’ recommendations (91%, 84%, and 77%, respectively).

Lighting Market Assessment and Saturation Stagnation Overall Report
 

Recent changes to the Massachusetts Residential Lighting Program have included providing incentives, and other program support for general service Light Emitting Diode (LEDs) bulbs and expanding program activity in retail channels and specific store locations thought to serve “hard‐to‐reach” (HTR) consumers. At the same time, the PAs continued providing substantial support for both standard spiral and specialty Compact Fluorescent Lamps (CFLs) in what could be called the more traditional lighting channels (e.g., home improvement and hardware stores).  This report summarizes results from the following three studies that explored market reactions to lighting program interventions: 2014 Market Assessment Study; 2015 Market Assessment Study; and 2014 Saturation Stagnation Study.

All three studies confirm what evaluators have found in over 15 years of evaluation activities:  the Massachusetts Residential Lighting Program and its predecessors have had a clear and dramatic impact on the residential lighting market. Over these 15 years, evaluators have observed a continuous trend of positive program performance including: increased LED penetration and saturation; nearly ubiquitous CFL penetration and increased saturation, especially among low‐income households; increased LED market share, with Massachusetts’ share being even higher than many other program states; steady CFL market share, even after the introduction of general service halogen bulbs, due at least in part to Massachusetts’ continued program support for CFLs; and decreased average wattage per bulb for all bulbs between 2013 and 2015. Still, while the program has had a clear impact on the market, the PAs must look to the future and consider the potentially evolving impact that LEDs and other lighting technologies, coupled with legislation that increases lighting efficiency standards, will have on the residential lighting market moving forward. In particular, the Energy Independence and Security Act (EISA) of 2007 impacts the availability of incandescents and the Energy Policy Act (EPAct) of 2005, as implemented through the General Service Fluorescent Lamp (GSFL) Rulemaking, affects linear fluorescent tubes, common in residences in Massachusetts. Consumers’ price sensitivity and overall lighting knowledge (awareness of EISA as well as different bulb types and performance) will continue to play an important role in their adoption of efficient lighting. Future planning should consider the implications of these multiple factors, while remaining forward‐looking and keeping pace with new developments in the market.

Cool Smart Incremental Cost Study
 

This incremental cost study estimates how manufacturing production costs (MPCs) and purchase prices of residential air conditioning (AC) and heat pump (HP) equipment change as equipment efficiency increases. The results of this study will support Cool Smart program enhancements and cost‐effectiveness analysis, as well as potential upstream residential upstream heating, ventilation and air conditioning (HVAC) incentive programs. The study methodology included examining rebate records, modeling costs and prices, and plotting the efficiency frontier.

The incremental prices presented in the cost‐efficiency plots represent the price increase above the least expensive baseline system in each category, at 13.0 SEER for ACs and 14.0 SEER & 8.2 HSPF for HPs.  The Methodology and Results sections of this report provide a detailed breakdown of these results, as well as cost‐efficiency charts based on energy efficiency ratio (EER) and heating seasonal performance factor (HSPF) ratings.

HES and HEAT Loan Program Assessment Final Report
 

The purpose of the assessment was to: determine the relative strengths and weaknesses of the Home Energy Services (HES) initiative by the lead vendor and home performance contractor delivery channels, with the intent to improve the overall initiative; assess the effectiveness of the current overlap in the HES, HEHE, and COOL SMART programs and the HEAT Loan; and identify any improvements that could increase the number of participants installing the recommended energy‐efficient improvements. The evaluation team calculated 20 key performance indicators (KPIs) for this study and used four years of tracking data to calculate each KPI by delivery channel. The evaluation team supplemented its quantitative KPI analysis with qualitative in‐depth interviews and surveyed 965 HES initiative participants to understand their perspective, in particular their experience and satisfaction with the HES initiative and HEAT Loan.

Evaluators drew several conclusions from the study. HPCs and lead vendors offer complementary approaches, each with their own benefits and drawbacks.  Providing services through both channels is likely helping to expand the reach of the HES initiative. HPC customers have lower rates of cross‐program participation than lead vendor customers, which may be a result of lack of HPC understanding regarding program offerings and less proactive promotion of non‐HES initiative offerings.  The majority of customers find that the Energy Specialist’s recommendations for making energy‐efficient upgrades in their homes are easy to understand; a smaller majority feel certain about how to navigate Mass Save offerings and incentives.  Customers who participated through the lead vendor delivery channel indicated higher overall satisfaction with the HES initiative and found that the lead vendor recommendations were easier to understand; however, all customers reported being at least somewhat satisfied with the initiative regardless of delivery channel. The HEAT Loan has been successful in promoting deeper and broader savings, acting as a motivating factor for customers and a business tool for contractors.  Although HEAT Loan awareness is relatively high among HES participants, opportunities remain for increasing HEAT Loan awareness among both HES participants and nonparticipants. Recommendations for improvement are also included.

HES and HEAT Loan Appendix A

 

Efficient Neighborhoods & Incremental Cost Assessment
 

This memorandum presents the results of the incremental cost analysis of the Efficient Neighborhoods+® (EN+) initiative. The results presented in this memo cover communities targeted during the first round of the EN+ initiative. Evaluators gathered incremental cost data through a series of data requests and follow-up discussions with Program Administrators (PAs) and implementation contractors.

Combined, the PAs spent an estimated additional $429,790 to administer the EN+ initiative relative to the standard HES program. Marketing and administrative costs represented the largest portion (84% combined) of the incremental costs. Total incremental costs vary considerably by PA because of the differing number of communities targeted by each PA, the scope of the marketing efforts, as well as the differing numbers of targeted customers and resulting participation levels.

Behavior Programs Process Evaluation
 

This report presents the results of a process evaluation for the Massachusetts Home Energy Report (HER) programs offered through Eversource Energy and National Grid in 2014. The HER program is implemented by Opower, and was first piloted in 2009 by National Grid. In this process evaluation, evaluators  sought to verify whether customers are still satisfied with the HER and interested in continuing to receive then reports. To achieve this objective, a telephone survey was administered to both program treatment and control group populations.

Results of the study suggest that overall, customers remain satisfied with the HER reports and the frequency of treatment. The survey results also suggest that the HER programs are enhancing customer satisfaction among electric treatment groups. The HER program is one among many ways customers are prompted to save energy in Massachusetts through various education and information initiatives, yet there is a clear lift in energy‐saving actions among treatment customers above their control counterparts. The survey results also suggest that program measure life may extend beyond one year. Although most energy saving actions appear to be independent of other programs (i.e. received no financial incentives) gas customers showed greater cross‐program participation. Self‐reported data suggests that cross‐fuel effects may be occurring. Cross Program Administrator customers are satisfied with the frequency of their reports, and do not differ significantly from dual fuel customers in their views on report usefulness. In‐depth interviews suggest customers could be better educated on the impact of various end uses on their household consumption. Interviews also suggest that customers may benefit from smarter home appliances and automation.  Among automation technologies, Wi‐Fi thermostats are of great interest to some, but many customers are skeptical that this technology will save more energy than their existing programmable thermostats. Recommendations for improvement are also included in the study.

Residential Point of Sale Modeling – Final Report

 

 

This report summarizes the findings of the Massachusetts Point‐of‐Sale (POS) modeling research. The research examines the influence of all the input variables—but most importantly program activity—in order to determine whether residential lighting programs have an important impact on CFL and LED sales. The purpose of the POS modeling research was to inform two major studies in the MA 2013 Residential cycle: the Saturation Stagnation investigation and the Multistage Lighting Net‐to‐Gross (NTG) research. The modeling lends itself to the former by investigating potential reasons for the lack of gains in efficient bulb saturation in MA between 2009 and 2012, and to the latter by providing MA‐specific NTG estimates for CFLs and LEDs (and both bulb types combined) by leveraging nationwide sales data.

Program activity, as defined by program budget measures, showed a consistently positive and significant relationship to the proportion of CFLs and LEDs sold across all available states and retail channels. The research also sheds light on the confounding finding in prior MA saturation studies that increases in CFL sales are not necessarily accompanied by increases in CFL saturation. It may be that saturation gains take longer to materialize than previously expected. Being able to observe this relationship, however, relies on the ability to control for intervening factors for which prior methods and exploratory analyses were unable to account.

Preliminary Lighting Demand Elasticity Findings Memo

 

This memo presents preliminary net‐of‐freeridership findings from a lighting demand elasticity analysis. The evaluators developed a model using program data on prices, the number of lamps purchased, and lamp and retailer characteristics. To determine the net‐to‐gross ratio for the upstream lighting program, the Evaluation Team employed multiple approaches. These approaches included supplier interviews with lighting product manufacturers and retailers, Point‐of‐Sale data modeling, saturation and comparison area analysis, and demand elasticity modeling.

Evaluators were unable to develop reliable LED elasticity estimates from a representative sample of products given potential stocking issues, missing months, negative sales, and long, irregular reporting periods that occurred among the two retailers that accounted for the majority of LED sales with varied prices. However, there were sufficient variation in price and sales data to estimate price elasticities and predict freeridership for standard and specialty CFLs. Estimated CFL freeridership was 44% for specialty CFLs and 61% for standard CFLs. The memo also describes the limitations of the available data and opportunities to improve the data for subsequent analyses.

Upstream Lighting Net-to-Gross Estimates Using Supplier Self-Report Methodology

 

This report presents estimates of net‐to‐gross (NTG) ratios for CFL and LED bulbs sold through the Massachusetts ENERGY STAR upstream lighting program in 2013 using supplier self‐report methodology. Evaluators calculated separate NTG estimates for bulbs sold through stores characterized as hard‐to‐reach (HTR) and through stores not characterized as such. HTR‐designated stores included the vast majority of stores making up the “discount” channel (such as dollar stores), ethnic grocery stores, and a few home improvement stores targeting low‐income citizens.

The report presents NTG ratios estimated for each bulb type sold through the Upstream Lighting Program. While evaluators aggregated to the bulb level and store type, this report also shows bulb‐type NTG ratios broken down by retail channel. The report compares NTG ratios provided by the 2013 program participants with those reported in previous program years, using the same supplier self‐report methodology. While the 2009–2010 and 2013 estimates included NTG estimates from all three market actor levels (e.g., manufacturers, retail buyers, and store managers), the 2011–2012 estimates only included estimates from store managers. Results suggest much higher NTG ratios for the 2013 program year than for the 2009–2010 program year—the last program year basing NTG ratios on estimates from all three levels of the lighting supply chain.

Supplier & Retailer Perspectives on Residential Lighting Market – Summary of Year 2014 Interviews Final Report
 

This report summarizes findings from retailer and supplier interviews conducted in 2014. The interviews supported the comprehensive assessment and monitoring of the Massachusetts lighting market and the Massachusetts ENERGY STAR® Lighting Program. Findings are intended to help develop a better understanding of the recent trends, new opportunities, and lingering barriers in the Massachusetts residential lighting market. The findings also serve to gauge lighting market actors’ satisfaction with this program and to identify opportunities for program improvements.

This report presents findings from lighting manufacturer and retailer interviews. Topics covered include impacts of EISA legislation, LED market assessment, hard to reach customers, program effects on CFL and LED sales, and program satisfaction.

Efficient Bulb Saturation Comparison of Massachusetts, California and New York – Final Report
 

The purpose of this research was to compare the saturation estimates for MA, CA, and NY, with a focus on the circumstances and strategies that may have been most responsible for boosting efficient bulb saturation by 8% in CA from 2009 to 2012. During this time period, MA and NY saw a saturation plateau. To answer this question, the evaluators interviewed individuals heavily involved in the design, implementation, and evaluation of the CA lighting program, as well as reviewed the evaluation and saturation calculation methodologies from CA and NY to examine any differences from MA.

Results of the study suggest that the earlier implementation of EISA in CA had an important impact on the move toward CFLs in that state, which was reflected in the saturation gains CA experienced relative to MA from 2009 to 2012. In addition, the CA program’s move away from big box stores and toward grocery, drug, dollar, and club channels also likely promoted saturation increases. CFLto‐CFL replacement rates in MA, which had higher CFL saturation than CA in 2009, also likely played a role in the saturation stagnation over that period. Finally, although the saturation point estimates of MA and CA had differed in 2012 (27% and 30%, respectively), when taking the confidence intervals into account, MA rates overlapped with those of CA, suggesting that while CA did demonstrate greater saturation gains over the 2009 to 2012 period, the actual saturation rates in 2012 between the two states were comparable. Other explanations that the Team considered were not supported by the research findings.

On-Site Lighting Inventory – Final Results
 

This report presents the findings of research related to the market for energy‐efficient light bulbs in Massachusetts. The objective of the study was to track various lighting market indicators—many of which have been measured in Massachusetts for over a decade—and to explore the data for possible impacts (particularly stockpiling of incandescent bulbs) associated with the Energy Independence and Security Act of 2007 (EISA). Specifically, this report presents an analysis of over four hundred residential lighting on‐site inventories collected in Massachusetts, Georgia, and Kansas in 2014. This report also compares findings from 2014 to those from previous on‐site studies conducted in Massachusetts in 2009, 2010, 2012, and 2013 and in Kansas and Georgia in 2009.

The report presents detailed findings from the on-site lighting inventories. The report also offers recommendations and considerations based on these findings. Evaluators recommend repeating the panel study and expanding on the panel size, adding 2014 saturation study participants. Considerations include: revising the retail program to include an inefficient bulb buy‐back program; considering program designs that address less efficient linear fluorescent tubes including T12s; undertaking more focused efforts on this lumen range when selecting and determining incentive levels for LEDs and CFLs to offer through upstream programs; revising the program design to encourage the purchase of multiple LEDs in one shopping trip; directly studying use and purchase behavior among hard‐to‐reach households.

High Efficiency Heating Equipment Impact Evaluation Final Report

 

The Massachusetts High-Efficiency Heating Equipment Rebate Program (HEHE) offers prescriptive rebates of up to $1,600 for the installation of new high-efficiency natural gas heating and water heating equipment. The objective of this evaluation was to determine gross energy savings for gas furnaces and boilers installed through the HEHE program, and refine the estimates of baseline efficiency and heating consumption. The field portion of the study included spot measurement of baseline and new equipment in situ efficiency as well as long-term metering of post-retrofit high efficiency equipment.

This evaluation provides revised savings estimates for high-efficiency furnace and boiler replacements as well as several program implications and conclusions. Key findings suggest: there are differences in annual heating loads between equipment types, though previous deemed savings used the same annual heating load for both furnaces and boilers; it is important to consider standby and cycling losses in addition to combustion efficiency when evaluating gravity-drafted equipment such as standard and early retirement boilers and furnaces; high-efficiency boilers are not being installed to maximize potential savings; many older gas furnaces and boilers considered “early retirement” equipment have AFUEs of at least 75 percent, even when considering actual instead of rated performance; evaluation research suggests that as many as 80 percent of new combination systems are replacing boilers with indirect water heaters, but the TRM currently assumes a boiler and a standalone water heater as the baseline.

Mass Save Mutlifamily Program Process Evaluation Report

 

This report presents the results of a process evaluation of the Mass Save Multifamily Building Program. The evaluation sought to assess and monitor the program’s evolution as an integrated offering since the last round of program evaluation was conducted; examine barriers to participation, the effectiveness of program operations, and customer experience; and review PA and vendor tracking data to assess whether these data would sufficiently support a planned future impact evaluation. Evaluation tasks largely included in-depth interviews, focus groups, and surveys that included a range of multifamily stakeholders and program experts. Evaluators also completed on-site visits to assess the comprehensiveness of the energy assessments and the measures installed as well as an extensive comparison of available program participation and energy assessment data.

The report presents the four key recommendations most critical for the PAs and EEAC to focus on in order to improve the program and achieve deeper and broader savings. First, the PAs and EEAC should consider creating a role for a single point of contact for each project. The program tracking systems should also be improved to allow for a holistic view of multifamily properties across PAs, fuels and programs. Improvement of the energy assessment process can be achieved through the training of auditors in completing a comprehensive job including a review of all the systems in common areas and major systems within in unit areas. Lastly, the evaluators believe a billing analysis is a feasible approach to determining savings among participating accounts.

Residential Lighting Consumer Survery – Final Report
 

The research presented in this report includes the results of a multi‐mode residential lighting consumer survey performed between March 6, 2014 and May 5, 2014. The goal of the consumer survey was to track key indicators of the market for CFLs, LEDs, and halogens as well as the impacts of EISA on the residential lighting market. The report provides comparisons with three similar previous lighting consumer surveys (Winter 2011, Summer 2012, and Winter 2012) where appropriate. In addition, results from comparison area surveys in Georgia and Kansas are included to provide a parallel between consumers’ lighting awareness and bulb use in Massachusetts and two other areas with little or no residential lighting program activity.

Evaluators drew several conclusions based on the results of the study. The Spring 2014 survey findings indicate that satisfaction with LEDs was higher than that of CFLs; particularly among users of both CFLs and LEDs, satisfaction with LEDs relative to CFLs was even more prominent. Although a majority of CFL users remains satisfied with CFLs, the indicator continues the downward trend observed since 2011. Results of the 2014 survey suggest a significant increase over previous waves in the proportion of households reporting familiarity with screw‐in LEDs. While it is likely that PA efforts to raise awareness about LEDs and provide incentives for them has played an important role in increased LED adoption, future studies would be needed to confirm this assumption. This survey differed from previous ones by offering some respondents the opportunity to answer via a web‐based instrument and sending some respondents a pre‐paid $5 incentive. The pre‐paid $5 incentive doubled the response rate. Also, households that actually completed the survey via the web differed from those that completed by phone demographically (more likely to own homes, have higher education, and, among those that reported their income, to have higher incomes) and in terms of their awareness, use, and satisfaction with energy‐efficient lighting. Additional recommendations and considerations are also included in the report.

Tube TV Recycling Program Assessment Memo

 

The goal of this study was to assess the feasibility of expanding the MassSave Appliance Rebate program to offer rebates for recycling operable but old, inefficient tube televisions (CRT‐TVs). the Evaluation Team collected television data from 150 Massachusetts homes in conjunction with the lighting market assessment study. The team also collected additional self‐reported usage information such as the average hours of daily use, estimated purchase date, and the willingness of the customer to replace the unit if offered with free removal for secondary TVs and a $50 voucher for primary TVs.

The Evaluation Team found that out of the 370 TVs recorded, 20% were CRT‐TVs and targets for replacement. The remaining 80% consisted of LCD/LED, plasma, or rear projection TVs (all considered efficient). While there was a reasonably high saturation of CRT‐TVs, research indicated that the remaining CRT‐TVs were underutilized, small, and being naturally replaced by users. For these reasons, the Evaluation Team does not recommend expanding the existing recycling program.

Behavior Program Impact Evaluation Memo

 

This memo presents results for the impact evaluation of the Home Energy Report (HER) programs for the Massachusetts Cross‐Cutting Research in Behavior and Education. In particular, this memo presents results for National Grid and Eversource Energy. The evaluation team conducted multiple analyses to determine cohort‐specific savings, including post‐program regression and linear fixed effects regression models. The evaluation also include channeling analysis to document the lift in other program participation due to the behavioral program treatment (participant lift). It also removed the savings co‐generated by behavioral and standard programs in order to avoid double counting savings across the portfolio at the program and measure level (savings adjustment). The evaluation team calculated savings estimate ratios for each of the HER program cohorts to examine the differences in savings as measured by the program implementer and the savings verified by the evaluation.

Total net electric savings from the Massachusetts HER programs are 127,854,643 kWh. Total net gas savings are 643,157 MMBtu. The evaluation team also provides recommended savings estimate ratios for the PAs to use in future years when third‐party impact evaluations are not completed.

 

2014

Home Energy Report Savings Decay Analysis Final Report

This report presents findings from an evaluation of the National Grid Home Energy Report reduced treatment experiment. The goal of this study was to determine what effect reduced treatment had on program savings – measured as the difference in program savings between participants for whom treatment is reduced, compared to those who continued to receive standard treatment. The study included three main activities: data request and review, implementation contractor interviews, and billing analysis.

“The study resulted in five key findings related to the treatment reduction experiment:
1. A reduction in treatment led to a decline in observed savings for both cohorts. The magnitude of the savings reduction likely depends on the reduced-treatment design, fuel type, and the duration of program intervention before the reduction.
2. The electric and gas cohorts responded differently to the reduction in treatment. Compared to the electric cohort, reduced treatment customers in the gas cohort seem to show a much sharper reduction in savings in response to the reduction in reports they experienced.
3. The electric and gas cohorts experienced a decay in savings at different times relative to the reduction in treatment. For electric, the decline did not set in immediately, whereas for gas, the decline was more immediate and precipitous.
4. Additional research in this area could help inform the design of treatment reduction strategies. We recommend that future experiments plan the timing of treatment reductions to further test the potential impact of the following factors: treatment duration prior to the experiment, seasonality of the reduction, duration of the reduction, and fuel-specific differences.
Implications of these findings are also discussed.”

Market Lift Assessment Final Report

 

“This report summarizes the findings of an evaluation of the Massachusetts (MA) Market Lift Initiative, a new program approach in MA geared toward increasing sales of compact fluorescent lamps (CFLs) by
rewarding participating retailers for increasing sales over a pre‐established baseline. In addition, evaluators explored how suppliers felt about Market Lift approaches, and Cadmus performed Quality Control by reviewing the report.”

The net lift achieved for MA stores for the 13W CFL bulbs over the entire lift period was 8,405 bulbs (15.3%), and the net lift achieved for 23W CFLs over the entire lift period was 2,705 bulbs (8%). The net lift achieved for MA stores for both 13W and 23W CFLs over the entire lift period was 11,110 bulbs (or 12.5% over pre‐lift period). Net Market Lift results are also presented under an alternative seasonal research design using a six‐month pre‐lift period. This approach decreases the net lift from 12.5% to 3.2% for the combined 13W and 23W CFLs. The decline is largely due to the effects of seasonality on purchasing. There was substantial disapproval toward Market Lift voiced by manufacturers and retailer buyers (as well as the claim from the one manufacturer who had participated that they would not do so in the future), calling into question the viability of the approach moving forward. While it does make sense that retailers and manufacturers would favor the standard upstream approach, in which incentives are guaranteed, the right Market Lift structure could still be attractive to at least some potential partners.

Northeast Residential Lighting Hours-of-Use Study Final Report

 

“The purpose of this study was to provide updated information to the Connecticut Energy Efficiency Board, the Massachusetts Program Administrators (Cape Light Compact, National Grid Massachusetts, Northeast Utilities, and Unitil), National Grid Rhode Island, and the New York State Energy Research and Development Authority to assist in the calculations of demand and energy savings for lighting programs. Specifically, this report presents load shapes, coincidence factors (CFs), and daily hours of use (HOU). Based on data collected from 4,462 loggers, the evaluators performed a series of regression
models to estimate HOU. ”

Evaluators concluded that the region comprising Connecticut, Massachusetts, Rhode Island, and Upstate New York had a household daily HOU of 2.7 hours for all bulbs and 3.0 for efficient bulbs, with HOU by room type varying from a low 1.7 in bathrooms to a high of 6.7 on the exterior of homes. Hours of use for Downstate New York exceeded those for the other areas included in the study, with a daily HOU of 4.1 for all bulbs and 5.2 for efficient bulbs for the household; room-specific estimates varied from 3.2 for bathrooms to 7.7 for kitchens. The evaluators also provide detailed HOU estimates by room type, home type (i.e., single-family or multifamily), and income level for the region overall and for each individual area included in the analysis. Additionally, the report presents load shapes as well as well as coincidence factors for winter and summer peak period and winter and summer peak hours to aid in load planning and the calculation of peak demand savings. Other topics addressed include comparisons of HOU for efficient and inefficient bulb types and comparisons to other existing HOU studies both in the Northeast region and throughout the United States.

Residential Lighting Shelf Survey & Pricing Analysis Final Report

 

Evaluators conducted a shelf stocking and price survey for stores currently participating in the e Massachusetts ENERGY STAR® Lighting program and those stores that participated in the program in a previous year but are no longer participating. Evaluators analyzed lighting inventory data from a shelf survey to assess CFL prices in the state relative to competing lighting products, the amount of shelf area dedicated to CFLs, the pricing, number of bulb packages, and shelf location of CFLs and LEDs relative to other lighting types. Evaluators also compared the results of the current survey to those of similar surveys conducted in 2010and 2012.

Participant stores continue to provide a wider variety, more inventory, and better prices of CFLs and LEDs (including discounts) than former‐participant stores. The majority of LEDs offered in participant stores are discounted through the program. CFL inventory appears to be losing ground in 2013, compared to 2012, for both participant and former‐participant stores, although the differences are not statistically significant. The proportion of shelf space devoted to CFLs among participant stores has increased since 2010. Inventory shares of 75W and 100W incandescent packages have dropped from 8% in 2012 to 3% in 2013 for participant stores. Shares of 60W bulbs stayed relatively constant as EISA efficiency standards do not affect these bulbs until January 2014. Former‐participant stores appear to have replaced CFL inventory primarily with incandescent bulbs. Average CFL prices for A‐line bulbs in participant stores are almost double those of corresponding halogen bulbs. Even though the EISA phase‐out date had started for each, both 100W and 75W incandescent bulbs are widely available, with 57% of participating stores still stocking 100W incandescents and 63% stocking 75W incandescents. The participant stores stock a wider variety of CFL bulb types and wattages than the former‐participant stores. Overall, average discounted prices are less than non‐discounted prices; however, the average PA incentive is larger than that difference, indicating that the full PA discount may not be passed through to consumers.

Single Family Non-Program Homes Code Compliance Results

 

Evaluators conducted on-site inspections at 150 recently constructed single-family non-program homes—fifty homes built at the end of the 2006 IECC cycle and 100 homes built at the beginning of the 2009 IECC cycle. The goal of this evaluation is to compare compliance rates between these two samples using a comparable approach for calculating compliance. The study used code compliance checklists developed by Pacific Northwest National Laboratory (PNNL) to develop compliance scores the homes. The results are representative of the average house-level compliance scores using the PNNL checklists. The checklist allows compliance to be assessed depending on which compliance approach the builder used: the prescriptive approach, the UA trade-off approach, or the performance approach.

Homes built at the end of the 2006 IECC cycle show significantly higher overall compliance scores (76%) than homes built at the beginning of the 2009 IECC cycle (63%). These results are not particularly surprising, given that significant changes took place when the 2009 IECC was implemented and the homes in the 2006 IECC sample were built at the end of the code cycle, while the homes in the 2009 IECC sample were built at the beginning of the cycle. One possible reason that homes built at the end of the 2006 IECC cycle show higher compliance is that builders and subcontractors had three years to familiarize themselves with the code before constructing these homes while builders and subcontractors building homes at the start of the 2009 ECC cycle were likely just getting familiar with the new requirements. The 2009 IECC sample homes are actually slightly more efficient that the 2006 IECC sample homes, even though the 2009 IECC homes display lower compliance with the applicable energy code. This finding supports the hypothesis that low compliance among the 2009 IECC homes is likely due to the fact that new requirements were implemented with the new energy code and builders were still familiarizing themselves with those requirements, resulting in lower compliance.

Ductless Min-Split Heat Pump Customer Survey Results

 

The Massachusetts COOL SMART program offers rebates to customers for installing qualified, high‐efficiency ductless mini‐split heat pump (DMSHP) equipment in their homes. A web survey was conducted of participants in this program to better understand motivations for participating and how the DMSHP equipment is being used. The final survey included responses from 430 participants.

The report identified several key findings for the program. DMSHPs are being installed as retrofit projects. The majority of participants are installing DMSHPs to obtain an improved level of comfort instead of a focus on saving energy. DMSHPs are more often installed for cooling than heating. Also, DMSHPs are being used to supplement, rather than replace existing heating systems.

Efficient Neighborhoods Plus Initiative Evaluation Final Report

 

“During 2013, the Massachusetts-based energy efficiency program administrators (PAs) conducted an initiative designed to engage hard-to-reach customers with an enhanced version of the Home Energy Services (“HES”) program. This report presents the findings of process and impact evaluations of the initiative. The initiative included three distinct components collectively referred to as the Efficient Neighborhoods+® (EN+®) initiative. These include the Core initiative, Cape Light Compact’s (CLC’s) EN+, and Fall River Neighborhood Energy Contest. The ultimate goal of the evaluation was to understand whether the three initiatives resulted in a lift in participation and energy savings. In addition, the evaluation sought to provide PAs with insights on the successful components of the initiatives and recommendations on possible improvements.

“Overall, the EN+ Core initiative was successful at increasing participation and savings in the target communities among lower to moderate income customers but less so in reaching rental properties. The Fall River Energy Challenge also resulted in increased participation and energy savings, even without increased incentives. The true impacts of CLC’s EN+ initiative are unknown. The process evaluation findings suggest that despite a variety of marketing and outreach efforts, awareness and knowledge can impede participation. Sources of awareness varied but overall results suggest that in-person outreach or phone calls are effective outreach strategies. Word-of-mouth marketing can also be an effective marketing strategy. Exposure to the initiative through more than one outreach method is an effective way of moving customers toward participation and costs appear to be the core barrier to participation. Lack of interest and perceived lack of need for improvements despite recommendations are additional barriers to participation. Pre-weatherization barriers exist, yet participant knowledge about the initiative offerings to mitigate those barriers is somewhat limited. Recommendations for improvement are also provided.

Residential New Construction Net Impacts Report

This study was designed to estimate the net impacts that may be attributed to the Massachusetts Residential New Construction Program for single-family homes. The study used a multi-step methodology consisting of a builder survey, a Delphi study involving a panel of experts in energy-efficient new construction, modeling of home energy usage under the counterfactual assumption that the Program had not existed from 2004 to 2011, and comparing the as-built modeled energy usage to the estimate from the counterfactual models. This approach was developed to account for as many factors as possible that might have influenced the adoption of energy-efficient construction practices.

The study found substantial net savings for the single-family component of the Massachusetts Residential New Construction Program. The Delphi panel estimated that homes completed in 2011 would have  been  much less efficient in the absense of the program. Combining a free ridership of 0.53 and non-  participant spillover of 1.39 gives the program a net-to-gross ratio of 1.87 when assessing  savings using the fuel neutral metric of MMBtu. Non-program homes are responsible for 75% of net savings in terms of MMBtu, 68% of  electric savings, and 71% of natural gas savings. The Delphi panelists noted that the program has had a particularly strong effect on air infiltration, duct leakage, lighting, insulation installation grades, and some heating system  efficiencies. When assessing net savings using the fuel-neutral metric of MMBtu, natural gas is the  fuel with the most net savings, followed by propane and then electricity. Lighting is responsible for 61.5% of all electric net savings.

Residential Lighting Controls Initiative Evaluation Final Report

Evaluators conducted an impact evaluation and a limited process evaluation of Cape Light Compact’s (CLC) Residential Lighting Controls Initiative. This report presents Cadmus’ findings and conclusions, which are drawn from pre- and post-installation metering, lab testing of dimming controls to determine lamp energy use at various dimmer control levels, and participant surveys.

Verified savings values are presented based on eight of the sixteen participating sites. Total gross demand impacts were 2.11 kW and the average impact per site was 0.26 kW. Total gross energy impacts were 1,987 kwh/yr and the average impact per site was 248 kWh/yr. Feedback from interviews with the CLC manager and RISE staff substantiated that this technology is challenging to implement as a program at this time. Compatibility of lighting controls with efficient lamp technologies was an issue and participants, the CLC manager, and RISE staff all had difficulty finding combinations of dimmers and bulbs that worked. Widescale efforts to replace lamps controlled by dimmer switches will require further research to identify working combinations of controls and efficient lamps. Survey and metering data revealed that some participants used their lights more after installation of the new dimmers and bulbs; however, substantial savings can still result since any savings are largely driven by the lamp replacement. Evaluators suggest that in order for an impact evaluation of lighting use with dimmer replacements to be successful, it is important to have detailed tracking of implementation activities.

 

2013

Year 2012 Home Energy Services Pre-Weatherization Initiative Evaluation Final Report

This report summarizes an evaluation of the 2012 Home Energy Services (HES) Pre-Weatherization Initiative (initiative) that was offered by the electric and gas program administrators (PAs) of Massachusetts. The Evaluation Team assessed the effectiveness providing customers with an additional incentive to help reduce their financial burden for addressing common pre-weatherization repairs that are required prior to installing certain HES measures, as well as on assessing the delivery of the initiative itself. The Team conducted interviews with HES PA program managers and lead vendors, analyzed initiative participation data through July 2012, and surveyed  participant and non-participant customers. They also followed-up on inquiries with PAs, conducted surveys with participants and non-participants from all PAs, and completed additional analysis of both complete initiative data and historic HES data at the end of September 2012 and in March 2013.

Evaluators drew several conclusions based on evaluation results. Evaluation results did not show a significant change in the measure adoption rate for National Grid and NSTAR customers who faced the knob and tube wiring barrier. While the turnkey option offers customers easy access to approved contractors, the PAs and lead vendors that offered the turnkey option were uncertain of the delivery option’s long-term viability. Non-participants indicated having confusion about what the initiative actually covered for knob and tube wiring. Stakeholders and customers that were subjected to the 30-day requirement indicated that additional time would have helped. However, an analysis of acceptance rates revealed that customers who were given a 30-day deadline had higher acceptance rates than those offered the 90-day deadline. Also, two of the three 30-day PAs (NSTAR and National Grid) offered additional delivery channels (HPC and turnkey) that were not available to some of the customers subjected to the 90-day timeline. Interviews with PAs and lead vendors indicate that elements of the initiative’s design and delivery varied across PAs. The evaluation also provides recommendations for initiative improvements.

Energy Star Lighting Program Early Impacts of EISA

This report summarizes the results of four different research efforts meant, in part, to identify the current and likely effects of EISA on the residential lighting market in Massachusetts. The research activities occurred from December 2011 through April 2013 and were completed for the Energy Efficiency Advisory Council Consultants and the Massachusetts Program Administrators (PAs).

This report provides key findings and recommendations related to consumer and store awareness of EISA, changes in bulb availability, changes in bulb purchases and use, stockpiling of incandescent bulbs, consumer understanding of key lighting concepts, and 60-Watt incandescent phase out. Some key findings are summarized below. Most consumers remained unaware of the increased lighting efficiency standards resulting from EISA and the subsequent phase-out of most general-service incandescent bulbs. Stores that participate in the program carry 23% more packages of CFLs and 4% more packages of LEDs than do stores that formerly participated. One-half of consumers shopping for 100-Watt incandescent bulbs could still find them during the last three months of 2012.  Households in Massachusetts did not exhibit substantial changes in their light bulb use between early 2012 and early 2013. When asked what type of bulb they would chose to replace 100-Watt incandescent bulbs, most consumers selected a lower wattage incandescent bulb followed by a CFL. The research found little evidence of widespread, EISA-induced stockpiling of incandescent bulbs. Sixty-Watt incandescent bulbs fill 22% of all sockets in Massachusetts homes, making this the most popular bulb in use in homes. More detailed findings from each of the individual evaluation research activities are provided in the report.

Onsite Lighting Inventory – Results Final Report

This report presents the findings of a lighting inventory conducted to understand use, saturation, and purchases of lighting products in Massachusetts households in support of the Massachusetts ENERGY STAR® Lighting Program (the Program). The study also sought to understand conditions during the implementation of the Energy Independence and Security Act (EISA) of 2007 and search for possible impacts on lighting use and purchase behavior that may be the result of the new lighting standards. To conduct this research, the Team performed 150 onsite lighting inventories in Massachusetts households between December 2012 and March 2013. The onsite respondents were recruited through a prior telephone survey among 600 households in Massachusetts also conducted in Winter 2012 and early-2013.

This report provides key findings and recommendations related to CFL use and storage, socket saturations, saturation potential for CFLs and LEDs; bulb storage; CFL and LED purchases; tracking CFLs over time; additional factors that influence CLF use, saturation, and purchase; and Federal lighting standards. Some key findings are described below. The full report presents these and other findings in more detail. Based on the onsite analysis, the Team concludes that most households in Massachusetts use CFLs. Despite high rates of penetration, the number of CFLs in use and the percentage of sockets in which they are installed appears to have leveled over the past three years, and there is evidence that recently purchased CFLs are largely being used to replace installed CFLs that have burned out. Between 2009 and 2010, statistically significant gains were made in increasing the number of specialty CFLs in homes, but this increase was not repeated between 2010 and 2013. LEDs remain an emerging technology in Massachusetts, with very few homes using any LEDs bulbs. When considering the most energy-efficient bulbs types – CFLs, LEDs, and fluorescent tubes – saturation currently stands at around 40%. Use of incandescent bulbs has decreased, but this trend started well before the implementation of the first phase of EISA. Evidence of incandescent stockpiling was mixed and inconclusive.

Lighting Retailer, Supplier Perspectives on the Massachusetts Energy Star Lighting Program Final Report

This draft report presents findings from retailer and supplier interviews conducted in support of the evaluation of the 2011-2012 Massachusetts ENERGY STAR® Lighting Program.

This report provides key findings and recommendations related to the impact of EISA, the market for LED Bulbs, program activity in the hard-to-reach lighting markets, program satisfaction, and program attribution. Some key findings are described below. The full report presents these and other findings in more detail. Store manager awareness of the EISA legislation has increased significantly since 2010. Fifty-five percent of EISA-aware store managers who said that the EISA legislation had impacted their stocking practices said that their stocks of incandescent bulbs had decreased. Two-thirds of the store managers who changed their stocking practices in response to EISA said their stocking of standard CFLs had increased. A slight majority of store managers who reported EISA-related impacts said that their sales and stocks of LED bulbs had increased. There was some disagreement among lighting manufacturers as to how well the EISA-compliant halogen bulbs were selling. According to the program tracking data, only 41 of the 240 participating stores in our sample (17%) had sold LED products through the program. There were many reasons why store managers were not selling LEDs, the most cited reason being that the bulbs are too expensive. Retailer recommendations for greater program access in the HTR sectors include doing more marketing and reducing prices. Seventy-one percent of store manager respondents were satisfied with the program as a whole.

Residential Lighting Shelf Survey and Pricing Analysis Final Report

This report describes research to assess the effect of the Mass Save® residential lighting program on CFL prices in the state relative to competing lighting products, the amount of shelf area dedicated to CFLs for participating retailers, the pricing, the number of packages of bulbs; and the shelf location of CFLs and LEDs relative to other lighting types in participating and former participating stores. Results of the current survey were compared to results from a similar survey conducted in 2010. To perform this analysis, the Evaluation Team designed and analyzed a shelf-stocking survey and a hedonic pricing regression analysis.

After controlling for differences in wattage, sales channel, manufacturer, and other characteristics between discounted and undiscounted standard CFLs, the impact of the PA discount reduced the register price of program-discounted CFLs by an average of $1.76. The point estimate of $1.76 is 86% of the amount of the PA incentive. This compares to an average impact of $1.46 in 2010, which was 117% of the PA incentive amount in that year. Thus, in 2012, it appears that partners are not passing the full PA incentive onto consumers. The PA incentive was associated with a decrease in A-bulb, reflector, and globe CFLs bulb prices ranging from $0.96 to $2.15. This price reduction was less than what was observed in 2010 when the price decrease ranged from $2.33 to $3.61. Stores selling PA-discounted CFLs tend to have a higher percentage of CFL packages than former participating stores. In participating stores, the percentage of shelf space allocated to CFLs increased across most distribution channels from 2010 to 2012.

 

2012 Residential Heating, Water Heating, and Cooling Equipment Evaluation Net-to-Gross, Market Effects, and Equipment Replacement Timing

This report summarizes net-to-gross (NTG) values, the timing of equipment replacement (ER), and the net market effects (NME) for the Cool Smart and High-Efficiency Heating and Water Heating Equipment (HEHE) programs for the 2010-2012 period. The Team based evaluation findings, conclusions, and recommendations on results from an array of data collection activities and evaluation tasks, and report NTG, ER and NME results at the measure level.

The measures responsible for the majority of savings due to equipment installations in the HEHE and Cool Smart programs are central HVAC systems: gas boilers, gas furnaces, CAC, and heat pumps. The estimates from both QIV contractors and participants suggest that significant shares of participants are replacing equipment early. A large proportion of customers replacing boilers and furnaces early are also switching from oil to gas. Policy implications of these findings are further discussed in the report. A range of NTG estimates were developed. All of the average NTG estimates are approximately 0.75 or greater with the exception of ductless mini-splits (0.62). Overall, these NTG estimates are supported by distributor NME analysis.

 

Residential New Construction Program Incremental Cost Final Report

This report provides estimates of the incremental costs per square foot involved in building high efficiency homes that meet the criteria of the 2013 MA Residential New Construction (RNC) Program. Incremental costs above the costs of typical homes being built outside the program are estimated for single family (SF), low-rise multifamily buildings of three or fewer stories (MF 1-3), and mid- to high-rise multifamily buildings of four stories or more (MF 4+) for each of the incentive options offered by the program. The  2013  MA  RNC  Program  offers  five  incentive options for the SF and MF 1-3 building sectors – two based on prescriptive measures and three performance  options  based  on  percent  savings – and  three  incentive  options  for  the  MF  4+ building sector.

Estimated incremental costs vary among the incentive options, particularly for homes in the SF attached building sector. The variation is due to both the kinds of measures that were implemented to satisfy the option requirements and the degree to which the measures were enhanced. Incremental cost per square foot for single family attached homes range from $1.03 to $5.61; multifamily with no master meter range from $0.10 to $1.50; multifamily with master meter gas range from $0.08 to $1.48;  and MF 103 overall ranged from $0.60 to $3.10.

 

Winter 2012 Consumer Survey Results Final Report

The research presented in this report compares the results of a telephone survey performed between December 4, 2012, and January 21, 2013, with the results of two similar surveys performed between December 8, 2011 and January 19, 2012, and June 18, 2012 and August 2, 2012, respectively. The Winter 2011 survey sought to establish a baseline at the onset of the changes in lighting standards resulting from the Energy Independence and Security Act of 2007 (EISA) while the mid- and Winter 2012 surveys searched for possible changes in the lighting market since the initial implementation of EISA. When possible, the Team also compares the results to those obtained from lighting surveys conducted in 2009 and 2010.

The Winter 2012 survey provides evidence that customer satisfaction with CFLs remains steady. A persistent concern with CFLs in Winter 2012 relates to their mercury content and disposal issues. Results of the Winter 2012 survey suggest that use, knowledge, and opinions of CFLs differed between those using many CFLs (experts) and those using few or none of the bulbs (novices). A subset of Winter 2012 survey respondents explained that they wanted to prefer LEDs as their lighting source, but were hesitant to spend $20 for a bulb. When the 75-Watt incandescent is no longer available, there was an increase in consumers who indicated they planed to move to a higher watt incandescent. Respondents to all three survey waves reported relatively low levels of familiarity with A-line CFLs, although there was a slight increase in familiarity with the bulb in Winter 2012 . The results presented for Winter 2012 show that one-fifth of respondents will consider stockpiling incandescent bulbs due to of the 75-Watt phase-out.  Consumers are becoming more familiar with the term “lumens” and understand that it means light output or brightness, but they still buy bulbs based on wattage or wattage equivalence. The full report presents these and other findings in more detail.

2012

Results of the Massachusetts Onsite Compact Fluorescent Lamp Surveys

This report presents the findings of research conducted to understand use, saturation, and purchases of lighting products in Massachusetts households in support of the ENERGY STAR®  Lighting Program. The study also sought to understand baseline conditions early in the implementation of the Energy Independence and Security Act (EISA) of 2007 and search for possible impacts on lighting use and purchase behavior that may be the result of the new lighting standards.  To conduct this research, the team performed 150 onsite lighting inventories in Massachusetts households between January and April 2012. The onsite respondents were recruited through a prior telephone survey among 582 households in Massachusetts in 2011.

Although phone survey results suggest that CFL usage has declined, onsite inventories suggest that most households in Massachusetts use CFLs. Despite high rates of penetration, the number of CFLs in use and the percentage of sockets in which they are installed have leveled off over the past two years and there is evidence that recently purchased CFLs are largely being used to replace installed CFLs that have burned out. Between 2009 and 2010, the number of specialty CFLs in homes increased, but the same results did not occur between 2010 and 2012. LED bulbs remain an emerging technology and few homes in MA use them. Use of incandescent bulbs has also decreased, although some stockpiling is occurring.

Massachusetts Residential Retail Products: Consumer Electronics Saturation

This report presents the findings of research conducted in Massachusetts to examine the use of selected consumer electronics and appliances and to explore related plug load concerns. The report focuses on the types of consumer electronics customers use, the characteristics of those products, and the configurations of products that customers use together. It also examines the use of power strips, both conventional and smart strips. Finally, it presents information on the use and characteristics of refrigerators, freezers, clothes washers, clothes dryers, dishwashers, and water heaters. The findings are largely based on the results of visits to 150 households in Massachusetts conducted between February and April 2012; however, the report also summarize responses to questions about consumer electronics asked in an earlier consumer survey.

The results of the saturation study strongly suggest that program potential exists to reduce plug loads by targeting television and home office equipment. The market for consumer electronics is large, diverse, and changing rapidly. Consumers buy electronics at a faster rate than many other ENERGY STAR qualified products (with lighting being an exception), which means consumers have numerous opportunities to choose between more and less efficient models. The analysis demonstrates that 70% of homes have clothes dryers, and most of these are heated with electricity, which are almost always paired with a clothes washer. It is likely that potential exists for a heat pump dryer program when they become available in the US market. Based on the homes included in this sample, some of the barriers that exist in securing a greater number of HPWH installations include access to a condensate drain and having a large enough space into which the larger HPWH could fit.

Massachusetts Consumer Survey Results 2012

This report compares the results of a telephone survey performed between June 18, 2012 and August 2, 2012, with the results of a similar survey performed between December 8, 2011 and January 19, 2012. The 2011 survey sought to establish a baseline at the onset of the changes in lighting standards resulting from the Energy Independence and Security Act of 2007 (EISA) while the 2012 survey searched for possible changes in the lighting market since the initial implementation of EISA.

The 2011 survey provided evidence that customer satisfaction with CFLs was declining, while the onsite study demonstrated that those dissatisfied with CFLs used fewer of them – even though they still typically used at least some CFLs. The current study, in contrast, found that CFL satisfaction was higher in 2012 than in 2011 but remained below 2009 and 2010 levels. Respondents to both the 2011 and 2012 study reported relatively low levels of familiarity with A-line CFLs, and use of these bulbs is still uncommon. In the 2011 survey results, NMR noted that users of dimmable CFLs generally liked them, but the most persistent complaint is that they do not dim consistently or as much as other types of light bulbs. The results presented for both the 2011 and 2012 surveys as well as the onsite saturation effort suggested that about one-fourth of respondents will consider stockpiling incandescent bulbs, and that some respondents have already started to do so.  While consumers are becoming more familiar with the term “lumens” and understand that it means light output or brightness, they still buy bulbs based on wattage or wattage equivalence. Although telephone survey respondents reported relatively high rates of familiarity with A-line halogen bulbs, the Team found very few of these bulbs in use in onsite households.

Massachusetts Consumer Electronics Potential Qualitative Research Study

The Massachusetts program administrators commissioned this study in order to (1) identify current factors affecting the energy efficiency of consumer electronics products and related savings opportunities, (2) better understand the current state of the market for these products, and (3) assess opportunities for the Massachusetts ENERGY STAR Appliances and Products Program (“the Program”) to address through program activities. A secondary goal of the study was to identify regulatory approaches to improving the energy efficiency of consumer electronics at the federal level, in California, and in other nations with an eye to how these can inform Massachusetts program planning. The literature review for this study was conducted in April and May, 2012. In-depth interviews were conducted between June and August 2012.

Findings from this research suggest that, while a products-focused program approach may be appropriate for refrigerators, freezers, room air conditioners and cleaners, as well as pool pumps, it may not be appropriate for consumer electronics products. Observations from the literature and interviews suggest 4 options to consider as alternatives to the current approach of the consumer electronics program:
1. Abandoning program support for consumer electronics.
2. Limiting support for consumer electronics to encouraging the development of more rigorous energy efficiency specifications and standards.
3. Changing the focus for the consumer electronics portion of the Program from individual products to overall reduction of miscellaneous plug load energy use intensity (EUI) through market transformation efforts, including but not limited to consumer education and behavior change.
4. Maintaining the current product-focused program, but with modifications to improve effectiveness.
Based on an analysis of these options, the report provides several conclusions and recommendations for program consideration. Product-Specific conclusions and related recommendations are also provided.

2012 Home Energy Services Pre-Weatherization Initiative Interim Evaluation Findings Memo

This report summarizes preliminary findings of Phase 1 of the evaluation for the Pre-Weatherization Initiative as part of the Home Energy Services Program. The initiative offers an additional incentive to HES customers who are facing barriers to installing certain assessment-recommended weatherization measures. Phase 1 tasks consisted of: interviews with PA and lead vendor HES program managers; analysis of initiative participation data available to date; and surveys with participant and non-participant NSTAR, National Grid, and Berkshire Gas customers. The Evaluation Team assessed initiative perspectives, assumptions, and processes to determine whether the initiative remains on track to meet its goal of encouraging customers to move forward with weatherization measures by offering additional incentives to remove pre-weatherization barriers.

Based on Phase 1 results, the evaluators drew several initial conclusions. Feedback from interviewed PAs and lead vendors indicated that the 30-day timeframe may be too short.  Incentive amounts may be appropriate for clearing barriers from improper dryer venting, combustion safety issues, and identifying whether knob and tube wiring in a home is live. However, the initiative incentives are not enough to cover the cost of rewiring a home if needed.  During PA and lead vendor interviews, issues arose about whether a turnkey option made sense. To determine the turnkey option’s success, the Evaluation Team will include questions in Phase 2 surveys to determine acceptance and completion rates of customers utilizing turnkey services, compared to those finding their own contractors.

Home Energy Services Impact Evaluation Report: Part of the Massachusetts 2011 Residential Retrofit and Low Income Program Area Evaluation

This report summarizes the gross impact evaluation findings of the 2010 Home Energy Services (HES) Program. The Evaluation Team performed an array of evaluation tasks to inform the reported estimates of per-unit gross savings for each HES measure, including billing analysis, engineering analyses, and calibrated simulation modeling.

The report presents evaluated gross savings estimates for all HES measures, covering electric, natural gas, and oil fuel types. The results are grouped by measure type and primary heating fuel type, although some measures have savings for more than one fuel type. Results are primarily provided in a series of tables.

Wi-Fi Programmable Controllable Thermostat Pilot Program Evaluation: Part of the Massachusetts 2011 Residential Retrofit & Low Income Program Area Study

This report presents the impact and process evaluation of the 2011 Wi-Fi Programmable Controllable Thermostat Pilot Program. The findings, conclusions, and recommendations have been drawn from data collection activities that included billing analyses, site visits, and interviews with program administrator staff, contractors, and participating customers.

Based on findings from the study, the evaluators drew several conclusions. The gas savings for single thermostat installations are considerably larger than for two thermostat installations. Also, when replacing non-programmable thermostats, the Wi-Fi thermostats achieve larger gas savings than when replacing standard programmable thermostat replacements. The electric savings for non-programmable thermostat replacements are effectively equal to those for programmable thermostat replacements. Additionally, electric savings associated with Wi-Fi enabled thermostats vary significantly from one house to another. For some participants, the energy savings benefits of a Wi-Fi enabled thermostat are similar to those of a standard programmable thermostat; however determining whether a participant saves more energy with a Wi-Fi thermostat than they would with a programmable thermostat is difficult to predict because the savings are reliant on participant behavior. In general, participants expressed satisfaction with the Wi-Fi pilot program processes and only few participants made suggestions for program improvements.

Major Renovations Pilot Memo

This memo summarizes findings from the interviews with homeowners, architects and builders involved with completed projects, focusing on satisfaction with the Pilot and suggestions for how the Pilot could be improved or made more user-friendly. In addition, it summarizes a discussion with a HERS rater who worked with several of the completed projects.

Preliminary interviews with potential participants who decided not to enroll in the Pilot showed the interviewed homeowners and builders were committed to energy efficiency; however, they did not think that participating in the Pilot would be cost-effective. Their suggestions for making the Pilot more user-friendly for homeowners focused on providing a clearer description of what the Pilot offered and the cost of participating in the Pilot, especially the cost of hiring a HERS rater. Interviews with homeowners who enrolled in the Pilot showed they were generally satisfied with their experience, satisfied with the rebates and incentives they received, and satisfied with the HERS rater they worked with. Most interviewed homeowners said working with a HERS rater was valuable and that the cost of working with the HERS rater was reasonable for the services  provided.  Participants’ basic suggestions for improvement were consistent with the suggestions from potential participants who decided not to enroll in the Pilot.

Massachusetts Consumer Survey Results 2011

The goal of the consumer survey was to track key indicators of the market for compact fluorescent lamps (CFLs), light emitting diodes (LEDs), and halogens as well as the impact of EISA. Evaluators conducted the survey in December 2011 and January 2012. The survey was timed to coincide with the EISA-mandated onset of the phase-out of 100 Watt incandescent bulbs. Therefore, the results provide a baseline understanding of these important indicators at the earliest stages of EISA.

Nearly all respondents in 2011 indicated that they were aware of CFLs before responding to the survey. This result represents an increase in awareness from the 2009 survey, but does not differ widely from awareness rates in 2010. The survey also provides evidence that customer satisfaction with CFLs continues to decline, but the reasons for the decline remain unclear. Respondents reported increased familiarity with all types of specialty bulbs, but the increase in familiarity was smallest for A-line, covered CFLs. These bulbs, however, closely resemble incandescents and can fit into some type of fixtures that standard CFLs cannot. Users of dimmable CFLs generally tend to like them, but the most persistent complaint about them is that they do not dim consistently or as much as other types of light bulbs. Dimmable users also complain about warm-up time. The results presented suggest that about one-fourth of respondents will consider stockpiling incandescent bulbs and evidence suggests that some of these respondents have already started to do so. While consumers are becoming more familiar with the term “lumens” and understand that it means light output or brightness, they still buy bulbs based on wattage or wattage equivalence.

Massachusetts Mini Baseline Study of Homes Built at the end of the 2006 IECC Cycle Final Report

This report presents the results of a Mini Baseline study of residential new construction building practices in Massachusetts at the end of the 2006 IECC (International Energy Conservation Code) cycle. On-site inspections of 50 non-ENERGY STAR® homes completed from late 2009 through early 2011 were conducted in April through June 2011. As part of this process, HERS (Home Energy Rating System) ratings were completed for each home. The report focuses on three main topics: consistency of study findings with current User Defined Reference Home (UDRH) assumptions; comparison to individual prescriptive energy code requirements at the end of the 2006 IEC code cycle; and identifying differences between custom and spec built homes.

Findings suggest that some current UDRH inputs may underestimate the energy efficiency of current building practices or equipment such as heating systems. In other cases, such as with insulation levels, UDRH inputs may overestimate the energy efficiency of current building practices. Inspected homes are most likely to have at least R-19 conditioned/ambient wall insulation (2006 IECC prescriptive requirement) and least likely to have R-38 flat ceiling insulation (2006 IECC prescriptive requirement). The HERS ratings suggest there is little difference between the energy efficiency of custom homes and spec homes. Custom homes tend to have higher R-value conditioned/ambient wall and flat ceiling insulation, while spec homes tend to have higher R-value floor and foundation wall insulation. Custom homes have slightly more efficient heating systems and spec homes have slightly more efficient water heating systems. Spec homes have lower duct leakage and air infiltration. Custom homes have more energy-efficient light bulbs and a slightly higher percentage of ENERGY STAR refrigerators and dishwashers.

Massachusetts 2011 Baseline Study of Single-Family Residential New Construction Final Report

The 2011 baseline study was conducted with several objectives in mind. The primary objective is to provide a baseline study of the characteristics of single-family homes recently completed across Massachusetts and permitted under the 2009 International Energy Conservation Code (IECC) that did not participate in the Massachusetts New Homes with ENERGY STAR® Program (Program), These homes can be used to update User Defined Reference Home (UDRH) assumptions used in calculating Program savings.

The 2011 baseline preliminary estimated UDRH inputs suggest that the current UDRH assumptions may underestimate the efficiency of some current building practices or equipment. In other cases, study findings suggest current UDRH inputs may overestimate the efficiency of current building practices. Almost all inspected homes (93%) have at least one instance where the R-value of installed insulation is below 2009 IECC prescriptive insulation requirements or mandatory duct insulation requirements. REM/Rate was used to assess heating and cooling system sizing. Results show most heating and cooling systems are oversized.  Average flat ceiling R-values and U-values, cathedral ceiling U-values, unconditioned basement foundation wall R-values, duct insulation R-values, air infiltration, duct leakage and HERS indices are all significantly less energy efficient in baseline homes. Compared to the Mini Baseline Study homes, 2011 baseline homes are smaller. Also, the average HERS Index is significantly lower (better) in 2011 baseline homes (HERS 79) than in Mini Baseline Study homes (HERS 83). Findings suggest that making potential homebuyers aware of ENERGY STAR homes is not enough to get them to act on that awareness when they look for a home or a builder to build a custom home.

Massachusetts Residential New Construction Four to Eight Story Multifamily Pilot Interview Findings

The Four to Eight Story Multifamily New Construction Pilot (Pilot), was introduced in 2010 to address the needs of mid-rise multifamily buildings. NMR conducted fourteen interviews with the Pilot’s sponsors, implementers, and participants with completed projects, examining the Pilot’s goals and objectives, the process of signing up and completing verification, outreach and the timing of projects served, the measures covered, the measures installed, barriers to energy efficient multifamily new construction, and satisfaction. The importance of getting a more efficient home with lower energy bills has risen from 2002 to 2010.

The Pilot has finished its second year with its process operating fairly smoothly to address the needs of the mid-rise multifamily new construction market with participants reporting high overall satisfaction. From the implementer’s perspective, the initial participation process in a similar program could be improved by providing an interactive project tool that would allow the customer user to enter the measure inputs and see the savings and incentives right away. The Pilot’s implementers and participants both pointed out the advantages of a performance based system. Timing remains an issue. About one-half of the fourteen projects that had completed participation, in terms of square footage, were in an early design stage with the HVAC equipment still under discussion when they learned of and signed up for the Pilot. Ideally, program engagement should occur during the conceptual design phase of a new project if it aims to provide technical assistance in considering all applicable energy efficiency measures. The project manager thus emphasized the need for the sponsors and regulators to take a longer term view.

Massachusetts Multifamily Program Impact Analysis Report & Appendix

This impact analysis has two primary objectives for the multifamily program activities of the MA Program Administrators (PAs). First, the goal of the impact work is to provide a set of savings approaches (i.e., algorithms and deemed values) that can be used by all PAs (statewide) in future program years. Second, the analysis collected information to inform program attribution, including the measurement of installation rates, persistence, freeridership, and spillover. Findings, conclusions, and recommendations are draw from a review of each PA’s multifamily program tracking data, the measure-specific savings estimates used by the PAs, as well as a related review of program audit data not contained in the program tracking databases.

The major deliverable of this work can be found in Appendix A. For each program measure, this appendix provides information similar to that provided in the TRM, detailing the recommended algorithm, required inputs to be collected during implementation, and underlying assumptions (to calculate algorithm-based assumptions) or deemed values. In general, evaluators also conclude  that the multifamily program is experiencing excellent installation and persistence rates, though ensuring that programmable thermostats are used to their full capabilities after installation is a challenge. Freeridership also appears to be well under control at a rate of 18% at the program level. “Likeprogram” spillover among property managers has been low. Participant data is tracked differently by each PA and there are significant gaps in the types of program information being collected and tracked.

Brushless Fan Motors Impact Evaluation: Part of the Massachusetts Residential Retrofit & Low Income Program Area Evaluation

This report presents the evaluated gross impact savings of brushless fan motors (BFMs) incentivized as a part of the Cool SMART program for upgrading heating, ventilating, and air conditioning (HVAC) units. This evaluation used on-site spot measurement and long-term metering of motors to determine the savings impact of a statistically significant sample of participants

The evaluated average energy savings of all units in the sample is 246 kWh. The resulting energy realization rate is 41%, which is the ratio of the evaluated savings divided by the deemed savings of 600 kWh. The average energy savings calculated includes interactive savings due to the reduced cooling load from a more efficient motor, assuming an SEER of 8.3. The heating penalty is calculated assuming 78% forced hot air efficiency. The low realization rate is largely the result of a difference between deemed hours of use and logged hours of use. Logging showed that the BFMs operated in Auto mode, cycling to maintain temperature set points, and averaged about 1,493 hours per year in operation, whereas deemed savings value implicitly assumes that the motors operate more than 5,000 hours per year.

Demand Impact Model User Manual

The MS Excel 2007-based model, Demand Impact Model, allows users to estimate seasonal and coincident peak demand savings of energy-efficient measures. Users can trace all inputs and calculations from load shape profiles to peak period definitions to measure level energy and demand savings.  This document provides an overview of this model and information about using it.

None available at this time.

Low Income Single Family Program Impact Evaluation: Part of the Massachusetts Residential Retrofit & Low Income Program Area Evaluation

This report summarizes the impact evaluation of the 2010 Low Income Program using an array of data collection activities and evaluation tasks, including billing and engineering analyses, to determine the evaluation findings, conclusions, and recommendations presented.

The primary results of this evaluation were reported as the average number of measures installed per household and the average energy savings per household per year by measure type. Natural gas measures included insulation and air sealing measures, heating system replacements, duct insulation, duct sealing, and DHW measures. Electric measures included insulation and weatherization measures, refrigerator replacements, second refrigerator removals, freezer replacements, window AC replacements, CFLs, torchieres, fixtures, DHW measures, and other measures from TLC kits. Oil measures included insulation and air sealing measures, oil heating system replacement, boiler reset controls, programmable Thermostats, Furnace Fans, duct insulation, duct sealing, and DHW measures.

Solar Hot Water Pilot Program Evaluation Report: Part of the Massachusetts Residential Retrofit & Low Income Program Area Evaluation

This report summarizes the findings of an evaluation of National Grid’s Solar Hot Water (SHW) Pilot Program. Data for this report were obtained through billing analyses, customer surveys, site visits, and engineering reviews of solar hot water systems installed through this program over the past several years.

The SHW Pilot Program gross gas savings, based on engineering estimates and modeling, is predicted to be approximately 701 MMBTU/yr, with average savings of approximately 14.2 MMBTU/yr per program participant. The SHW Pilot Program net gas savings, based on a billing analysis to account for takeback and other effects, is approximately 512 MMBTU/yr, with average savings of approximately 10.9 MMBtu/yr per program participant. Site visits confirmed the quality of SHW installations, with the only consistent problem being the lack of a UV-resistant jacket over the foam insulation on outdoor piping. The most common non-plumbing issue observed was excessive shading of solar collectors. The cost-effectiveness of SHW systems installed through this program is low, with simple post-rebate payback periods to customers of 50 years, on average.

Home Energy Services Net-to-Gross Impact Evaluation: Part of the Massachusetts Residential Retrofit & Low Income Program Area Evaluation

This report summarizes the 2011 Home Energy Services (HES) Program net-to-gross (NTG) impacts. Evaluation findings, conclusions, and recommendations are based on results from several data collection activities and evaluation tasks including participant and non-participant surveys and self-report and discrete choice (DC)-based assessments of measure-level NTG ratios. The Evaluation Team estimated measure-specific NTG participant free-ridership and spillover, as well as non-participant spillover for some measures, to determine NTG ratios.

The program-level NTG ratio was estimated to be to be 113%. For direct install measures, the CFL NTG estimate is 73% and air sealing is 129%. For incentive measures, the insulation NTG ratio estimate is 123% and the refrigerator estimate is 86%. The evaluation does not report NTG components that overlap with other programs, such as CFL NPS and refrigerator PS, which would double count the impacts of the concurrent Upstream Lighting and ENERGY STAR Appliance Rebate programs.

Heat Pump Water heaters Evaluation of Field Installed Performance

This evaluation quantified the in-situ performance of three recently-released HPWH products (General Electric GeoSpring™, AO Smith Voltex®, and Stiebel Eltron Accelera®300) through the installation and monitoring of 14 units in Massachusetts and Rhode Island for over one year.

Though a small sample set, the overall performance of these 14 HPWHs shows great promise for this technology. In general, these HPWHs were more than twice as efficient as a traditional electric resistance tank water heater; though there are large variations in overall efficiency as performance is dependent on ambient temperature/relative humidity, mains temperature, hot water setpoint temperature, and water usage (consumption and concentration). Despite some issues with siting and location, these new units performed with remarkable energy and cost savings over ERWHs. Evaluators suggest that homeowners need to be aware that these integrated HPWHs often are not a direct size-for-size replacement for existing water heaters due to their lower heat recovery rates. Also, manufacturers’ minimum space requirements need to be adhered to. Depending on air conditions in the water heater location, expectations of the efficiency performance of these units need to be adjusted. To ensure proper operation of a HPWH, filters need to be cleaned regularly, condensate drains need to be checked periodically.

Massachusetts 2011 Low Income Program Process Evaluation Final: Part of the Massachusetts Residential Retrofit & Low Income Program Area Evaluation

This report summarizes the process evaluation of the 2011 Low Income Single-Family (LISF) and Low Income Multifamily Retrofit (LIMF) programs (collectively known as the Low Income Program). The
evaluation findings, conclusions, and recommendations presented are based on results from an array of data collection activities including interviews with Program Administrator (PA) Low Income program managers, Community Action Program (CAP) agencies, stakeholders from the Low Income Energy Affordability Network (LEAN), program energy auditors, installation contractors, and multifamily building managers.

The Team’s evaluation revealed numerous ways in which the Low Income Program is succeeding and identified opportunities for improving processes and outcomes. For  example,  program evaluators recommend that PAs provide savings goals for the Single Family program to their lead agencies to improve transparency between PAs and program implementers.  For the Multifamily program, the LIMF Advisory Committee should encourage more standardization across PAs by developing standardized project screening criteria or a tool to determine savings and cost effectiveness for both gas and electric projects.  Additional recommendations are also described.

Massachusetts Multifamily Market Characterization Potential Study

This study assesses the potential energy-efficiency savings available in Massachusetts’ multifamily buildings. A market characterization was done based on data collected from workshops with multifamily property managers and PA program implementers from MA, as well as on-site visits from 193 multifamily buildings in MA. These data, supplemented with information from PA data tracking systems and secondary
sources, provided the foundational elements for estimating the technical, economic, and achievable potentials.

Results of the study for electricity indicate that 2.8 million MMBTU of technically feasible, electric energy-efficiency potential will be available by 2030. Once screened for cost-effectiveness, this technical potential translates into an economic potential of 1.8 million MMBTU. Should all of this cost-effective potential be deployed, the result would be a 14% reduction in 2030 forecast energy consumption. These percentage estimates of electric technical and economic potentials are comparable to those for multifamily buildings in a 2010 Consolidated Edison potential study. For gas, results indicate that 7.0 million MMBTU of technically feasible, natural gas energy-efficiency potential will be available by 2030. This technical potential translates to an economic potential of 5.2 million MMBTU. Summing the totals across fuels, evaluators estimate 34 million MMBTU of multifamily electric and gas energy consumption in 2030 and indicates nearly 10 million MMBTU (29% of baseline sales) of technically feasible energy-efficiency potential and 7.0 million MMBTU (21% of baseline sales) of total economic potential will be available by 2030.

Massachusetts Multifamily Program Process Evaluation: 2011 Energy Efficiency Annual Report

This report summarizes the process evaluation of the 2011 Multifamily Program. The evaluation findings, conclusions, and recommendations presented have been drawn from an array of data collection activities, including interviews with Program Administrator (PA) staff and Implementers, surveys of participating property managers and tenants, a focus group of property managers, a literature review, and a robust data review.

Although streamlining of the Multifamily Program participation process has greatly improved, a lack of awareness of the program and of what the program can offer was reported as a significant barrier to participation in the program. In the coming years, increasing program goals will likely require the Multifamily Program to seek out new opportunities for energy savings. On average, Multifamily Program participants were highly satisfied with almost all aspects of the program. Property managers and tenants expressed a need for follow-up technical assistance on some of the measures installed through the program. Additional findings are also described.

2011 Home Energy Services Packaged Measure Pilot Process Evaluation: 2011 Energy Efficiency Annual Report

This report summarizes an evaluation of the 2011 Home Energy Services (HES) Packaged Measure Pilot (pilot) offered by the electric and gas Program Administrators (PAs) of Massachusetts. In March 2011, the PAs launched the pilot to encourage HES participants to install a greater number of the recommended measures, thereby ensuring the HES program achieves deeper savings within each home. The report focuses on evaluating the effectiveness of the additional incentive in meeting the pilot’s stated goal, as well as assessing the delivery of the pilot itself.  Results are based on interviews with PA Program Administrators and Vendors as well as participant and non-participant surveys.

The PAs offered eligible new and past HES Program participants a total of 641 packages; however, only 52 customers decided to participate (8%). Of the 52 who accepted the pilot, 46 were new participants and six were past participants.  These results indicate that the pilot was more likely to be accepted if the participant was new and the pilot was offered at the time of their HES audit, rather than if the participant was offered the pilot after having completed the audit.  Pilot participants were also more likely to adopt the measures than were the typical HES participants eligible for the measures. This demonstrates that the additional incentive of the pilot may increase the installation of all measures that were packaged in the pilot. Evaluators identified barriers to participation including high upfront costs, timeframe requirements, and extra paperwork. Generally, the PAs and vendors expressed dissatisfaction with the pilot design and results, although they also named a few implementation elements that were satisfactory. Both PAs and vendors suggested improvements such as extending the offer timeframe, re-evaluating packages, and increasing marketing.

Home Energy Services Realization Rate Results Memo

This memo describes results of a completed impact evaluation of two Home Energy Services (HES) program measures: insulation and air sealing. The HES program is available to single-family residential customers living in buildings with up to four units throughout the Commonwealth of Massachusetts. Ex post savings for both measures were determined using a billing analysis and an engineering analysis.

Results report ex post savings determined for insulation and air sealing by PA and all four fuel types. The ex ante and ex post savings as well as realization rates varied across PAs. This variation resulted from multiple factors, including: the profile of customers served, the mix of measures installed, the weather the service territory experiences, and the vendor-specific software used. Comparing Commonwealth-wide realization rates for insulation and air sealing from the August 2012 evaluation (based on data prior to the software updates) and the current evaluation (based on data after the software updates) indicated that the updated software produces ex ante savings estimates that are closer to the ex post estimates determined by the Evaluation Team. Specifically, this evaluation resulted in a Commonwealth-wide realization rate of 76% as compared to the 61% realization rate found in the previous evaluation (for insulation and air sealing only). Additionally, average savings per participating home – using natural gas for heat – also increased for the participant homes included in the 2013 evaluation (139 therms) versus participant homes included in the 2012 evaluation (92 therms) due to the increased adoption of recommended measures.

2011

Cross-Cutting Behavioral Program Evaluation

 

Appliance Turn-In Program Evaluation

 

New Homes with ENERGY STAR® Program

 

ENERGY STAR Lighting Program

 

2010 Residential Retrofit Program and Low-Income Programs

2010