Residential Program Studies

Synopsis: The purpose of this study was to document the persistence of savings after the Eversource Home Energy Report program was halted. The study sought to determine first year savings in absence of the program, and the rate of decay of these savings. Additionally, any additional factors in savings persistence were identified.

Conclusion: In the first year after program termination, the evaluation team found aggregate savings net of uplift for all waves to be 37,359 MWh and 857,253 Therms. Total electric savings across all waves decreased by 5.71 percent and total gas savings across all waves decreased by 1.91 percent from the final program year to the first year after reports ceased. Percentage savings in both the final year and first year after program termination were higher and more likely to be statistically significant at the 90% level for earlier program waves. This led to lower, more stable decay rates for earlier waves. It is possible that length of program enrollment is correlated with the persistence of savings after program termination. Due to the uncertainty surrounding the savings findings in later waves, the evaluation team recommends periodically updating this study to develop stronger knowledge of the trends in persistence, which will help with future planning cycles and filings.

Renovations and Additions Incremental Cost Study

Synopsis: The goal of this study is to estimate the incremental costs associated with participating in the Program Administrators’ (PAs’) Residential Renovations and Additions Program (referred to as the R&A program). The study is designed to estimate both retrospective and prospective incremental costs, reflecting the previous design of the program but also addressing recent program changes as it pertains to baseline assumptions. A secondary goal of the study is to provide a brief summary of program practices.

Conclusion: Table 1 in the memo provides the incremental cost estimates for the R&A program in units of dollars per square foot of conditioned floor space (CFA) across program homes. It shows the incremental costs from renovations and additions separately. Renovation costs were calculated for two baselines: the incremental costs of the pre-existing conditions baseline and the incremental costs of the industry standard practice baseline (ISP). The incremental cost associated with participating in the R&A program is estimated to be $9.11 per square foot of home CFA (excluding any addition area) for renovations using the pre-existing conditions baseline and $3.05 using the ISP baseline. The incremental costs for the additions portion of the program are estimated to be $1.28 per square foot of addition CFA.

Wi-Fi Thermostat Process Evaluation: Customer Survey Findings Memo

Synopsis: This report outlines the results of the customer survey portion of the wider Wi-Fi thermostat process evaluation. The survey investigated customer behavior regarding Wi-Fi thermostats, characteristics of Wi-Fi thermostat adopters, and concurrent household changes typically associated with the installation of a Wi-Fi thermostat. The survey was designed to explore the causal mechanisms and channels behind savings, paint a clearer picture of customer behaviors that may affect savings (e.g., setpoints, schedule, fiddling behavior), and qualitatively explore which characteristics may be changing over time in the thermostat-adopting population.

Conclusion: Survey results suggest participants’ thermostat behavior changed after purchasing a Wi-Fi thermostat, as they started to take advantage of the device’s scheduled and automated features and the ability to adjust setpoints remotely. Respondents reported using the automated or learning mode, or setting a program or schedule, more frequently with their new Wi-Fi thermostat. They also reported manually controlling their devices less frequently. Respondents indicated they used more efficient average setpoints with their new Wi-Fi thermostats relative to old thermostats, specifically at night and when the home was unoccupied. Demographic findings from the survey suggest that though many types of households participated in the program, few were low or moderate income. Only 14% of respondents reported low-to-moderate incomes for their household size, compared with 29 – 41% of Massachusetts household, while 62% reported incomes above 120% of the state median for their household size.

Comprehensive TRM Review

Synopsis: This report summarizes the mid-term update to the Technical Reference Manual (TRM). The TRM is foundational to the calculation of savings claimed by residential energy efficiency programs in the state. As a result, mid-plan revision of the TRM is critical to ensure that it uses the best available data and that its deemed savings values are based on credible, standardized assumptions. This revision process improves the accuracy of the savings claimed by energy efficiency programs that rely on the TRM. The primary goal of this review is to ensure that relevant data from the RES 1 Baseline Study and other recent studies are incorporated into the TRM. A secondary goal of this effort is to assess the quality of the values in the residential portion of the TRM and to conduct analysis to improve these parameters.

Conclusion: The scope of this TRM review is limited to the market-rate residential efficiency measures described in Section 1 of the 2019-2021 Plan Version of the Massachusetts TRM. The evaluation team’s review focuses on assumptions pertaining to the calculation of deemed kWh, kW, and MMBtu savings. Table 1-1 in the report contains the listing of all measures updated by the TRM review. [/wptab] [end_wptabset]

Residential Coordinated Delivery Virtual Home Energy Assessment Study

Synopsis: In March 2020, in response to COVID-19 social distancing requirements, the Massachusetts Program Administrators (the PAs) temporarily replaced the Residential Coordinated Delivery (RCD) initiative’s customary in-person home energy assessments (HEA) with virtual (i.e., remote) home energy assessments (VHEA). This report provides initial evaluation results as to the efficacy of the VHEAs compared to a traditional HEA, and what role they may fill in future program offerings.

Conclusion: The study found that while VHEAs are a satisfying experience for homeowners, that they typically result in lower savings/lesser outcomes than their in-person counterparts. However, they were found to effective enough to warrant their continued implementation in the Residential Coordinated Delivery offering, as they provide an immense opportunity for increased accessibility and convenience for both contractor and homeowner.

2020

Lighting Supplier Insights

2019 Massachusetts Summer Thermostat Optimization Evaluation

Synopsis: The goal of this year’s thermostat optimization evaluation was to explore persistence from thermostat setpoint schedule adjustments from year to year and to estimate savings in electric consumption and peak demand. The evaluation included both an exploratory persistence analysis of thermostat setpoints and runtime, and an impact analysis of energy and demand savings. The evaluation also yielded realization rates, comparing evaluated and vendor-generated savings; a demand-to-energy factor; and an on-peak savings percentage.

Conclusion: The study found that cooling savings per thermostat was 19.7 kWh and 46 W per thermostat, which equates to 3.0% of energy requirements and 6.7% of demand. With 33,941 statewide participants, the total verified on-peak savings were 494,792 kW, and the total verified off-peak savings were 173,846 kW.

2019/20 Massachusetts Winter Thermostat Optimization Evaluation

Synopsis: The goal of this evaluation was to conduct an energy impact evaluation with an exploratory analysis using thermostat telemetry data from Google’s SS program in Massachusetts during the 2019/20 winter. In the 2019/20 winter, the program was deployed to Massachusetts Program Administrator (PA) customers who had a Nest thermostat installed, specifically natural gas customers for Eversource and National Grid (including non-natural gas devices in natural gas homes) and non-natural gas customers for Cape Light Compact.

Conclusion: The study found the program to have some significant impacts in terms of heating load saved. For natural gas based homes, 1.2 MMBtu and 5.4 kWh was saved per thermostat, which equates to 3.6% of both fuel and electric heating load savings. For non-natural gas heated homes, 0.8 MMBtu of the primary fuel was saved along with 15.2 kWh of electric savings per thermostat. This equates to 2.7% and 6.0%. The results of the study indicate that the program does have a substantial impact on winter heating loads.

Massachusetts Residential Lighting On-Site Study

Synopsis: The goal of this study was to update estimates of lighting saturation and other market indicators. The data came from on-site lighting inventories conducted in Massachusetts homes that are part of the Residential Baseline Study.

Conclusion: In 2020, efficient bulb saturation (the percentage of sockets filled with LEDs or CFLs) was 61%, compared to 28% for incandescent and halogen bulbs. Overall, LED saturation was 45% in 2020, composed of ENERGY STAR LEDs (37%), non-ENERGY STAR LEDs (3%), and LED fixtures (5%). LED saturation increased from 2017 to 2020 for all bulb shapes, while saturation of CFL, incandescent, and halogen bulbs decreased. In 2020, reflectors had the highest LED saturation (60%) of all bulb types, followed by A-lines (47%) and candles (47%).

2019 Regional Lighting Sales Data Analysis

Synopsis: This study examines light bulb market share, shipment, and prices in the New England region. The study examines market share and bulb prices (provided by CREED) in four study states: Connecticut, Massachusetts, New Hampshire and Rhode Island. It company the study states to non-program areas to determine the impact of the programs.

Conclusion: This study provides an update to the growing trend of market transformation in the LED lighting space. LED market share is higher in program states, but non-program states are quickly catching up. LEDs are now surpassing most other lighting devices in market share.

Energy Optimization Measures and Assumptions Update Memo

Energy Optimization Study Summary for 2019 Plan Year Report

Energy Optimization Measures and Assumptions Model

Synopsis: This memo summarizes the evaluation team’s work to update the Energy Optimization Model (EOM) per the requests of the Massachusetts Program Administrators (the PAs) and the Energy Efficiency Advisory Council (EEAC) consultants. The evaluation team initially developed the EOM in October 2018 to estimate the costs and benefits associated with a variety of energy efficiency measures that use electric heat pumps and natural gas heating equipment to displace the consumption of delivered fuels. The PAs have used this model to develop prescribed savings values used for energy efficiency measures that involve fuel switching. This memo describes updates implemented in the model, including revised input values, new installation parameters, and new measures covering groundsource heat pumps (GSHPs). In addition, this memo describes further improvements implemented in the model as part of the C&I Energy Optimization Study (MA19C04-E-EO) conducted for the Massachusetts commercial PAs.

Conclusion: The update to the Energy Optimization Model largely consist of 5 primary tasks/goals. First was to introduce pre- and post-weatherization heating and cooling loads, to determine the impact that weatherization may have on the energy optimization models. Efficiency levels and performance curves for heat pumps were updated to be more accurate to the most recent data. Modifications were made to heat pump calculations to determine an effective way to estimate real-world heating system performance based on lab-rated performance data. Most major inputs into the model were revised to some degree to reflect most recent programmatic data. Finally, ground-source heat pump measures were added to the model.

2019/2020 Residential Energy Storage Demand Response Demonstration Evaluation – Winter Season

Synopsis: This study evaluates National Grid’s and Unitil’s battery response demonstrations during the winter of 2019-2020. National Grid’s program was a “bring your own battery” program that had 148 devices enrolled in it. Unitil’s demonstration paid for and installed battery storage for 4 participants. The goals of the program demonstrations were to determine the validity of battery response programs for reducing system peak demand and flattening the solar PV output curve for residential customers.

Conclusion: National Grid’s demonstration typically had 60% of devices enrolled on a given event day successfully perform in the event. If you exclude devices without telemetry data then 78% performed, and if you only consider the 102 devices that participated in at least one event then 75% performed. Per event the program saved 559 kW on average, which works out to be 6.9 kW/device. Unitil’s program had an average of 2.4 kW/battery discharge over two-hour event periods, resulting in an average of 8.3 kW per event. Battery discharge was front loaded, and often would be fully discharged before the event was up.

2019 Electric Vehicle Supply Equipment Direct Load Control Demonstration – Process Evaluation

Synopsis: This memo summarizes the methods and results of the participant survey and literature review conducted for the evaluation of Eversource’s 2019 Electric Vehicle Supply Equipment (EVSE) Direct Load Control (DLC) demonstration. THE EVSE DLC demonstration aims to reduce peak demand of home electric vehicle charging load by way of Wi-Fi-enabled Level 2 EVSE. Participating customers were provided incentives for allowing Eversource to restrict their EV charging during demand response events, as well as for access to their charging data.

Conclusion: During the period evaluated, Eversource had 74 customers enrolled in the program, and conducted 4 events, with a duration of 3 hours each. Survey respondents were generally very pleased with their program experience, and made suggestions to improve communications. Based on survey feedback, electric vehicles were most often connected to home chargers between 6pm and 6am, indicated that demand response events outside of these hours may see limited participation. The EVSE DLC demonstration shows promise, but more research is needed into best practices and ensuring that events have high participation to create impact.

2019 National Grid Behavioral Demand Response Evaluation Findings

Synopsis: The objective of this study was to estimate demand reductions from National Grid’s 2019 behavioral demand response (BDR) program via a literature review and an analysis of a subset of National Grid Massachusetts customers with interval utility metering. The literature review of BDR programs consisted of case studies of four BDR programs that were most similar to National Grid’s program. The only dissimilarity in the programs researched was that they included customized post-event communications that National Grid’s program does not have.

Conclusion: The four programs researched had savings for residential BDR participants during event hours ranging from 1.3-4.0%, with an average of 2.3%, of whole home electricity usage. Because all these programs include a customized post-event communications, it is likely that their savings may exceed what National Grid’s program achieves. In the actual evaluation of National Grid’s program, it was found that the event savings from the program were not statistically different from non-event customers. This may change with a larger sample size, as the initial sample size of the BDR program is on the small size.

Residential New Construction Non-Program Model Review

Synopsis: The goal of this study is to assess the relative merits of using Home Energy Rating System (HERS) data, obtained from Ekotrope Inc., as an additional data source to inform research on the performance of new homes built outside of the Program Administrators’ (PAs’) low-rise residential new construction (RNC) program. This study seeks to evaluate how non-participant sample homes compare to the baseline study homes, as well as evaluating the strength and weaknesses of the Ekotrope HERS data, and what role it may play in the future of the program.

Conclusion: The non-participant data provided by Ekotrope was heavily weighted towards stretch code communities, with 75% of the homes being in these municipalities, this provides a somewhat skewed sample to compare against. Code compliance rates were found to be significantly different between the Ekotrope samples and the baseline samples, while HERS rating were not found to be significantly different. Many details, such as prevalence of propane heating, air infiltration, and details related to insulation were found to be different between the two samples. The study suggests that going forward, a blended methodology using HERS-rated homes, energy audits, and other methods would likely create the most accurate outcomes for updating new construction baselines.

Passive House Offering Program Theory and Logic Model

Synopsis: This report presents a final program theory and logical model for the Passive House program offering. The program theory is a formal description of the program’s activities and the short-, medium-, and long-term outcomes each activity is designed to achieve. The logical model is a graphical representation oft he program theory. By providing the theory and logical model for the program, a better understanding of why the program exists and what it hopes to accomplish can be obtained.

Conclusion: The report outlines the activities, outputs and outcomes at different time intervals of the passive house offering. The eventual goal of the program, like other energy efficiency offerings, is to achieve market transformation so that the relevant activity, passive house construction, has a defined space in the market in absence of the program.

2019 Residential Wi-Fi Thermostat Direct Load Control Offering Evaluation

Synopsis: The Residential Wi-Fi Thermostat Direct Load Control (DLC) offering evaluated in this study is part of the Program Administrators’ broader active demand response initiatives. The offering aims to reduce peak demand of residential central air conditioning (AC) systems by directly controlling the scheduled setpoints of Wi-Fi thermostats on specific days during the summer. Participating customers are provided sign up and annual incentives for providing the right to adjust their temperature setpoint during events, as well as for access to their thermostat data.

Conclusion: Events during the 2019 demand response (DR) season were generally 3 hours with a 3 – 4°F temperature adjustment during events. Events were called between the hours of 2 p.m. and 7 p.m. on non-holiday weekdays between June 1 and August 31 (evaluated DR season). The study found that 96% of thermostats that were enrolled since September 30, 2018 were enrolled through the entire 2019 DR season. 70% of respondents found that the temperature change was noticeable and 50% found it less comfortable. The study also found that the average savings per treated thermostat was 0.8 kW, although it was only 0.6 kW per participating thermostat. These savings were found to be a significant portion of the cooling load of the participating households.

Consistent Methodology for Self-Reported Residential Net-to-Gross Measurement

Synopsis: The primary purpose of this study was to develop specific questions and algorithms when using the self-report approach for downstream residential net-to-gross (NTG) studies. This study will allow for a consistent measurement of net-to-gross across residential downstream programs and measures. The study team along with an advisory group made up of NTG experts from the residential evaluation team, the PAs, and the EEAC reviewed best practice approaches in self-report net-to-gross studies in the residential sector to develop a recommended approach to future residential net-to-gross studies.

Conclusion: The study developed questions and algorithms for the self-report approach for six different categories of net-to-gross studies. Participant event type, participant free-ridership, participant spillover, contractor free-ridership, contractor spillover, and participant and contractor free-ridership reconciliation were the categories of studies that had questions and algorithms created.

2019 Residential New Construction Baseline/Compliance Study

Synopsis: Similar to the Residential Baseline Study, the Residential New Construction Baseline/Compliance study seeks to determine the characters of both program and non-program homes to better understand the impact of programmatic activities. This helps establish new baselines for the New Construction program to aid it in continuing to push standards. One important outcome of the stuy is providing an update to the User Defined Reference Home (UDRH), the model home to which program homes are compared to to calculate savings.

Conclusion: The study found that the new construction market has evolved since the last major evaluation in 2015. New non-program homes have become 14% more efficient since then, and while program homes are still significantly more efficient than non-program homes, the difference between the two is decreasing. It was also found that the program comprises 70% of the new construction market. The study did not detect a statistical difference in efficiency between homes built under the new stretch code versus those built under the base code. Code compliance in new construction was also found to be 96%, which is the highest one of these studies has found. Finally, building practices have been found to be changing to comport with the high efficiency standards.

Evidence for Market Effects from Support for Ductless Mini-split Heat Pump Integrated Controls

Synopsis: This memo assess the evidence for the program administrators (PA) support of ductless mini-split heat pump (DMSHP) integrated controls having made a substantial impact on the market for these controls and for cold-climate DMSHPs. The memo then goes on to lay out how the market effects from the PAs’ support of this market could be quantified through a market-focused approach to measuring net-to-gross using the structured expert judgment method.

Conclusion: The study found that the PAs are partly responsible for launching the Massachusetts market for DMSHP integrated controls for use in homes heated primarily by delivered fuels and accelerating the development of this market in both Massachusetts and the Northeast region. However, the market is not yet established. The study lays out some key considerations, providing advice towards future research into this subject, and how continued research may benefit the program.

Residential Baseline Study

Appendix B-3 MA Statewide Saturation Results

Appendix B-4 MA Statewide Characterization Results

Appendix B-5 Electric Energy and Peak Demand Results

Appendix B-6-1 HVAC Electric Load Shape Results

Appendix B-6-2 Water Heating Electric Load Shape Results

Appendix B-6-3 Kitchen Electric Load Shape Results

Appendix B-6-4 Laundry Electric Load Shape Results

Appendix B-6-5 Miscellaneous Electric Load Shape Results

Appendix B-6-6 Lighting and Other Electric Load Shape Results

Appendix B-7 Gas Energy and Peak Demand Results

Appendix B-8 All End Use Gas Load Shape Results

Synopsis: The primary goal of the Massachusetts Residential Baseline Study is to collect saturation, characterization, and usage behavior data for all major electric and gas appliances, heating and cooling equipment, and electronics in Massachusetts homes. This data supports energy and peak demand savings calculations for program evaluation and design, and provides additional insight on the savings potential in the existing residential buildings market.

Conclusion: The findings of the Massachusetts Residential Baseline Study are extremely varied and are best summarized on pages 2 through 9 on the main report. These pages summarized the main electric end use findings, including annual electricity consumption, summer and winter demand, and end-use saturations. It also goes into gas and delivered fuels end use saturations and consumption and the relationship between thermostats and space temperatures.

Residential Lighting Hours-Of-Use Quick Hit Study

Synopsis: This study re-analyzed metered hours of use (HOU) data collected as part of the 2014 Northeast HOU study to take into account the effects of efficient lamp saturation on HOU, as well as exploring and addressing the distribution of HOU. The revised model was applied to recent lighting inventory data (RLPNC 18-10) to produce estimated HOU values. The study also calculated the percentage of sockets whose HOU rounds to zero by room type and by efficient vs inefficient lamps to account for the distribution bump at zero. Using the state-wide collaborative process, the PAs and EEAC consultants then came to a consensus on how the HOU values would be implemented.

Conclusion: The models revealed a positive and statistically significant relationship between saturation and HOU. HOU initially declines at low levels of saturation but as saturation levels increase HOU also begins to increase for each room type. The result was that recent increases in saturation lead to higher estimated HOU.

Residential New Construction Incremental Cost Update

Synopsis: This report provides incremental costs for single-family homes and low-rise multifamily housing units, describing the incremental costs per square foot of conditioned floor area (CFA) to move from baseline construction practices to construction practices commonly found in recent RNC program projects. High-rise multifamily buildings are not included in the scope of this update. The updated incremental cost calculations included in this study are intended for use in the PAs’ 2019 annual report filing.

Conclusion: The updated incremental cost estimates for the low-rise RNC program are presented in Table 1, and are presented in units of dollars per square foot of CFA across program homes. The incremental costs for all home types have decreased from the original 17-14 incremental cost study. Non-program home performance has increased for nearly all measures since the 17-14 study at a faster rate than program homes. This yields lower incremental costs to participate, given that it takes limited labor and materials to achieve program performance relative to a rising baseline.

2018 – 2019 Winter Thermostat Optimization Evaluation

Synopsis: This study evaluates the Winter Seasonal Savings (SS) program, which is a thermostat optimization program offered by Google designed to make incremental adjustments to scheduled setpoints over a 3-4-week period to achieve energy savings and demand reductions during the winter while maintaining customer comfort. On average, scheduled setpoints are adjusted downwards by 0.5°F-1.5°F during the heating season, with the largest temperature adjustments taking place when customers are typically asleep (i.e., late evening through early morning).

Conclusion: The winter SS program reached 52,238 customers starting on January 14, 2019. Through April 30, 2019 they found that in natural gas customers, the program saved 4.7% of the fuel based heating load, and 4.6% of the electric base heat load. In non-natural gas customers (delivered fuels or only electricity) they found that fuel savings were 2.8% of the heating load and electric savings were 6.2% of the heating load. Realization rates were found to be 97% for MMBtu savings and 98% for kWh savings. These finding indicate that there is a large benefit in control of winter demand, as significant savings can be achieved

Residential Coordinated Delivery Logic Model

Synopsis: This memo provides a logic model diagram for the statewide Residential Coordinated Delivery (RCD) program, serving existing residential buildings throughout Massachusetts under the Mass Save brand. The logic model diagram provides a graphical representation of the activities, outputs, and outcomes expected from successful execution of RCD. The accompanying tables and explanatory text provide additional detail for reviewers and provide examples for how the Program Administrators (PAs) could potentially measure RCD’s progress toward these outputs and outcomes over time. The memo also includes a supplementary participation path diagram that depicts, at a high-level, how participants move through RCD delivery process.

Conclusion: Figure 1 on the fourth page of the memo serves as an excellent summary of the Residential Coordinated Delivery program’s logic model. In essence, there are a wide assortment of parallel activities that go into creating opportunities for positive short, mid, and long-term outcomes of the program beyond what a prescriptive program could achieve.

Renovations and Additions Market Characterization Study

Synopsis: This memo discusses the results of a detailed assessment of the size, scope, and potential savings of the single-family renovations and additions (R&A) market. The study was conducted to inform the design of the PAs’ current R&A offering within the Residential New Homes and Renovations imitative. For the assessment, a detailed review of online building permits and four other primary data collection activities with market actors were synthesized together to obtain cohesive results.

Conclusion: The study found that the single-family R&A market in Massachusetts includes approximately 130,000 projects annually, inclusive of both permitted and non-permitted projects. Of this market, roughly 55,000 are program-eligible, significantly larger than just the new construction market, which is approximately 7,200 homes. This indicates that the R&A market has a very high potential for program intervention and possible savings. Estimated potential savings were over 1,000,000 MMBtu per year, with a very high portion of that coming from home heating.

Delta Watt Report Memo

Synopsis: The Delta Watt study was conducted to update inputs that are used to calculate LED delta watts in the RLPNC 17-6 Lighting Market Adoption Models (MAMs). The equivalent wattage of bulbs with similar lumen output and the sales weights based on program tracking data tie delta watts directly to program sales. These equivalent wattage bins and sales weights had been previously updated in 2018, and this study provides a further update to that.

Conclusion: For full details of the results of this study, see table 1 on page 2 of the study. Beyond minor updates to the delta watts, the most notable result of this study was to continue to use the market share values for screw-based bulbs from the 2017 MAMs. This choice has been consistent between this update and the previous 2018 update. It was made due to the uncertainty regarding federal lighting standards in the coming years.

Home Energy Services Realization Rate Assessment

Synopsis: The Home Energy Services (HES) Realization Rate Assessment was conducted to determine the efficacy of air sealing and insulation measures installed through the program. The measures, collectively known as weatherization, were assed to aid in determine deemed savings from the programs.

Conclusion: The HES Realization Rate Assessment determined that the average natural gas-heated HES participant receiving air sealing and/or insulation realized 73% of the total energy savings predicted by the HES Lead Vendor (LV) or Home Performance Contractor that treated their home. The results of this evaluation were nearly identical to the realization rate (0.76) determined by the prior HES impact evaluation (Completed in 2013) for natural gas weatherization. The similarly of these results, as well as the fact that none of the PAs or HES LVs achieved a statistically significant realization rate for natural gas higher than 0,85 as a part of either evaluation, indicates a persistent disconnect between LV ex ante and evaluator ex post savings.

Nonparticipant Market Barriers Study

Synopsis: The Nonparticipant Market Barriers Study was conducted in conjunction with the Customer Profile study to gain insight into what conditions and characteristics lead to nonparticipation in Mass Saves programs. While the Customer Profile Study used secondary data sources to determine the common characteristics of program nonparticipants, the Market Barriers Study used primary data sources to identify barriers to participation that potential program participants face.

Conclusion: The study found that nonparticipants expressed a number of beliefs that may be barriers to participation. Often expressed is a lack of trust in government and their landlords, and had a fear of scams. Often nonparticipants said that they had to prioritize their time and resources on needs that they thought to be more important. They also often needed more information about the programs and what they offered, as they believed that energy efficiency was not something applicable or helpful to them. Recommendations can be found in the study.

MA19X06-B-Residential Nonparticipant Customer Profile Study

Synopsis: The Residential Nonparticipant Customer Profile Study was conducted to better understand the characteristics of the sets of people who do not participate in the Mass Saves programs. In particular the study focused on renter non-participants, moderate income non-participants, and non-English speaking non-participants. Beyond these characteristics, the study looked at populations at the census block level, to establish whether there was a geographic component to non-participation. Additionally, this allowed the study to utilize any other data points the census had.

Conclusion: The study found that there was a negative relationship between rentership, moderate income households, and limited English speaking households and program participation. While this was expected from known barriers with these population subsets, the study does now provide data to allow for more targeted efforts. The study does indicate that efforts in 2013-2017 seem to have been successful in reaching low-income and multi-family households. There was no appreciable difference between electric and gas programs.

MA19DR02-Residential Energy Storage Demand Response Demonstration Evaluation

Synopsis: This report presents the results of a study that evaluated two residential storage demonstrations. The first demonstration, conducted by National Grid, was a “Bring Your Own Battery” style demonstration. National Grid called 27 events between July 3, 2019 and August 22, 2019. 50 devices participated in at least one of these events. The second demonstration was conducted by Unitil and consisted of four installations paid for by Unitil. These four installations were called on for daily events from August 1, 2019 to October 31, 2019.

Conclusion: The National Grid demonstration was largely successful, having high participation of registered devices and satisfied customers. 94% of customers never opted out of an event, 97% are likely to continue with the program, and 97% would recommend it to others. Over the course of the demonstration, the average demand savings per event was 139kW. Batteries that participated in 2-hour events saved an average of 5.5 kW/unit. The Unitil demonstration had longer discharge periods (4 hours vs 2 hours) and smaller per unit demand reduction. The four batteries averaged 5.1 kW reduction per event, with a minimum of 0.5 kW and a maximum of 6.8 kW. The demonstration also tested Solar PV + Battery balancing, which had a wide variability of success rate depending on duration. 15 minute durations had a 99% success rate, 1 hour durations had a 94% success rate, but when looking at a 4 hour duration only 14% of event days were successfully balanced.

MA19R05 Residential New Construction Passive House Assessment

Synopsis: This report presents the results of a detailed modeling comparison for multifamily high-rise, multifamily low-rise, and single-family projects build with Passive House design principle. In additions, this report presents the result of a series of in-depth interviews with various market actors to better understand the opportunities, barriers, incremental costs, and incentive issues associated with a new Passive House offering.

Conclusion: Total savings modeled varied between the three categories. For each of the models, savings are presented in overall site energy use intensity (EUI) and is compared against the baseline modeled EUI. For the multifamily high-rise models, savings varied between 11.5 and 12.7 units of EUI. For the multifamily low-rise models, savings varied between 12.0 and 12.7 units of EUI. Single-family projects are projected to experience the most savings, with savings between 15.0 and 21.8 units of EUI. For each of the categories, a large portion of the savings provided come from the heating and hot water end-uses. Starting on page 10 is an extensive list of recommendations and considerations stemming from this study.

2019

MA19R06-E Lighting Sales Data Analysis

Synopsis: This report summarizes results from a light bulb market share study conducted in Massachusetts. The study compared market share and bulb prices in Massachusetts, the United States, and various comparison areas with different levels of lighting program activity. The report explores 2018 market share by bulb type, shape, and ENERGY STAR® status, compares bulb prices, and considers trends in market share from 2009 to 2018. Although the sales data suggest that ENERGY STAR Lighting Programs in Massachusetts and across the nation still have a positive effect, the impact may be dwindling as market transformation of the LED market progresses across the nation.

Conclusion: Massachusetts’s market share for efficient bulbs (LEDs + CFLs) continue to be high, standing at 57% in 2018, with LEDs alone accounting for 53% of all bulb sales. LED reflector sales were high in all areas that were examined, suggesting that the market is nearly transformed. A-line sales were near of above 50% in all areas, pointing towards progress on market transformation. Globe market share was above 50% in Massachusetts, but less than 30% in non-program areas. Candelabra market shares were low in all areas, showing weak progress towards market transformation.

2018 Summer Thermostat Optimization Evaluation Report

Synopsis: The goal of the thermostat optimization evaluation was to confirm the technical feasibility of using thermostat setpoint adjustments to reduce household energy consumption and peak demand. The evaluation included both an exploratory analysis of thermostat setpoints and an impact analysis of energy and demand savings. The evaluation also yielded realization rates, providing the PAs with a point of comparison between the evaluated savings and the vendor-generated savings.

Conclusion: The program was largely successful, with only minor variation in results between program administrators. National Grid has 15,186 participating thermostats, with a savings of 30.6 kWh and 60W per thermostat. Wen including Cape Light Compact, the number of participants increases to 16,667, while the savings fall to 30.5 kWh and 57 W per thermostat. It is recommended that this program be carried forward by additional PAs in future years and that evaluation of the program continue.

TXC65_Analysis of Residential HVAC Sales Data from HARDI Distributors Memo

Synopsis: The memo summarizes findings from a study conducted on purchased residential HVAC sales data (HARDI data) that spans 2013 to 2019. The dataset contains information on central air conditioners, air source heat pumps, gas and oil furnaces, and gas and oil boilers. The goal of the study was to assess Massachusetts market share and compare against estimates collected from distributors via the Residential HVCAC Efficient Market Share Estimates study (TXC52). Additionally, the study sought to determine whether the HARDI data could be used to quantify market effects and see if there was any need to purchase additional data.

Conclusion: The study found the HARDI data to be useful, with a high potential for quantifying market effects. If the HARDI data were chosen to be used for quantifying market effects, it would be best to first explore the viability of modeling, and, if modeling proves to be a viable approach, undertake both direct comparison and modeling to triangulate he market effects. The full suite of conclusions and key findings can be found in the report on pages 3 through 6.

MA19R07-E Residential Lighting Turn-in/Exchange Study Report

Synopsis: The Massachusetts Program Administrators (PAs) are considering offering a Residential Lighting Turn-In/Exchange Program, which would be designed to facilitate customer exchange of inefficient bulbs or fixtures for new light emitting diode (LEDs) bulbs. The program objective would be to accelerate adoption of LEDs by reducing the large stockpiles of stored inefficient bulbs or to encourage customers to change out inefficient bulbs before failure. The program design means that it should be considered an early replacement program. This report covers the considerations for impact factors related to the program and established the deemed savings values for the program.

Conclusion: The final Net-To-Gross Ratio, Lifetime In-Service Rate, Adjusted Measure Life, Delta Watt, and Hours Of Use values adopted by the PAs and EEAC are included in Table 1 in the report. These estimates assume that the PAs’ existing upstream program will end on or after December 31, 2021. Rationale is provided to explain how each of the values was determined, and how to potentially improve them and ensure their long-term accuracy.

MA19X04-B Analyses of Immediate CCSI Residential Classroom Training Surveys Memo

Synopsis: This memo summarizes findings from the immediate survey responses collected during the residential 2015 International Energy Conservation Code (IECC) trainings held in the spring of 2019. The trainings were held in person as part of the Code Compliance Support Initiative (CCSI). This memo provides early feedback to PAs, the EEAC, and implementers on how specific aspects of the trainings are being received. The findings also feed into the savings attribution process for Residential New Construction programs.

Conclusion: Overall, respondents found the training sessions to be useful. In total 144 attendees filled out the surveys, 61 of whom were building code officials, and 83 were building professionals. On a scale of 1 to 6, the mean rating for the slides and handouts of the trainings was 5.4 and the mean rating for the presenter’s skills was 5.6. 99% of responders said they would recommend the training. Most topics covered were found to be very useful by most attendees. Where comparisons for training segments were deemed appropriate, the 2019 ratings were always higher than the 2018 ratings. The main recommendation made by attendees is that they would like to see more contractors in the classroom.

Residential Customer Profile 2013-2017

Residential Customer Profile Stakeholder Summary

Synopsis: The 2013-2017 Residential Customer Profile Study (RCPS) analyzes and reports on the Massachusetts Program Administrators’ (PAs’) energy efficiency program tracking and residential customer consumption data to offer diverse views of participation, savings, and geographic dynamics within the PAs’ residential customer population. The overall goal of the RCPS is to assess the energy efficiency program tracking and billed usage data for residential gas and electric customers, to identify and summarize salient trends in participation and savings.

Conclusion: At a high level, the study found that during the years 2013 to 2016, over one third of the Program Administrator served addresses in Massachusetts received at least one measure from an opt-in energy efficiency program. Upstream programs, while unable to be traced to specific addresses, continued to be effective at securing savings. During each of the years, electric customers saved 2% to 4% of their consumption, while cast customers saved 0.5% to 1.5%. Additional findings can be found in the summary report and the full report.

MA19R05 Passive House Literature Review

Synopsis: This memo presents the results of a literature review designed to provide additional details on key questions identified by the Massachusetts Program Administrator’s (PA’s) and the Energy Efficiency Advisory Council (EEAC) Consultants for the new Passive House offering within the Residential Homes and Renovations Initiative. The primary focus of this memo is to provide a high-level summary of Passive House using currently available literature. The memo covers the following topics: The history of Passive House; Prevalence of Passive House in the market; Descriptions of the two current Passive House certification bodies in the U.S.; General details on requirements and certification process for both certification bodies; Review case studies and existing reports on the actual vs. modeled performance of passive buildings; Review incremental costs associated with building Passive House buildings.

Conclusion: The literature review finds that there is a fundamental disconnect between modeled energy demand and actual energy demand in American passive homes. They mainly attribute this to differences in the modeled behavior of occupants being based on European households, who use lighting and plug-based appliances differently. The stock of passive houses to study in the United States is rather low now, as it is still a developing industry here, with limited Passive House professionals to service the market. For a full summary of key findings, see the bottom of page 1 of the memo.

MA19R05 Residential New Construction Passive House Assessment – Interim Report

Synopsis: This is a preliminary report outlines the results of detailed modeling of a proposed Passive House offering within the Residential New Construction Program. In addition, this report presents the results of a series of in-depth interviews (IDIs) with various market actors to understand the opportunities, barriers, incremental costs, and incentive issues associated with a new Passive House offering. This preliminary report focuses on multifamily buildings four stories or greater in size as this will be the initial target market for the new program offering.

Conclusion: The findings of the report can be broken into four categories: Model comparison, incremental costs, incentive suggestions and structure, and preliminary potential savings. The three models used for analysis were generally aligned in their projected savings and were averaged together to establish an estimate of savings. Potential incentive structures were derived from 21 IDIs to understand what current market actors believed would work. They determined that incentives from 4,000 to 5,000 per unit payable to the owner/developer based on certification would work best. Specific cost estimates can be found in table 4 in the report, and preliminary potential savings can be found in table 6.

TXC50 TXC50 LIMF NEI Phase 1 Findings

Synopsis: This memo presents qualitative results from the first phase of a two-phase study that is being conducted as part of the Special and Crosscutting NEIs contract. The study team was tasked by the Program Administrators of the Commonwealth of Massachusetts to evaluate the health and safety-related non-energy impacts (NEI) attributable to improvements in the energy efficiency of income-eligible multifamily buildings served through the Mass Save® program.

Conclusion: In Phase 1, the study team developed preliminary NEI estimates for all 13 core NEIs that were previously monetized by the MA PA’s LISF study and/or the 2014 national evaluations of the U.S. DOE’s WAP. The study team also developed a new NEI for reductions in arthritis symptoms. Because of difficulties achieving the expected response rate and some demographic differences among the study groups, and thus generalizing to the population of interest, it is not appropriate to report NEI values based on Phase 1 data alone. For this reason, the NEIs will be quantified and reported as monetary values in Phase 2 only. Table 1 in the report contains the NEIs, their qualitative assessment, and the summary of their metrics.

RES 43 Census of Massachusetts Multifamily Buildings

Synopsis: The Census of Multifamily Buildings study focused on identifying the population of multifamily buildings in Massachusetts, as well as the subset of this population that are condominium buildings. Multifamily buildings, for the purposes of this study, are defined as having five residential units or more. The census also focused on characterizing interactions of these properties with Mass Save initiatives. This study had two main objectives: Identify multifamily properties in Massachusetts and characterize any past program interactions with this population; Analyze this compiled building data to identify opportunities for customized multifamily program delivery strategies to maximize energy savings.

Conclusion: The study came up with five potential strategies for how the Program Administrators could improve participation rates: The multifamily database and dashboard should be regularly updated and improved to show the most accurate and up-to-date data; More research should be done into the issue of multifamily properties consisting of multiple buildings to better identify whether they fully qualify for programs; Utilize the multifamily database to identify underserved areas that could hold potential for new participants; Identify fuel types and heating systems of multifamily homes across the state to better tailor incentives to them; Increased effort should be put into identifying low-income properties that are not being served by current programs.

2018 Residential Wi-Fi Thermostat Demand Response Evaluation

Synopsis: This report reviews the results of National Grid’s Residential Wi-Fi Thermostat Demand Response program, ConnectedSolutions, in which Massachusetts residential customers can contribute toward reducing the system peak demand. The ConnectedSolutions program was first offered to National Grid customers in 2016 and continued to grow in 2017 and 2018. The demonstration project tests controllable thermostats as a demand response (DR) technology and customer acceptance of the DR program offering.

Conclusion: In 2018, the program reached 5,228 customers and 7,087 thermostats. This study confirmed the technical feasibility of using thermostats to reduce household peak demands; however, it has not examined whether that control will be cost-effective for the electric system, program administrators, and/or customers. In 2018, there were 7 events days where the average high temperature was above 90°F compared to 0 event days in the 2017 DR season. Average demand savings per thermostat reached 0.71 kW by the end of the year.

Cost-Effectiveness of Electric Demand Response for Residential End-Uses

Synopsis: This report presents results from a study that estimated the benefits and costs of demand reduction for DR-enabled appliances. For each of the appliance, enabling device, and DR strategy combinations listed in Table 1 in the report, the study provides estimates of: 1) Potential savings per unit during the summer peak period;1 2) Current and forecasted market saturation data; 3) Approximate incremental costs for summer DR; 4) Benefit-cost ratio.

Conclusion: Table 2 in the report provides results for each appliance, enables device and demand reduction combination. The results are categorized into the phase 1 literature review, the phase 2 baseline study for all days, the phase 2 baseline study for the hottest days, and the overall estimated unit impacts.

RLPNC 18-10 Lighting Market Assessment Study

Synopsis: This report presents the results of the 2018-19 Massachusetts Residential Lighting Market Assessment Study conducted by NMR Group, Inc. The study was designed to update estimates of lighting saturation and other critical market indicators in Massachusetts. The data for this study came from nearly annual on-site lighting inventories of homes in Massachusetts and a comparison area (portions of New York). The most recent wave of visits was completed between October and December of 2018.

Conclusion: The study found that Massachusetts programs continue to have a strong impact on saturation of LEDs, as shown by comparisons to adoption, saturation and replacement rates in New York. LED saturation is 34% in Massachusetts, compared to 22% in New York. Massachusetts households are more likely than New York ones to replace existing bulbs with LEDs (60% vs 48%). Specialty LED lamps are significantly more present in Massachusetts than New York for both reflectors (49% vs. 27%) and other specialty bulbs (44% vs. 29%). Despite the strong evidence of program effects, this study also found preliminary indications that LED saturation growth has begun to level off. Bulbs being replaced by LEDs has held steady at 60%. The gap in ENERGY STAR LED saturation between New York and Massachusetts has held steady, after widening for the last three years. First-year in-service rate fell 14% from 79% to 65%. This potential slowdown is not entirely unexpected and may signal that Massachusetts is reaching an inflection point on the S-curve of market adoption. However, the study cautions that one year of data do not constitute a trend, and it would be premature to conclude that the market has reached maturity without further supporting data.

MA19R02-E Delta Watt Report

Synopsis: This evaluation was conducted on behalf of the Massachusetts Program Administrators (PAs) to update some of the inputs used to calculate LED delta watts in the RLPNC 17-6 Lighting Market Adoption Models (MAMs). The PAs and EEAC consultants agreed to update equivalent wattage and sales weights based on 2018 program tracking data, which ties the delta watts directly to program sales. However, they also agreed to retain the estimated future market share values from the 2017 MAMs (RLPNC 17-6). The decision to keep the same future market share estimates reflected the continued market uncertainty related to federal standards, which existed when the estimates were developed in 2017 and remain in place now. Further, based on developments over the past year, the market is generally moving in the direction that MAM participants assumed when they made their predictions in 2017. Therefore, the PAs and EEAC consultants agreed that little value would be gained in updating market share for 2018.

Conclusion: Generally, in 2018, consumers purchased a higher proportion of high equivalent wattage LEDs in the GSL and Specialty categories, and lower equivalent wattage LEDs in the Reflector category through the program. Updating the wattage by bulb type and sales share resulted in increased delta watts for GSL and Specialty and decreased delta watts for Reflectors, as can be seen in Table 1 in the report. The authors offer two pieces of consideration for future studies, which can also be found in the report.

TXC60_Solar Knob Tube Asbestos Mitigation NEIs Memo

Synopsis: This memo presents the results of the Solar and Home Energy Services Safety Remediation Non-Energy Impacts Study (TXC60) that was conducted from September 2018 through January 2019. Using a literature review and other secondary research, the study identified non-energy impacts (NEIs) associated with the installation of solar photovoltaic (PV) panels, energy-storage batteries, and the remediation of knob and tube (K&T) wiring and asbestos.

Conclusion: The non-energy impacts the study suggests being included for energy-storage batteries are $0.14 per household for reliability of medical equipment and $1.72 per household for avoided deaths. The non-energy impacts associated with solar PV panels with battery storage should include low-income rate discounts using existing values, equipment maintenance at -$308.30 per household, home durability with no value given, and thermal comfort with no value given. The non-energy impacts associated with knob and tube remediation should include avoided fire, avoided electrical shock, avoided trip and fall all with algorithm suggested values, and reduced rodent infestation with no value given. The non-energy impacts associated with asbestos remediation should include avoided medical costs, although no value was given.

RES 1 – Residential Baseline Study – Comprehensive Report

RES 1 – Residential Baseline Study – Executive Summary

Synopsis: The primary goal of the Massachusetts Residential Baseline Study is to collect saturation, penetration, and usage behavior data for all major electric and gas appliances, mechanical equipment, and electronics in Massachusetts homes. This data supports energy and peak demand savings calculations for program evaluation and design and provides additional insight on the savings potential in the existing residential buildings market.

Conclusion: The study found that cooling and heating make up an important portion of the summer and winter load shapes. Saturation of efficient heating and cooling technologies are increasing but have room to grow. While lighting comprises a sizable portion of the electric load, the energy intensity of lighting has largely fallen due to program interventions and federal standings. Water heating is noted as having potential for further peak demand reductions.

UPDATED 2013-2015 Residential Customer Profile Report

Conclusion: The study had a wide array of findings, all of which can be found in the report. One of the key findings was that both across and within program administration jurisdictions there are geographic differences in program participation and savings. At a geographic block level, savings ratios were consistent across poverty index groups, although participation rates and contribution ratios were not as consistent. Saving ratios were also consistent across renter quintiles, although participation rates may have been skewed by some multifamily buildings falling under C&I programs. Geographic distributions of participants show more clustering than one would expect, indicating the potential for system barriers in these areas. While the electric residential products program has been decreasing in overall share of savings, it is still the largest contributor because of the upstream lighting offering.

MA19R01-E Appliance Recycling Report

Synopsis: This report summarizes the results of a 2018 program ran under the Mass Save® umbrella that targeted refrigerators and freezers for recycling by offering incentives. The incentives paid were either $50, or $100 during special promotion periods. In 2018, the program provided 18,405 rebates: 15,717 for recycled refrigerators and 2,688 for recycled freezers.

Conclusion: The main finding of the study was an update in per-unit saving estimations for both refrigerators and freezers. The per-unit savings for refrigerators was 1,027 kWh in gross savings, 904kWh in adjusted gross savings, and 398 kWh in net energy savings. The savings for freezers was 769 kWh in gross savings, 523 kWh in adjusted gross savings, and 295 kWh in net energy savings. The report provides an equation for the calculation of unit savings below Table 5.

RLPNC 18-8 Residential Lighting Market Scan Report

Synopsis: This report presents information published in Q4 2017 through early 2019 on the U.S. residential lighting market. It also presents information from five previous quarterly memos for reference. This report is organized by topic, then by chronology, with the newest information presented first, framed in green, followed by earlier information from previous memos. All sources are from documents published in 2015 or later and, with a few noted exceptions, based on research conducted in 2014 or later. The market scans focus on information gained from areas outside of Massachusetts.

Conclusion: The report finds that market share of LEDs has continued to grow across the country. A study of the lighting market of the Pacific Northwest found LEDs made up 47% of all general-purpose bulbs in 2017-2018. The National Electrical Manufacturers Association (NEMA) found LEDs to be 65% of shipments in the A-lamp market in Q3 2018. However, some market sectors are still dominated by incandescents, with the Pacific Northwest finding that they made up 74% of decorative and candelabra-based lamp bulbs. Despite these changes in LED market penetration, Net to Gross ratios have not changed in states that have had new studies. New studies regarding the hard to reach markets indicate that LEDs were significantly harder to obtain and more expensive in regions with higher poverty rates. Finally, the report discusses the potential changes that EISA 2020 could create, and what the impact of the DOE-approved rollback might do to the market.

RLPNC 17-3: Advanced Power Strip Metering Study Report

Synopsis: This report presents the results of the RLPNC 17-3 Advanced Power Strip (APS) Metering Study conducted by NMR Group, Inc. and Navigant Consulting on behalf of the Massachusetts Program Administrators (PAs). The study was designed to update impact factors associated with APS units, namely, baseline energy use and Energy Reduction Potential (ERP). It is important to note that this study was designed as a metering study and not as an impact evaluation. Since customers were not asked to set up or install the APS units on their own, it is possible that saving results are upwardly biased. To capture the full scope of savings, this report also separately considers the set-up, in-service (ISR), short-term retention (RET), and persistence rates of APS units.

Conclusion: This study found that the implementation of Advanced Power Strips (APS) decreased the average energy consumption of both home entertainment centers (HEC) and personal computers (PC). At a baseline, they found that both HECs and PCs used 603 kWh annually on average. When outfitted with a tier 1 APS, the average HEC consumed 471 kWh annually and the average PC consumed 399 kWh annually. These findings can be found in greater detail in Table 1 in the report. By surveying existing APS units in customer houses, the study was able to determine a realization rate of 92% based on how many of the units were properly set up. The report contains some additional figures that they were able to measure that do not directly impact the study. Considerations can also be found on page 8.

CLC 2018 Smart AC Savings Program Evaluation Report

Synopsis: This report outlines and evaluates the Cape Light Compact’s (CLC) Smart A/C savings program for the year 2018. In 2018, a total of 91 devices were enrolled in this demand response (DR) program, which was up significantly from the original enrollment of 22 devices in 2016. The program runs from June 1st to September 30th on non-holiday weekdays between the hours of 10 a.m. and 9 p.m. During these times, CLC could call events in which participating A/C units set temperatures would be shifted between 2°F and 4°F to reduce electrical demand during times of high use. Customers could either opt out or choose to manually control their thermostat if they desired.

Conclusion: The program was evaluated by assessing participation by thermostat manufacturer, surveying customers, and estimated the demand reduction impact of the 2018 program. Device participation varied heavily by manufacturer. Figure 1 in the report shows the average participation by device type. Notably Centralite devices fully participated 83% of the time. Table 3 in the report outlines the findings in customer experience. The customers were satisfied by the program, with 86% responding positively when asked if they would participate in the future. The demand saving reductions for participating devices were estimated to be 0.19 kW for mini-split heat pumps and 0.62 kW for central AC systems. The report includes a list of 7 considerations for the study that can be found in Table 4.

RES 38 2017 Income Eligible Process Evaluation Findings

Synopsis: This report is both a process evaluation and recommendations for the Income Eligible Program. The Income Eligible Program serves single-family households that are at or under 60% of the state’s median income, and multifamily buildings where at least 50% of household are living at or under 60% of the area’s median income. The data for the process evaluation was gathered from 4 primary sources: 1) in-depth interviews with Program Administration (PA) and Low-Income Energy Affordability Network (LEAN) staff; 2) in-depth interviews with agencies including lead agency and sub-agency staff; 3) interviews with multifamily property owners and managers; and 4) observational ridealongs with select implementing agency staff (e.g., installation specialists, auditors, quality control inspectors) combined with in-person customer interviews while onsite with customers.

Conclusion: The key findings of this study are organized by research topic. The topics are Program Administration, Current Marketing Strategies, Opportunities for Engagement, Consistency in Program Delivery, and Measures for Further Consideration. Summaries of their findings in these topics can be found in the report on pages vi – viii. Tables 2 and 3 in the report give an overview of the recommendations and considerations that the authors created based upon the results of the research.

TXC 53 Analyses of Immediate Code Compliance Support Initiative Residential Classroom Training Surveys Memo

Synopsis: This report provides analyses of the immediate survey responses collected through paper surveys, registration data, and poll questions from two Code Compliance Support Initiative (CCSI) residential trainings held in 2018. These are the first surveys to be collected since implementation of the CCSI passed from CLEAResult to Performance Systems Development (PSD). Both trainings focused on the residential 2015 IECC code (which became mandatory in Massachusetts at the beginning of 2017) and solar-ready provisions (which became mandatory at the beginning of 2018).

Conclusion: Respondents to the surveys found that the two 2018 training sessions were of high quality. The survey had 62 total respondents, comprised of 47 building code officials and 15 building professionals. The survey had a lower response rate (43%) than previous surveys (68%), potentially due to decreased training times (1.5 hours in 2018 vs 3 hours in 2016-2017). The average rating of slide and handout information quality was 5.3 out of 6. On average the presenters were given a 5.4 out of 6 for their skills and 5.5 out of 6 for their handlings of questions. Figure 2 in the report shows that the specific topics in the presentation were rated highly by participants, although the average rating of 5.0 out of 6 has declined from the 2016-2017 training sessions, which averaged 5.2 out of 6. Some participants expressed a desire for more practical examples and field applications.

2018

RLPNC_17-14_Mini-Split Heat Pump Incremental Cost Assessment

Synopsis: This report compares the initial costs (i.e., equipment and installation costs) and operating costs associated with heating, cooling, and water heating for two versions of a single-family home that meets the Massachusetts Stretch Energy Code target—an Energy Rating Index (ERI) of 55. The traditional version of the house has a gas furnace, electric Central Air Conditioning (CAC) system, and an instantaneous gas water heater, while the mini-split version of the house has a ducted mini-split heat pump and a heat pump water heater. The two versions of the house were assumed to be of identical structure and were modeled to achieve an ERI of 55 at minimal costs.

Conclusion: The traditional house with gas heat and electric Central Air Conditioning (CAC) costs less to build and operate than the mini-split house. Table 2 in the report presents the initial cost summary of the two model houses. The mini-split house was found to have an initial HVAC cost $1,586 higher than the traditional home, while the DHW was $832 lower than the traditional home. This results in an overall cost increase of $754 over the traditional home, or a 6% cost increase. Table 3 shows the operating cost summary for the two models. The mini-split house was $485 more expensive to heat, $8 cheaper to cool, and $19 more expensive for water heating. This resulted in an overall cost increase of $496, or a 33% increase over the traditional home. The study also modeled the homes heated by an electric furnace, which was found to be significantly more expensive than either the heat pump or the gas furnace.

RLPNC 18-11 Lighting Sales Data Analysis

Synopsis: This report describes recent screw-base light bulb market share, shipments, and prices in Massachusetts, in the entire United States, and in program and non-program states. The analyses draw on light bulb sales data compiled by the LightTracker Initiative of the Consortium for Residential Energy Efficiency Data (CREED), and shipment data reported by the National Electrical Manufacturers of America (NEMA). This study is meant to provide information only and refrains from making recommendations or offering considerations.

Conclusion: The sales data analysis strongly suggests that the Massachusetts Residential Lighting Core Initiative (the Program) continued to have a positive impact on the energy-efficient bulb market in 2017. Several factors culminated in LED market shares overall and for ENERGY STAR qualified bulbs in Massachusetts that were higher than in non-program states, the nation, and even most other program states. In fact, while the combined market share of energy-efficient bulbs (LEDs and CFLs) in the nation remained stable between 2015 and 2017, the efficient bulb market share in Massachusetts increased. The LED incentive per bulb in Massachusetts was the second highest in the nation, and the state’s LED market share was the sixth highest in the nation. The aggressive incentives in Massachusetts help explain why LED prices in the state, including both ENERGY STAR and non-ENERGY STAR models, fell below those of not only of non-program states and the nation, but also other program states as well.

RES34 Home Energy Services Impact Evaluation Report with Executive Summary

Synopsis: This Home Energy Services (HES) Impact Evaluation was designed to estimate the gross per-unit energy savings associated with the HES measures offered in 2015 and 2016. In total, the team evaluated 29 measures across four fuel types (natural gas, electric, heating oil, and propane); nine of which were not part of the previous evaluation (completed in 2012). The evaluation also yielded realization rates, for insulation and air sealing, that the PAs will use to adjust the ex ante gross savings produced by each HES Lead Vendor’s (LV) proprietary energy modeling software. The evaluation team relied on billing analysis, building simulation, and engineering algorithms to estimate savings.

Conclusion: The evaluation team’s billing analysis determined that HES participants who weatherized their natural gas heated homes (i.e., installed air sealing and/or insulation) saved, on average and statewide, 130 therms per year. This result is somewhat lower than the statewide findings from the previous evaluation in 2012. Evaluators recommend that the PAs use the realization rates found on page 3 of the summary to adjust their LV-estimated weatherization savings. Programmable and Wi-Fi thermostats are estimated to have save 3.6% and 6%, respectively, of HES participants’ annual heating consumption. Furnaces and boilers had an Increased heating load, relative to the previous evaluation and resulted in ex post savings exceeding ex ante savings for furnaces (while the opposite is true for boilers). The savings associated with refrigerator replacement is greater than the previous evaluation (1,001 kWh/year, compared to 661 kWh/year).

RES21 Energy Optimization Study

Energy Consumption, Demand, Customer Cost, and Carbon Emissions Analysis Spreadsheet

Synopsis: This memo summarizes the evaluation team’s efforts to characterize the savings and costs of a variety of fuel-switching measures included in the draft implementation plan for the 2019 – 2021 program cycle. The evaluation team created and revised a spreadsheet model, submitted with this memo, that calculates the consumption and savings of different measures for three scenarios: (1) full/early replacement, (2) partial displacement, and (3) replace on failure. The evaluation team used numerous publicly-available data sources throughout the analyses conducted for this study as listed in the memo.

Conclusion: The consumption and savings estimates are presented in detail in the spreadsheet model submitted with this memo. The results of the analysis are as follows: • All of the measures considered in this study result in annual energy cost savings for the customer. • The early replacement measures with the highest annual energy cost savings were the measures involving the replacement of propane and oil boilers with gas-fired combination boilers. • All of the measures considered in this study result in reduced CO2 emissions. The CO2 emissions reductions are comparable for the early replacement scenario (with measure savings ranging from 0.32 to 6.40 tons CO2/year) and the replace-on-failure scenario (ranging from 0.22 to 6.96 tons CO2/year).

RES22 Low Rise Measure Review

Synopsis: The purpose of this workbook is to document the evaluation team’s recommendations for gross and net savings assumptions for Low Rise measures. The tables in the workbook summarize the team’s rationale for gross savings review, and recommendations for future evaluation and analysis topics. The workbook includes the following tabs: Electric Recommendations – Summarizes recommended savings for electric measures; Gas Recommendations – Summarizes recommended savings for gas measures; Participation Analysis – Includes past participation data used to estimate gross savings in some cases; EEYr1_Electric, EEYr1_Gas, Electric Lookups, Gas Lookups – Includes current savings assumptions data for electric and gas measures; MA RES 34 Summary Results – Includes summary of RES 34 evaluation results, used to estimate gross savings in some cases.

Conclusion: As documented on the Electric and Gas Recommendations tabs, the team applied the following rationale in recommending gross savings values. Water heating for low-rise buildings are typically be the same as single-family systems. 89% of low-rise units have in-unit water heaters (similar to single family). HVAC systems for low-rise buildings are typically be the same as single-family systems. 93% of low-rise units had HVAC systems similar to similar to single family. Envelope measures for low-rise buildings typically have to meet the same building code standards as high-rise buildings and have similar construction components. 2011 RBSA Multifamily Characteristics and Energy Use Study compared actual energy use of low-rise, mid-rise, and high-rise multifamily buildings, and found low-rise unit to use an average of 1.54 times the energy use of a high rise unit for electrically heated units, and 1.65 times the energy use of gas-heated units. With 34.2% of MA multifamily buildings having gas accounts, the average weighting applied to these low-rise measures is 1.6. In addition to recommending values for Low-Rise savings assumptions, the team recommends consideration of several opportunities for future research including: gas measures; measures that don’t currently have a low-rise measure, but do have single-family or high-rise measures to compare to; showerhead and aerator savings; general measure category evaluation; and NTG evaluation.

CLC NGRID Education Kits Evaluation Report

Synopsis: This report presents the results of an evaluation of the Cape Light Compact and National Grid Education Kit programs. The programs provide teachers with lesson plans, in-class activities, and training on energy efficiency topics to incorporate into their science curricula. The primary source of energy savings from the program are take-home kits with energy savings measures provided to students. To estimate energy savings and identify opportunities for improvement, the evaluation team sought to address these objectives by reviewing survey data and deemed savings assumptions, interviewing program and implementation staff, conducting literature review of similar programs delivered in other parts of the country, and conducting in-depth interviews with participating teachers.

Conclusion: Based on interviews with teachers and program implementers, evaluators found that three key barriers negatively impact the In-Service Rate (ISR): 1) lack of parent awareness of kit distribution; 2) physical barriers to measure installation, such as rental status of residents; and 3) language barriers. Evaluators identified two key barriers to increasing survey responses: 1) kits being ordered too close to the end of the year to collect and submit surveys; 2) teachers are lacking time to submit survey results (specific to NGRID); and 3) language barriers. The study findings also indicate opportunity for meaningful spillover savings from the current program interventions and kit configurations. Removing faucet aerators emerged through our research as an opportunity to improve the existing kits content. Possible additions to the measures offered in the kits include: advanced smart strips, door sweeps, shower timers, toilet leak detector tablets, and weather stripping, though many of these measures have limitations for implementation. Detailed conclusions, recommendations, and considerations related to these findings as well as to program delivery are included in the study.

RES 36 Heating and Cooling Early Retirement Net To Gross Report

Synopsis: This report provides net-to-gross (NTG) estimates for the Early Retirement HVAC program and additional context and process findings from research with participants, HVAC contractors, and energy specialists who provide Home Energy Assessments. The evaluation team used participant survey results to develop free ridership (FR) estimates, NTG, and assessments of equipment condition and age. The contractor and energy specialist surveys provided context and process findings, but do not directly impact the net savings results. As agreed with the PAs and EEAC in planning stages, NTG results encompass free ridership only; the NTG ratio does not include spillover due to the nature of the program design which likely limits spillover

Conclusion: Net-to-gross results are presented by measure and fuel type. Overall, NTG for the Early Retirement HVAC program is high. Participants reported that both HVAC contractors and energy specialists who provide Home Energy Assessments are influential in their decision to participate in the program. A key motivation for customers to participate in the program is a desire to eliminate the risk of equipment failing at an inopportune time. The various components of the Early Retirement HVAC program, including Mass Save marketing and HEAT loan offerings, all appear to be working well together to influence program participation. Contractors report that some customers replace HVAC systems before failure outside of the program, most frequently because customers do not want to wait for a Home Energy Assessment. Based on customer self-report of equipment age and condition, the evaluation team estimates that 37% of equipment rebated through the Early Retirement program should actually be categorized as replace-on-failure.

TXC 56 Memo on Initial Considerations for Attribution/Net-to-Gross Estimation for Energy Optimization

Synopsis: This report identifies initial considerations for assessing attribution and estimating net-to-gross ratios (NTGR) for energy optimization. This research is designed to help the PAs and EEAC prepare for a possible policy change that would allow PAs to claim savings from energy optimization. Energy optimization refers to a fuel-neutral approach to energy efficiency that will allow the PAs to pursue net energy reductions. Evaluators identified evaluations and other information for programs outside Massachusetts that offer support for energy optimization, with a focus on documenting reported attribution and NTGRs associated with this support. In addition to reviewing reports and documents from these peer programs, evaluators reviewed other literature that addressed issues related to the design, implementation, and evaluation of energy optimization-related programs.

Conclusion: Based on the literature review, evaluators were unable to find NTGRs measured for conversions from delivered fuels. Given this, evaluators developed an approach for assessing attribution and estimating net-to-gross ratios for the PAs to consider. The overall approach involves measuring six separate types of gross savings and applying a specific NTGR to each type. Evaluators group conversions into two scenarios: (1) Fuel Oil/Propane Furnace/Boiler to Gas Furnace/Boiler and (2) Fuel Oil/Propane Furnace/Boiler to Electric Heat Pump. Evaluators propose that for 2019-2021, the PAs consider using negotiated NTGRs for conversion and service change. For high-efficiency equipment impacts, evaluators propose the the PAs consider the following: When conversions are to gas heating equipment, the early retirement prospective NTG ratios from the Early Retirement of Heating/Cooling Study (RES36). When conversions are to electric heat pump equipment, a negotiated NTGR would be used. For 2022-2024, evaluators suggest that the PAs consider undertaking a new study late in the 2019-2021 period to estimate retrospective NTGRs for early retirement, and develop prospective 2022-2024 NTGRs informed by the retrospective NTGRs.

Memo on Wi-Fi Thermostat Impact Evaluation-Secondary Research Study

Synopsis: The purpose of this study is to provide a comprehensive secondary literature review, utilizing existing research performed for other jurisdictions, to summarize savings findings around the country, by year, thermostat type, fuel type and other key considerations, and to provide recommended savings values for interim use based on these secondary findings. This memo includes a comprehensive annotated bibliography, in Excel workbook form, facilitating searches and summarization of findings by different categorizations (similar climate zones, most recent studies, etc.). The bibliography includes a section on recent Seasonal Savings studies for reference.

Conclusion: Evaluators recommend annual savings values of 160.9 kWh and 31.1 Therms for use until primary research and analysis findings are complete. These values should be applied prospectively for 2019 and future planning. Secondary research does not provide sufficient detail or granularity to subdivide savings by AC system type, fuel mix, household type or other considerations, but the PAs may decide to adjust the interim savings values proportionally to the assumptions for these differences reflected in current savings values in Massachusetts.

BEH 2 Weather in Air Conditioning Use Behavior and Demand Response Program Participation Report

Synopsis: The purpose of this study is to understand the effects of weather on air conditioner (AC) use behavior and demand response (DR) program participation. Understanding how weather affects customer AC usage will allow the PAs to improve the design, implementation, and customer messaging around these programs. Similarly, understanding the role of weather on a participant’s likelihood to opt out of DR events will allow the PAs to balance opt-out rates in calling DR events. This study was undertaken in a staged approach and combined several unique datasets, including advanced metering infrastructure (AMI) data and thermostat telemetry data from National Grid’s Smart Energy Solutions (SES) pilot in Worcester, as well as thermostat telemetry data from National Grid’s Residential Wi-Fi Thermostat DR demonstration project.

Conclusion: In line with previous research, the study quantifies the strong effect of heat on first AC use, use throughout the season, and people’s likelihood of opting out of DR events. The team’s results demonstrate that humidity is not a key factor in determining first AC use, but that it does play a strong role in determining AC use throughout a season. Moreover, humidity also plays an important role in causing people to opt out of DR events. Categorizing thermostat users into AC user types is less valuable than focusing on quantifiable observed behaviors people exhibit when using their thermostats, and focusing targeted messaging or other behavioral interventions on users with specific thermostat use traits rather than user type categories is likely to be more effective. By combining thermostat and AMI data, this study also estimates a model to translate electricity usage into AC runtime, which may prove a useful starting point in future studies related to AC use where thermostat data is not available. This study also shows that program configuration considerations such as duration and timing of events, thermostat type and vendor, and proximity of events may have as large an effect on DR event opt-out behavior as weather variables do. This suggests the PAs carefully consider program characteristics in addition to weather variables when planning DR events to optimize savings.

RES 28 Ductless Mini-Split Heat Pump Cost Study

Synopsis: The goal of this study was to evaluate the energy-efficiency related total and incremental costs of single family home installations of DMSHP systems currently rebated through the Residential Heating and Cooling program. This study relies on three main data sources: 1) a survey of HVAC contractors in Massachusetts, 2) retail prices gathered by webscraping, and 3) a sample of scanned invoices for system installations that were rebated through the program. This study combines data from these three sources to construct cost efficiency curves that describe the total installed cost of DMSHP systems across a range of different system sizes and efficiency levels.

Conclusion: One main finding of this study is that for a given capacity level, the cost changes associated with increasing efficiency are due entirely to changes in the cost of equipment (i.e., the installation costs do not vary with efficiency). Table 3 in the study illustrates how the incremental installed costs for regular and cold-climate systems compare to the current rebate amounts for systems with different sizes and number of zones. At smaller system capacities of 24 kBtu/h and less, the incremental cost of high-efficiency systems exceeds the rebate amounts offered for those systems. However, at larger system capacities of 30 and 36 kBtu/h, the rebate amount exceeds the incremental cost of efficiency for regular (non-cold-climate) systems. There are three strategies the Program Administrators (PAs) could pursue to improve the cost effectiveness of the DMSHP rebate offerings: Adding a premium rebate level at 28 SEER and 14 HSPF; Limiting the rebate eligibility for DMSHP systems to systems that are cold-climate qualified; Add-on task to evaluate how equipment and installation costs of ducted systems compare to the non-ducted systems examined in the current study. Section 4 of the report describes these strategies in more detail.

RLPNC 18-1 Appliance Recycling Report

Synopsis: This study estimates gross, adjusted gross, and net energy savings (and a net-to-gross ratio) based on the characteristics and alternative outcomes for refrigerators and freezers currently recycled through the program. This report summarizes that research and addresses participants’ experiences with the program. Evaluators used the approach advocated in the Uniform Methods Project (UMP) to guide the estimation of gross and net energy savings. The UMP developed algorithms for estimating the UEC – equivalent to gross energy savings – of refrigerators and freezers by developing regression models from data collected in prior metering appliance studies.

Conclusion: Table 3 in the report presents the a gross energy savings estimates for 2017, developed through application of the UMP algorithm. Section 2 lists the values that served as inputs to the algorithm and compares them to the inputs and adjusted savings estimates reported in a 2011 study of the program. Based on the characteristics of units recycled in 2017, the UMP regression algorithm suggests that the current gross energy savings is 1,019 kWh for refrigerators and 718 kWh for freezers. On average, survey respondents reported that they had their refrigerators plugged in 88% of the year prior to recycling and their freezer for 68% of the year; these percentages serve as the realization rates. Their application yields an adjusted gross energy savings of 897 kWh for refrigerators and 488 kWh for freezers.

RES 23 Task2 Memo on Cost Study of Heat Pump Installations for Dual Fuel Operation Quick Hit Study

Synopsis: The purpose of this study was to provide the PAs with an understanding of the total installed costs and cost categories (i.e., equipment, labor, supplies, and other costs) associated with early replacements and new installations of residential central A/C and heat pump (HP) products. The 2015 Cool Smart Incremental Cost Study evaluated the equipment costs of residential A/C and HP products. However, the PAs had not recently evaluated the total installation cost of installing these products in the PAs’ service areas. This memo summarizes the evaluation team’s findings from a review of customer invoices that were submitted with rebate applications for residential A/C and HP installations. The cost results from this study will be used by the Energy Optimization program to evaluate measures that would incentivize the installation of heat pumps to displace heating equipment fueled by oil or propane.

Conclusion: As shown in Table 2 of the memo, all three product classes in this analysis show a higher average cost/ton for new installations compared to replacement installations. Four reasons stand out for why new installations may cost more than replacement installations: (1) New installations require installing refrigerant lines between the condenser unit and the indoor unit, while replacement installations may sometimes make use of the already-installed lines. (2) New installations tend to require the installation of a complete air handler unit, while replacements may only require an indoor coil replacement. Air handler units are typically more expensive to install than coil replacements. (3) New installations may require ductwork to be installed throughout the house if they cannot be integrated with an existing furnace. (4) New installations are more likely to require electrical system upgrades, while replacement installations may already have adequate electrical supply available. Other findings suggest that customers should avoid oversizing HPs for heating because that would lead to grossly oversizing the cooling capacity, particularly in a partial displacement scenario where the heat pump need not provide the full heating load. Also, compared to the 2015 CoolSmart Incremental Cost Study, the invoice review conducted in this study predicts smaller incremental costs between heat pump installations at 16 SEER and 18 SEER.

RLPNC 17-4-5 Products Impact Evaluation in In-Service and Short-Term Retention Rates Study Report

Synopsis: The purpose of this study is to evaluate key Massachusetts-specific impact factors for general products (RLPNC 17-5 General Products Net-to-Gross) and advanced power strips (APS) (RLPNC 17-4 Advanced Power Strip Literature Review and Consumer Survey). The products addressed in this joint study were distributed by the PAs through either the Residential Consumer Products Core Initiative or the Mass Save® Home Energy Assessment (HEA) Program. This document refers to these programs as the products program and the direct install program, respectively, but we stress that the HEA energy technicians left Tier 1 APS behind during audits and did not install them directly. Thus, the program is the direct install program, but the Tier 1 APS are left behind for customers to install on their own. The results are based on web-based surveys of two different populations of PA program participants: (1) those who purchased products via the Mass Save® Website or using a mail-in (or online-submitted) rebate between January 2016 and June 2017, and (2) those who received Tier 1 APS through a direct install program from January to October 2017

Conclusion: The report presents key findings, conclusions, recommendations, and considerations. Key findings are as follows. Combined ISRs and short-term retention rates range from 75% to 97%, with smaller, easily removed products (APS, showerheads) having lower rates and appliances having higher rates. Table 1 in the study presents the ISR and short-term retention rates, separately and combined, that were directly measured in this impact evaluation. Overall, 82% of respondents who purchased products through the program voice satisfaction with product performance and similar percentages say they would recommend the products. Respondents who removed APS (both tiers and program delivery modes) and showerheads from service voice skepticism about whether they will reinstall the products. Evaluators recommend that the PAs should use the combined Massachusetts ISR/short-term retention rates listed in Table 1 for the 2017 Annual Report, the 2018 Annual Report, updates to the TRM, and program planning for 2019 to 2021 for all evaluated products.

RLPNC 18-4 Products Net-to-Gross Report

Synopsis: The purpose of this study is to set 2019-2021 prospective net-to-gross ratios (NTGRs) for products offered through various residential programs. Evaluators lead a consensus process approach, asking a panel of experts to review and assess NTGRs. Inputs in these prospective discussions were self-reported NTGRs derived from primary research conducted in Massachusetts for a variety of products supported by the PAs. Evaluators included NTGR questions in the web-surveys conducted as part of the RLPNC 17-4 and RLPNC 17-5 studies. After reviewing materials, the consensus panel members developed their own predictions of NTGRs for 2019 to 2021.

Conclusion: Table 1 in the study presents the recommended prospective NTGRs for the products researched by evaluators for RLPNC 18-4. Products include: freezers, room air cleaners, clothes dryers, dehumidifies, room air conditioners, pool pumps, advanced power strips, temperature sensing shower heads, refrigerators, clothes washers, and dishwashers.

RES 37 HEAT Loan Assessment

Synopsis: This study determined the percentage of HEAT loan authorized customers (i.e., those who received a home energy assessment and authorization from their lead vendor to seek a HEAT loan) that completed the HEAT loan process and secured a loan. The evaluation team also explored the barriers that prevented some customers from completing the process, and whether those customers install recommended measures regardless.

Conclusion: Of the 80,000 customers that had a Home Energy Assessment in 2017, approximately 6,600 (8%) worked with their lead vendor (LV) to become authorized to seek a HEAT loan. Most of these customers (approximately 75%) completed the process and received a HEAT loan. There are four main reasons why authorized customers did not subsequently take out a HEAT loan: Declined by a lender, cost too high, other financial priorities, timing was not right. Just over half of authorized customers still installed at least one of the recommended energy efficiency measures despite not utilizing the HEAT loan, using other funding sources. However, a number of these customers reported that they installed less efficient equipment and/or fewer measures than they would have with the HEAT loan. Heating systems are by far the most common measure installed by all customers in the study and HVAC contractors present an important opportunity for driving participation. Most customers indicated that the LV authorization process was smooth and clear. However, almost a quarter of customers reported needing to apply multiple times or needing to resubmit missing or incorrect documentation.

TXC 34 Residential HVAC Net-to-Gross and Market Effects Study

Synopsis: The primary purpose of this study was to estimate and recommend net-to-gross ratios (NTGRs) for selected heating, cooling, and water heating measures that will receive Mass Save® Standard rebates in 2019-2021. Another purpose was to measure market effects indicators for evidence of progress toward market transformation that may be attributed to the program, and to set baselines for comparison with future measurements. Evaluators fielded participant and contractor quantitative web and phone surveys in 2017 and 2018. A Consensus Group used those results to recommend NTGRs for Massachusetts planning tools, such as the 2019-2021 Technical Reference Manual and benefit-cost ratio models. Surveys also asked about market effects and other topics.

Conclusion: This study’s recommended NTGRs differ from current TRM values, which are mostly based on a 2012 study. The recommended 2019-2021 NTGRs for ductless mini-split heat pumps (DMSHPs) and boilers increased from the 2016-2018 NTGRs. The recommended NTGRs for heat pump water heaters (HPWHs), central air conditioning (CAC), central heat pumps (CHP), and furnaces decreased. Contractors offered anecdotal evidence that, to some degree, the program’s intended effects on stocking have occurred. Attribution question results for equipment costs provide anecdotal evidence that the programs played a modest role in effecting changes in the high-efficiency versus standard efficiency equipment prices that contractors incur. Additionally, customer demand for high-efficiency HVAC equipment has increased, and contractors attributed much of this change to the programs. Contractors perceived that their own recommendations, compared to other factors, have had the most influence on their non-program high-efficiency sales; moreover, the programs strongly influence their recommendations. This suggests that the programs have substantial indirect influence on what appears to be the strongest driver in the selection of qualifying equipment outside the program.

TXC 48 Residential New Construction Attribution Assessment

Synopsis: This study forecasts net-to-gross (NTG) ratios for the low-rise Residential New Construction (RNC) program and the residential portion of the Code Compliance Support Initiative (CCSI) for the 2019-2021 program period. The RNC and the CCSI programs both address the single-family RNC market and influence similar market actors. For this study, the team considered the effects of these programs together to avoid two potential issues: 1) Leaving savings attributable to the PAs undocumented. 2) Double counting savings from these programs. This study used a Delphi panel approach to estimate the effect of the programs on single-family new construction building practices. The methodology for calculating net savings and developing NTG ratios is consistent with the 2014 Residential New Construction Net Impacts Study.

Conclusion: Based on the findings from this study, evaluators offer the following conclusions, recommendations, and considerations. The NTG ratios for the low-rise RNC program and the residential portion of the CCSI range from 0.96 in 2019 to 1.10 in 2021. The baseline conditions used to calculate gross savings strongly affect the calculated NTG ratio. After consulting with the PAs and EEAC, the evaluation team recommends that the PAs use the prospective 2019 NTG ratio of 0.96 for the duration of the 2019-2021 program period. The PAs should apply the prospective 2019 NTG ratio to the gross savings for all RNC participant housing units for the 2019-2021 program period. Delphi panelists indicated that the programs have had the largest impact on duct leakage, air infiltration, and insulation installation quality. The PAs should consider focusing future training efforts on these measures.

RLPNC 17-14 Residential New Construction Incremental Cost

Synopsis: The goal of this study is to document the incremental costs associated with participating in the Massachusetts Residential New Construction (RNC) Low-Rise Path and Multifamily High-Rise (MFHR) programs. Specifically, this study identifies the incremental costs associated with moving from baseline new construction practices to common practices identified in program participant housing units. This report addresses incremental costs for single-family detached homes, single-family attached homes, low-rise multifamily housing units, and high-rise multifamily buildings. To complete this study, the team developed a list of efficiency measures based on a review of program home sample data. The team then worked directly with building contractors to procure information on the cost for each measure and aggregated measure-level costs to develop incremental cost values for single-family detached homes, single-family attached homes, and apartments/condominiums.

Conclusion: Based on the findings from this study, evaluators offer several conclusions, recommendations, and considerations. Conclusions include the following: The overall incremental cost associated with participating in the MFHR is $2.09 per square foot for all project types. Building participant projects have incremental costs ranging from $2.05 per square foot for mid-rise buildings with all electric mechanical equipment to $2.53 per square foot for mid-rise gut rehab projects. Projects participating in the “in-unit only” path have the lowest incremental cost ($0.32 per square foot). Recruiting contractors to estimate building costs can be extremely difficult, primarily due to time constraints and sensitivity to releasing price information. It came to the attention of the evaluation team that there have been issues regarding the allocation of incremental costs across electric and gas PAs in the past.

RLPNC 185 Home Energy Assessment LED Net-to-Gross Consensus

Synopsis: This study was designed to provide a means through which the Program Administrators and EEAC consultants could come to consensus on what prospective Net-to-Gross ratios (NGTR) should be used for LEDs installed as part of the Home Energy Services (HES) initiative. Prospective NTGR for LEDs distributed as part of the Retail Products initiative were set as part of the RLPNC 17-11 LED NTG Consensus study (presented separately)

Conclusion: The final NTGR and Effective Useful Life (EUL) values adopted by the PAs and EEAC are included in Table 1 of the study report. Since it is uncertain if the upstream LED program will exist for all years 2019 – 2021, the PAs and EEAC consultants agreed to varying NTGR values based on when the upstream program ends. For example, the base case is the upstream program ending on or after December 31, 2021. The other scenarios assume the upstream program ends at some point before or during the 2019 – 2021 program period—resulting in higher NTGR values for LEDs installed as part of Home Energy Assessments.

RES 1 Baseline Load Shape Study

Appendix C-1 Saturation Results Plots

Appendix C-2 Annual Results Plots

Appendix C-3 All End Use Whole Period Results

Appendix C-4-1 HVAC Load Shape Results

Appendix C-4-2 Water Heating Load Shape Results

Appendix C-4-3 Kitchen Load Shape Results

Appendix C-4-4 Laundry Load Shape Results

Appendix C-4-5 Miscellaneous Load Shape Results

Appendix C-4-6 Lighting Load Shape Results

Appendix C-4-7 Remaining Load Load Shape Results

Synopsis: The primary goal of this study is to collect saturation, penetration, and usage behavior data for all major electric and gas appliances, mechanical equipment, and electronics in Massachusetts homes. These data will support energy and peak demand savings calculations for program evaluation and design, as well as provide additional insight on the savings potential in the existing residential buildings market. The research team surveyed 6,673 households, asking questions about household occupants and home equipment. Survey results were trued up and expanded upon with onsite metering of 25 end uses at 356 sites.

Conclusion: Evaluators drew several conclusions from the study: Across all homes in Massachusetts, central AC and room AC are by far the largest contributors to peak demand. Collectively, cooling makes up about half of total residential ISO-NE peak demand. Across the remaining end uses, individual homes have a wide variety of significant end use loads during peak times. Heat pump water heaters appear to use about half as much electricity as domestic water heaters, which corroborates much of the expected energy savings. Lighting is the end use with the largest contribution to total winter consumption and winter peak. Electric resistance heat has a surprisingly flat hourly load shape on peak days, more like a refrigerator than an air conditioner on a peak day. Electric resistance heating consumption is highly variable. The top 25% of households with electric resistance heat presently consume approximately 10 times as much as the median. Electric water heaters offer the largest non-HVAC, non-lighting opportunity for winter peak demand savings.

RLPNC 17-11 LED Net-to-Gross Consensus Panel Report

Synopsis: This report describes the process by which the Massachusetts Program Administrators (PAs), the Energy Efficiency Advisory Council Consultants (EEAC consultants), and the residential retail lighting evaluation team, led by evaluators, developed prospective net-to-gross ratios (NTGRs) for light emitting diode bulbs (LEDs). The PAs and EEAC consultants will use these prospective NTGRs to help plan the 2019 to 2021 Residential Lighting Core Initiative. The study relies on a consensus-building process to develop prospective NTGRs for the Residential Retail Initiative for 2019 to 2021 for the following LED types: Standard (medium screw base, meant to reflect A-line bulbs), Reflector/floods, and Specialty (focused on globe and candelabra bulbs).

Conclusion: Table 7 of the report summarizes the consensus panel’s recommended NTGRs for LED bulbs and fixtures. Evaluators recommend that the PAs and EEAC consultants use these NTGRs for planning the 2019 to 2021 programs.

RLPNC 1715 Shelf Stocking Study Report

Synopsis: This study conducted a shelf-stocking and price survey to evaluate the impact of the Mass Save® residential lighting program on consumer retail lighting in Massachusetts. The study focused exclusively on participating store locations in Massachusetts and does not include any data collection or analysis of data from a comparison area. The objectives of this study were to assess the following indicators at participating Massachusetts retailers: The amount of shelf area dedicated to LEDs, CFLs, halogens, and incandescent lamps in Massachusetts; The amount of shelf area dedicated to linear LEDs and linear fluorescent lamps; The pricing, number of lamps per package, and shelf locations of LED, CFL, halogen, and incandescent lamps in Massachusetts; The differences in pricing and availability of LED ENERGY STAR vs. Non-ENERGY STAR products in Massachusetts; and the current stock of lamps to determine coverage by the Energy and Independence Security Act of 2007 Phase I and Phase II.

Conclusion: When compared to 2016 data, the current data show an increase in the shelf share of LEDs and a decrease in the average unit price of LEDs – especially for ENERGY STAR. Detailed findings related to stocking and display, product pricing, EISA coverage and exemptions, as well as guidance for future studies are included in the report.

2013-2015 Residential Customer Profile Report

2013-2015 Residential Customer Profile Study Stakeholder Summary

Synopsis: The 2013 – 2015 Residential Customer Profile report provides a “snap-shot in time” data retrospective though tables, charts, and geographic analysis of the Massachusetts Program Administrators residential and low income energy efficiency program activities for the 2013-2015 term period. The analysis and reporting of the statewide data allow the PAs and the Energy Efficiency Advisory Council (EEAC) Consultants to accurately quantify and report on the trends and time series evolution of energy efficiency in the Massachusetts residential landscape. The data presented in the Residential Customer Profile report is compiled from the individual program administrator’s customer information, billing, and tracking databases and is augmented by third party Massachusetts demographic data from the US Census Bureau’s American Community Survey (ACS) and building characteristics data from the Massachusetts Level 3 Tax Assessor Data and the City of Boston Tax Assessor Data. The bulk of the analysis is conducted at the account level, though some analysis sections like the demographic profiles aggregate accounts up to a geographic block group reporting level.

Conclusion: Results of the study are presented at a series of different analysis grains including program administrator, fuel, and geographies. Key findings suggest that the block group level geographic assessments using the ACS and tax data confirmed that both across and within individual PAs, there are geographic differences in program participation and savings. As a quintile grouping, block groups with proportionally more lower income households tended to have lower participation rates and savings rates. As a quintile grouping, block groups with proportionally more rental households tended to have lower participation rates and savings rates. Geographic hot spots and clusters show more spatial clustering than a random distribution would produce. Over the 2013-2015 term period 14.5% of unique electric accounts and 10.8% of unique gas accounts participated in a PA program offering. Gross electric population level savings for the individual years generally vary between 2% to 4% of consumption for that year; gross gas population level savings vary between 0.5% and 1.5%. The electric residential products program had a declining share of savings over the three year term; but still contributed the largest share of savings due to the upstream lighting offering. Last, the Home Energy Services core initiative contributed heavily to account participation and savings in the Whole House Program.

RES 29 Quick Hit Study: Ductless Mini-Split Heat Pump Survey Final Memo

Synopsis: Through the Mass Save® Heating and Cooling program, Massachusetts residential customers are offered prescriptive rebates for the installation of qualifying ductless mini-split heat pump (DMSHP) systems. In 2014, an evaluation team conducted a web survey of participants who had received incentives for installing DMSHPs. The purpose of the previous survey was to understand participant motivations for participating in the program and how the DMSHP equipment was being used. From the survey results, the evaluation team was planning to determine the appropriate baseline for program incentivized DMSHPs. In 2017, the Heating and Cooling program began adopting changes to encourage greater use of the rebated DMSHPs for heating and support a higher EFLH value and enacted several program changes. The primary research question explored by the current study was as follows: Have any recent changes to the DMSHP program resulted in changes to how participants use their installed DMSHP equipment, specifically whether they are using their DMSHPs more for heating? This question was assessed through a participation survey, which mirrored the participant survey (and in many cases replicated questions from the survey) completed in 2014.

Conclusion: The primary finding from this survey is that a larger percentage (89%) of 2017 program participants are using the DMSHPs rebated through the Mass Save Heating & Cooling Program for heating than 2014 program participants (75%). The survey results also indicate that the enacted program changes are resulting in a significant number of program participants learning about the heating capabilities of their DMSHPs through contractors and program materials. Fifteen percent more program participants report using their DMSHPs for heating through the 2017 participant survey as compared to the 2014 survey. Evaluators therefore recommend that the program use 535 EFLHs as the basis for estimating heating savings for DMSHPs, which is higher than the average (451 EFLH) derived from the 2016 metering study and currently being used by the PAs. Sixty-one percent of survey respondents indicated that they learned about the heating capabilities of their DMSHP from their contractor or the Mass Save program. The Mass Save Heating & Cooling program should therefore continue enforcing the program changes enacted in 2016, as these program changes are resulting in higher heating usage of rebated DMSHPs.

Residential Baseline Study 2017 Saturation and Characterization Results

Residential Baseline Load Shape Study Heating Season Report

Residential Baseline Load Shape Study Cooling Season Report

Residential Baseline Study Appendix C-1: Heating Season Results Plots

RLPNC 17-6 Market Adoption Model Findings Memo

RLPNC_17-6 Reflector Market Adoption Model

RLPNC 17-6 GSL Market Adoption Model

RLPNC 17-6 Specialty Market Adoption Model

Synopsis: The 2017 Market Adoption Models (MAM) effort encompassed four main objectives: Update 2016 Annual Report General Service Lamp MAM; Update 2016 Reflector MAM; Develop a specialty-specific MAM; Estimate delta watts for each separate MAM. To accomplish these goals, the evaluation team worked with the PAs and EEAC consultants to predict light-emitting diode (LED), compact fluorescent lamp (CFL), halogen, and incandescent market share in the absence of the program from 2017 through 2025 for general service, reflector, and specialty (mostly candelabra and globe) bulbs. Data sources included program suppliers (manufacturers and retailers), bulb shipment data, bulb sales data, consumer panel data, and on-site saturation panel data. Market information came from program suppliers, ENERGY STAR® partners, news articles, blog posts by industry leaders (e.g., NEMA, ACEEE), and reviews of recent rulemakings and lawsuit settlements stemming from the Energy Independence and Security Act (EISA).Through an iterative process, the evaluation team presented evaluation information (which became available over time, rather than at a single point in time). Using these data, as well as expertise and knowledge, the PAs, EEAC, and NMR each made predictions about future market share. Evaluators also used program data and internet-based research to update the crosswalk between wattage and lumen bins.

Conclusion: The MAMs utilized multiple wattage categories within each model. Evaluators assigned a single wattage value per bulb type, representing the most common wattage linking bulbs of similar lumen. Evaluators included the lowest wattage bulbs in the specialty MAM but not the general service or reflector MAMs. Evaluators also developed sales weights for each wattage category based on 2017 program sales. Table 1 of the memo lists the wattage categories and sales weights. The models calculate gross and net delta watts for the upstream program and delta watts for the direct install programs. For upstream, gross delta watts excludes baseline LED sales, while net delta watts includes them. This distinction does not apply to the direct install programs because they only switch out inefficient bulbs. Table 2 of the memo presents the delta watts produced by the general service, reflector, and specialty MAMs

CLC and National Grid Education Kits Deemed Savings Review Memo

Synopsis: This interim memo provides deemed savings review results for the Cape Light Compact (CLC) and National Grid (NGRID) Be Energy Efficiency Smart (BEES) Program. The CLC and NGRID BEES Programs provide K-12 students with take-home kits containing energy efficient measures, including light-emitting diodes (LEDs), kitchen and bathroom faucet aerators, showerheads (NGRID only), thermostat temperature cards (CLC only), and hot water temperature cards. Evaluation tasks included: Deemed savings updates based on Massachusetts TRM 2017 values; census and survey data on the single family and multifamily mix, and an engineering-based review of hot water and thermostat temperature card savings; and calculation of cumulative lifetime savings for LEDs, accounting for bulbs installed in future years.

Conclusion: The evaluation team recommends using the values in Table 1 of the memo as preliminary deemed gross savings estimates for CLC, and the values in Table 2 for NGRID. These calculated results incorporate the mix of single-family versus multifamily households and the weighted mix of fuel types for hot water and temperature cards. These tables also include the estimate useful life of each of the measures as established by the Massachusetts TRM3(for LEDs, aerators, and showerheads) and the Illinois TRM (for temperature cards).

National Grid 2017 Residential Wi-Fi Thermostat DR Evaluation

Synopsis: This report presents results of an impact and process evaluation of National Grid’s residential Wi-Fi thermostat DR program. The 2017 evaluation builds upon the 2016 evaluation by further focusing the evaluation objectives and refining the evaluation approaches. The evaluation approach relied on several methods: 1. Post-season survey to gain feedback from 2017 Massachusetts program participants 2. Thermostat usage assessment that combines and analyzes thermostat telemetry data and event participation data 3. Regression analysis to estimate demand and energy impacts.

Conclusion: The evaluation of the 2017 DR season found the program was successful both in testing the effectiveness of thermostats as a residential DR technology and in customer acceptance of the program offering. Although the study confirmed the technical feasibility of using thermostats to reduce household peak demands, it has not looked at whether that control will be cost-effective for the electric system, program administrators, and/or customers. Furthermore, there was only one event in the 2016 and 2017 summer seasons when the average outdoor temperature exceeded 90°F.1 As a result, National Grid has not yet had the opportunity to test program performance under more extreme weather conditions. Nevertheless, National Grid is positioned to leverage the experience of the 2016 and 2017 program years to further test the technology and program offering in 2018 to inform future program design.

National 2017 Seasonal Savings Evaluation

Synopsis: In this report, evaluators presents findings of an impact evaluation of the Seasonal Savings (SS) program. The SS program is designed to make small adjustments to scheduled setpoints over a 3-week period (i.e., tune-up period) while maintaining customer comfort. On average, scheduled setpoints are adjusted up by 1.5°F during the cooling season, with the biggest temperature adjustments typically taking place when customers are away from home (e.g., the middle of weekdays). The SS program was deployed using a randomized encouragement design (RED), in which all customer with a Nest thermostat were randomly assigned into an intent-to-treat (ITT) group or a control group. To complete the evaluation, evaluators performed an exploratory analysis of thermostat telemetry data examining scheduled setpoints and cooling runtime across the ITT, treated, and control groups. In addition, evaluators estimated impacts using regression analysis.

Conclusion: The evaluation confirmed the technical feasibility of using thermostats to reduce household energy consumption and peak demand and identified the energy and demand savings achieved during 2017 in Massachusetts and Rhode Island. The program achieved energy and demand savings of 189 MWh and 366 kW in Massachusetts, and 57 MWh and 134 kW in Rhode Island. The evaluation shows promise for thermostat optimization, though important questions remain regarding incremental savings from future deployments, persistence of savings, and expected savings from a full season deployment under warmer weather conditions.

RES 19 Water Heating, Boiler, and Furnace Cost Study

Synopsis: The goal of this study is to evaluate the energy-efficiency-related costs of single family home installations of water heater, boiler, and furnace products currently rebated through the Residential Heating and Cooling program. This study relies on two main data sources: 1) a survey of HVAC contractors and plumbers in Massachusetts, and 2) retail prices gathered by webscraping. This study combines data from these two sources to construct cost-efficiency curves that describe the total installed cost of water heater, boiler, and furnace products across the available range of efficiency levels for each product type. The evaluation team first defined the representative product sizes that are typically installed in the PAs’ service areas. The team then gathered cost data by surveying contractors and webscraping retail prices, and merged the data from these two sources to construct cost-efficiency curves.

Conclusion: Table 2 in this report presents the total installed cost for each product type at the baseline efficiency level and at the efficiency levels rebated through the Residential Heating and Cooling Program. Section 3.2 of this report presents complete cost curves covering the full range of product efficiencies available for each product type. Based on the results of the study, there are two strategies the Program Administrators (PAs) could pursue to improve the cost effectiveness of water heater, boiler, and furnace rebate offerings. First, the PAs should consider discontinuing the lower rebate level of 90% AFUE for combination boilers, and continue incentivizing the adoption of higher-efficiency 95% AFUE models that appear to be a less expensive option for customers. Second, if the high-volume electric resistance models are found to be prevalent, the PAs should consider reassessing the baseline for the large electric storage water heater product class. The evaluation team is reaching out to survey respondents to ask how frequently these large electric resistance water heaters are installed in Massachusetts. However, the number of survey respondents was limited, and any data they provide will be limited. The PAs should consider pursuing additional survey efforts to understand how prevalent these models are in Massachusetts.

RES 44: Multi-family Program Impact and Net-to-Gross Evaluation

Synopsis: The Massachusetts Multi-Family Retrofit Program (MMRP) provides comprehensive rebate opportunities for standard income multi-family (MF) buildings with a minimum of five residential units. To improve the accuracy of MMRP-reported savings, stakeholders requested an updated impact evaluation of lighting measures and program-wide net-to-gross (NTG) research. The focus on lighting measures for impact evaluation activities reflects lighting’s high contribution to overall program savings. The lighting impact evaluation objectives were addressed through database analysis, project engineering file reviews, and site visits. The evaluation team conducted NTG research through surveys targeted at MMRP participating and nonparticipating renters, condominium owners, and property managers/owners.

Conclusion: This study resulted in a series of conclusions for the MMRP. Some of these are quantitative, which affect the calculation of program savings, and some qualitative, identifying opportunities for improving program activities. The quantitative and qualitative recommendations from both the impact and NTG portions of this evaluation are presented in Table 1. Evaluators found that the vendor-reported common area HOU had no correlation with the logged HOU. The high variance led to low precision on the HOU realization rate. The findings around common area HOU resulted in different recommendations by PA, as the PAs use two separate practices for assigning ex ante common area HOU. The evaluation team found ISRs of 1.0 for common areas and 0.88 for in-unit areas. The evaluation team found a free ridership ratio of 0.21 and a provisional spillover ratio of 0.28, resulting in a provisional program-level NTG ratio of 1.07. In general, quality assurance/quality control (QA/QC) processes had improved for the MMRP since the implementation; however, paperwork used by the implementation vendors to track measure installation still varies tremendously. The PAs indicate that in process QA/QC visits are important to review existing conditions; however, it seems that there is confusion as to what type of visit is supposed to occur and what data is meant to be collected in each instance. Almost half of renters surveyed, all of whom live in 2017 MMRP participating buildings, did not know that their building had participated in the program.

RES 35: Home Energy Services Process Evaluation Comprehensive Report

Synopsis: This report presents results of a process evaluation of the Home Energy Services (HES) Program. The evaluation plan focused on key aspects of HES to prepare for the next 3-year (2019-2021) implementation plan. Specifically, evaluation objectives were organized within four primary research areas: 1. Performance of the moderate-income and renter-landlord offers 2. Effectiveness of HEAs 3. QA/QC 4. Overall performance. To meet the research objectives listed above, the evaluation team completed seven primary evaluation activities: Stakeholder interviews, Energy specialist and HPC interviews, Review of program data and materials, Secondary literature review, KPI analysis, Mystery shopping, and Participant surveys.

Conclusion: Key findings related to the overall performance of the program suggest that annual participation waned somewhat in 2016 and 2017 relative to previous years. Word of mouth remains the dominant way customers learn about HEAs. The online assessment is encouraging participation. One-third of all customers installed an HES incentivized major measure. There is a difference in LV recommendation rates. Real differences continue to exist between the LV and HPC channels. Based on these findings, evaluators suggest that the program administrators (PAs) need to lessen the gap between delivery channels and that the next implementation cycle needs to include a no-cost assessment option. They recommend that the PAs examine historical trends in participation and consider the implications on future HES design. Topic-specific key findings and recommendations related to target offers, QA/QC, and HEA effectiveness are also presented in this report.

RLPNC 16-11: Lighting Market Scan Memo

Synopsis: This fifth and final quarterly market scan memo for the RLPNC 16-11 study presents information published in the third quarter of 2017 on the U.S. residential lighting market. It also presents the information from four previous quarterly memos (i.e. Q3 2016 through Q2 2017). This memo is organized by topic, then by chronology, with the newest information presented first. All sources are from documents published in 2015 or later and, with a few exceptions evaluators note, based on research conducted in 2014 or later. The market scans focus on information gained from areas outside of Massachusetts.

Conclusion: Throughout all five memoranda, results have consistently indicated a national trend of increased LED adoption, demonstrated through studies of market share, socket saturation, and manufacturer and supplier forecasts. The reports that were reviewed in this study attribute the LED growth (and the decline of other technologies) to the upcoming implementation of EISA, ENERGY STAR 2.0 (which eliminated CFLs), and increasingly LED-specific program activity. There is some ambiguity as to the rate at which LEDs will replace halogens as the dominant technology, due to uncertainty surrounding the implementation of EISA. In all the states and regions that are covered in this report, results suggest that greater program activity has led to increased LED adoption. Along with reports of socket saturation and market share, this study looks at program influence, and presents CFL and LED NTG ratios from across the country. Recent policy shifts have led to uncertainty in the enforcement of EISA. Despite uncertainty, during interviews as part of a NEEA market scan, several manufacturers reported that they expect EISA to have a limited effect on their future bulb production.

RLPNC 17-9: Residential Lighting Market Assessment Study

Synopsis: This report presents the results of the 2017-18 Massachusetts Residential Lighting Market Assessment Study. The study was designed to update estimates of lighting saturation and other critical market indicators in Massachusetts. The data for this study came from on-site lighting inventories of homes in Massachusetts and a comparison area made up of portions of New York. Unlike previous waves of the study, the 2017-18 Market Assessment relied entirely on visits to panel households, some of which first took part in onsite saturation studies in 2013. Throughout the report evaluators refer to the saturation and penetration of various lighting technologies. Saturation is the percentage of sockets filled with a specific bulb type. Penetration is the percentage of homes with one or more of a specific lighting technology.

Conclusion: Evidence from this study suggests that the Massachusetts programs continued to have a strong impact on saturation and penetration of LEDs. LED saturation was 27% in Massachusetts compared to only 14% in New York. LED penetration was 86% in Massachusetts compared to 72% in New York. Not only did LED saturation in Massachusetts continue to outpace that in the New York comparison area, but the gap in saturation between the two areas widened in each of the last three years – indicating that LED sales growth has yet to reach a plateau in Massachusetts. Further, ENERGY STAR® LEDs (the only type of LEDs supported by Massachusetts program efforts) accounted for nearly the entire difference in LED saturation between the two areas, providing strong evidence that the Massachusetts programs are continuing to have a profound impact on the market. The report also provides updated discounted lifetime inservice rates for LEDs, found in Table 1, as well as additional key findings, considerations, and guidance.

RLPNC 16-2: Task 8A-2017 Lighting Supplier Interviews Final Report

Synopsis: This report summarizes findings from 2017 in-depth interviews with lighting manufacturers and high-level lighting buyers. This research supports the continued assessment and monitoring of the Massachusetts lighting market and the Massachusetts ENERGY STAR® Lighting Program. Evaluators conducted two waves of lighting supplier in-depth interviews: Wave 1 (September-October 2016) and Wave 2 (October 2017). This report summarizes Wave 2 interview findings and represents the final study deliverable. A separate report summarized results from three other study tasks. During Wave 2 in-depth interviews, lighting suppliers offered market share predictions and shared their perspectives on current lighting market trends.

Conclusion: Evaluators identified several key findings based on results of the study. Lighting suppliers predicted that LED market shares will still dominate without program incentives through 2022, but these LED shares will be much lower. The suppliers predicted that halogen market shares for standard, reflector, and specialty lamps will primarily make up the difference between program and no program scenarios. Suppliers were fairly evenly divided as to the future of federal standards (GSL and backstop). As discussed in the body of this report, some of the suppliers were uncertain about their predictions. Suppliers primarily defined residential lighting market transformation in terms of customers’ buying habits or retail shelf saturation of LEDs; however, they thought that an LED market share of greater than 50% signaled that program incentives were no longer necessary. Suppliers continued to report concerns about non-ENERGY STAR LED lamp quality, although a large majority said product quality had either improved or stayed the same over the past few years. Suppliers estimated that nearly one-half of low lumen (<310) lamp shapes on Massachusetts store shelves use incandescent technology and that manufacturers will primarily transition these lamps to LEDs in the next five years. Suppliers were split about whether California’s early implementation of EISA Phase 2 standards would impact Massachusetts markets, but said unequivocally that LEDs will still dominate the U.S. market, even when taking national/international market trends into consideration.

RLPNC 16-6: Web Scraping Results June 2017-November 2017 Memo

Synopsis: This memo is the final update to MA RLPNC 16-6: Lighting Shelf Stocking report. The purpose of this memo is to provide updated information on web scraping results for average bulb pricing, pricing trends over time, product turnover, and ENERGY STAR® product availability. This memo utilizes data collected from June 2016 to November 2017. For this study, evaluators collected location-specific data from the websites of two major home improvement retailers in Massachusetts and Upstate New York.

Conclusion: Evaluators present serval key findings from the study. LED prices were approaching halogen prices for A-line and reflector/flood bulb types, but remained the more expensive option by high margins in the candle and globe categories. The average cost of standard A-line and reflector/flood LEDs offered for sale at two major home improvement retailers fell between June 2016 and 2017. Among globe and candelabra categories, CFL options appear to be on the decline, and evaluators observed only a small number of halogen products.

RES 40: Moderate Income Market Characterization Survey Findings Final Report

Synopsis: The PAs began providing enhanced incentives to income-eligible households that participate in Home Energy Services in April 2016. This offering, referred to as the Moderate Income offer, targets customers with incomes between 61% and 80% of the state median income (SMI). This market study is intended to characterize the population targeted by the Moderate Income offer. The evaluation team designed the study to explore the following research objectives: 1. Estimate the size of the eligible population within both the existing and potentially expanded moderate income classifications. 2. Describe customer characteristics such as demographics, tenure (rent or own), and awareness and participation in HES and the Moderate Income offering. 3. Assess customer response to the current Moderate Income offering and its requirements. 4. Assess differences between the moderate income groups and those that are income ineligible for the enhanced offering, but could participate in the HES initiative.

Conclusion: Evaluators summarize the most notable conclusions resulting from the Moderate Income Market Characterization study, organized by two topics: 1) population characteristics, and; 2) response to the Moderate Income offering. They found that just under half of the population in Massachusetts self-identifies as qualifying for either the existing Moderate Income offer or one of the potentially expanded moderate income groups. Customers that fall within the existing moderate income group face greater economic and energy challenges than the higher income households surveyed. The existing moderate income group is as likely to receive a HEA and install recommended measures as those with higher incomes. The existing moderate income group is more likely to cite “affordability” as a participation barrier than those that report higher income. Participants’ life stages may correspond with program awareness more than income. Household characteristics other than income may be a greater determinant of whether customers install energy efficiency measures. Customer time and availability, perception of their need for energy efficiency, and need for more information are the greatest barriers to participation in the Moderate Income offering. The income verification process and perception of eligibility may be a barrier to participating in the Moderate Income offering. Personalized and more in-depth discussion on the requirements may help to overcome that barrier. Customer trust in their PA and positive prior experiences drive willingness to participate.

TXC 46: Analysis of Immediate Code Compliance Support Initiative Residential Training Surveys on 2015 IECC Memo

Synopsis: This memo provides analyses of the immediate survey responses collected through paper surveys, registration data, and Audience Response Systems (ARS) from ten Code Compliance Support Initiative (CCSI) residential trainings held from May 31 through November 21, 2017. The CCSI contractor held the trainings on Envelope and Building Science (EBS) and HVAC and Indoor Air Quality (HVAC-IAQ), based on the residential 2015 IECC code which became mandatory in Massachusetts at the beginning of 2017.

Conclusion: The overall response rate of 73% is similar to that for the trainings held in the early spring of 2017 and much higher than the 58% response rate from trainings held in the fall of 2016. Four hundred individuals attended only residential trainings, 333 attended only commercial trainings, and 154 individuals attended both residential and commercial trainings. Respondents provided positive feedback on the quality of the trainings. Two-thirds of respondents said they expect to use the information offered in the training immediately, while an additional 21% said they expected to use it within the next three months. Using an ARS provided during the training, all training attendees indicated that an average of just over three-fifths of the permits they drew, or which were drawn in their jurisdictions, were for retrofit projects, and three out of five of these retrofits were energy-related. EBS training attendees found information about different types of insulation to be the most useful information provided in the trainings. For the most recent EBS trainings the mean usefulness (on a 6-to-1 scale, in which 6 is extremely useful and 1 is not at all useful) is 5.3. Relatively few respondents offered additional comments.

RLPNC 18-1: Appliance Recycling Database Review and Savings Update Memo

Synopsis: This memo presents results the first task of a proposed refrigerator and freezer recycling program evaluation the Residential Consumer Products Core Initiative. The overall study scope is not yet final. Task 1 of the proposed study used program tracking data, information from a recent literature review, and savings algorithms recommended by the Uniform Methods Protocol (UMP) to update unit energy consumption (UEC) and per-unit energy savings values for refrigerators and freezers. Evaluators compare the results of this effort to the 2011 results and to the UEC found in recently completed evaluations of other appliance recycling programs across the nation.

RLPNC 17-12: Lighting Decision Making Report

Synopsis: This study investigated what factors motivated consumers to purchase efficient lighting and what factors influenced consumers to instead purchase inefficient lighting. Evaluators analyzed 2016 and 2017 lighting purchase data and administered surveys to recent shoppers in 2016 and 2017 in Massachusetts and portions of New York from a panel of lighting purchasers provided by InfoScout.

Conclusion: Results of the study show that program support for efficient bulbs is continuing to have a positive influence on the retail lighting market in Massachusetts. Based on key findings from purchase data and survey results, evaluators provided the following considerations. First, the PAs should consider increasing the emphasis on the improved quality of LED bulbs in the areas of color and light quality. Such bulb characteristics might include light quality, appearance, or shape (wider availability of specialty bulbs). Second, the PAs should make sure to continue leveraging numerous communication channels to extol the virtues of efficient lighting to customers. Casting a wide net will help ensure customers are educated about efficient lamps prior to making a decision to visit a retailer to purchase a bulb.

RLPNC 16-10: What Is Next for Products – Market Scan

Synopsis: This study is intended to help the Massachusetts Program Administrators as they plan for the future of the residential lighting and residential consumer products initiatives, given significant market changes and increasing federal standards. This includes identifying products for possible inclusion in the MA residential program portfolio in the 2019 to 2021 plan cycle that could successfully achieve savings. This report expands on the Task 1 Documentation of Codes and Standards memo (included as Appendix A). The products discussed in this report center mostly around white goods, consumer electronics, and lighting controls. Although HVAC equipment is not specifically discussed, some strategies or behavioral-based products in the report (e.g., smart thermostats) may impact HVAC system usage. The goal of the Task 2 Market Scan and report was to gather and report market intelligence from experts and thought leaders across the United States who have examined, or are in the process of examining, new residential energy and demand savings measures. Evaluators completed 17 interviews with 23 respondents throughout August and September of 2017. Evaluators then conducted follow-up research based on the key takeaways of these conversations, identifying reports, products and technologies, and other valuable information.

Conclusion: While respondents agree that savings from products are unlikely to reach the level of savings from lighting programs, there are still opportunities to achieve savings by leveraging the most efficient products on the market through well-structured programs. Increasing consumer awareness of efficiency and having the utility be present when they are researching or purchasing a new product was deemed critical in these efforts. Many respondents also advocated for switching some products to a midstream approach (specifically the ENERGY STAR Retail Products Platform approach) as a way to increase market influence at lower per-unit costs. Finally, respondents thought that embracing new program designs and/or technologies would provide a substantial opportunity for energy efficiency and demand-response. Interviewees specifically mentioned smart thermostats, connected home devices, advanced power strips, EV chargers, and EV infrastructure and grid management strategies in general.

RLPNC 17-2 – Single-Family New Construction Mini-Baseline Study

Synopsis: This report details the results of a single-family residential new construction (RNC) “mini-baseline” study. The study was designed to answer two key questions about the single-family homes that were built at the end of the 2012 International Energy Conservation Code (IECC) cycle: 1) How has the market baseline changed since the beginning of the 2012 IECC cycle for key measures such as air leakage, duct leakage, lighting, and mechanical equipment? 2) How has code compliance changed since the beginning of the 2012 IECC cycle? The term “mini-baseline” refers to the reduced data collection effort that took place during on-site inspections. This study included 50 site visits, conducted in 2017, to new, non-program single-family homes (29 spec- and 21 custom-built) across 37 Massachusetts cities and towns that have not adopted the stretch code.

Conclusion: The MA-REC compliance approach showed a significant improvement in compliance with the 2012 IECC lighting requirements from the beginning to the end of the 2012 IECC cycle (64% to 92%). Alternatively, compliance with air leakage and duct leakage requirements (using the MA-REC approach) remained relatively stable over the course of the code cycle. Evaluators suggest that the CCSI should continue to focus training efforts on air leakage and duct leakage requirements. Program homes far outperform non-program homes in terms of overall efficiency (HERS score) and for all of the examined measures. These measures include air leakage, duct leakage, and lighting. Evaluators indicatat that the program should continue its successful promotion of high efficiency residential new construction. As new energy codes are adopted, the program should consider adjusting participation requirements in order to maintain a substantial efficiency gap between program and non-program homes.

2017

RLPNC 16-5 and 17-10 Sales Data Analysis and Modeling Report

Synopsis: This report describes the results of two studies that analyzed sales data from a dataset compiled by the Consortium for Residential Energy Efficiency Data (CREED). One study describes market share of screw-base light bulbs, and the other estimates net-to-gross ratios (NTGR) for light emitting diodes (LEDs). The study results will primarily be used to inform the development of market adoption models and the LED NTG consensus process.

Conclusion: The studies show that energy-efficient lighting programs in Massachusetts and beyond have had a profound impact on the market. Compared to non-program states, Massachusetts and other program states have higher market shares of LEDs and compact fluorescent lamps (CFLs). In addition, the data suggest that greater adoption of energy-efficient bulbs (in terms of socket saturation and sales) leads to lower bulb sales overall due to the longer measure life of CFLs and LEDs. As expected, the market shares for LEDs and halogens have increased since 2009, whereas market share for incandescents and CFLs are decreasing. When examining market share by lumen ranges, evaluators observe that inefficient lamps dominate the lowest and highest lumen ranges. Many of these bulbs are currently exempt from federal lighting standard increases, and some will remain exempt in 2020. This offers an opportunity for targeted program opportunities. Statistical modeling results confirm the importance of program spending for boosting LED sales. Other factors also influence LED sales: program age, income, concentration of certain retail stores, and political leanings. The significance of program age suggests that a portion of efficient lighting sales are due to permanent changes in the market resulting from ongoing program activity. When evaluators estimated NTGRs for LEDs, they compared predicted LED sales with actual 2016 Massachusetts program spending and assuming zero spending in the state. Evaluators treated program age two ways: one assuming the program never existed, and the other assuming the program existing until 2015 but not in 2016. The NTGR for the first approach is 55% and for the second approach is 39%.

TCX 46 – Code Compliance Support Initiative Interview Follow-Ups

Synopsis: In 2017, as part of the ongoing evaluation of the Massachusetts Code Compliance Support Initiative (CCSI), evaluators conducted follow-up in-depth interviews (IDIs) with 40 individuals who had attended one or more residential classroom training approximately six months earlier. This report summarizes the findings from these interviews.

Conclusion: Key findings include the following: Most respondents made some changes to their work because of the trainings. These changes were predominantly focused on insulation and envelope areas, builder and contractor education, and ventilation. The most useful aspects of the trainings for all respondents were the updates and summaries on code changes. Almost all respondents had shared some of the information from the trainings with other parties; close to one-half also shared the handouts and other materials provided at the trainings. More than one-half of all respondents have recommended the trainings to others previously, and the vast majority said they would recommend the trainings to others in the future. Respondents’ suggestions for improving the trainings include having a stronger focus on specific areas and focusing on best practices for code compliance. Both municipal building code employees and building professionals suggested that offering continuing education credits could increasing training awareness and attendance among builders and contractors.

RES 17-Connected Thermostats and Technology Literature Review Memo

Synopsis: This memo summarizes findings from a comprehensive review of literature pertaining to smart thermostats and other connected devices in residential settings. There were several goals for this study:Provide an understanding of the state of existing technologies and their effects on energy consumption, describe the likely direction of emerging and future technologies, and outline potential future effects on home energy use and savings. The research team synthesized its findings from a wide range of sources and distilled key findings to inform the program administrators’ (PA) decision whether to invest in deeper, targeted research on savings attributable to these connected devices.

Conclusion: The review of existing literature on smart thermostats and other connected devices revealed these products are gaining traction in the consumer market, with thermostat market share increasing most rapidly. For this reason and others outlined in this memo, the research team recommends performing a primary research study of smart thermostat energy savings specific to Massachusetts. Energy savings have been identified for smart thermostats in different utility programs across North America, but reports from recent studies are not consistent in their findings across multiple years, climate zones, or thermostat types. Therefore, a primary research study will provide current, geographically and programmatically relevant data to further inform future decisions regarding incentivizing smart thermostat purchases by Massachusetts utility customers. Furthermore, due to the lack of consistent data regarding other connected devices and the relatively sparse market for these products, evaluators recommends not performing a primary research study for such devices at this time. However, as the market for other connected household devices grows and the technology develops, such an updated environment could justify a primary research study in the future.

RLPNC 16-11 Fourth Quarterly Lighting Market Scan Memo

Synopsis: This fourth quarterly market scan memo for the RLPNC 16-11 study presents information published in the second quarter of 2017 on the U.S. residential lighting market. This memo is organized by topic, then by chronology, with the newest information presented first, followed by earlier information from previous memos. All sources are from documents published in 2015 or later and, with a few exceptions, based on research conducted in 2014 or later. The market scans focus on information gained from areas outside of Massachusetts.

Conclusion: This Q1 2017 market scan presents one new discussion of the effects of EISA 2020 (Effects of EISA) and three new sets of NTG estimates. Suppliers interviewed for a CT LED NTG evaluation expect that EISA 2020 will cause LED sales to rise. (This is an expectation we have reported in previous market scans.) These suppliers also expect that ENERGY STAR 2.0 will stimulate LED sales while causing CFL sales to gradually decline, and that EISA 2020 will cause halogen sales to decline. The CT LED NTG evaluation also included market share predictions (Sales). With continued program support, suppliers expect that in 2021, LEDs will account for 61% of standard residential bulb sales and 53% of reflector residential bulb sales in CT. However, without continued program support, the suppliers expect LEDs would account for only 45% and 38% of standard and reflector residential bulb sales, respectively. The three new sets of NTG estimates are for 2015 and 2016. The 2015 NTG estimates range from 0.21 to 0.46 for standard CFLs (CA), from 0.31 to 1.1 for specialty CFLs (CA), and from 0.30 to 0.83 for LEDs (CA and CT). The 2016 NTG estimates range from 0.57 to 0.77 for LEDs (CT and WI).

Lighting Insights Final Report of 2016 Activities

Synopsis: This report summarizes findings from three primary research tasks conducted in 2016 with lighting manufacturers, high-level retail buyers and store managers (“lighting market actors”) to support the continued assessment and monitoring of the Massachusetts lighting market and the Massachusetts ENERGY STAR® Lighting Program (“the program”). Primary data collection tasks included: In-depth interviews conducted with 14 participating manufacturers and four high-level retail buyers as follow-up to interviews initially conducted in April 2015, CATI surveys completed with 243 store managers participating in the Massachusetts lighting program, and Meetings and informal discussions at the ENERGY STAR Partners Meeting.

Conclusion: The key findings from the three research tasks address market share and sales predictions, price predictions, EISA 2020 implementation scenarios, and how to improve program effectiveness for low-income/renter customers. Complete findings for each research task appear in the main body of this report including in-depth interviews (Section 2), CATI survey (Section 3) and the ENERGY STAR Partners Meeting (Section 4). These research tasks form part of a larger study, Lighting Supplier Interviews and Store Manager Surveys (RLPNC 16-2). This study also includes one additional outstanding task, a second round of lighting supplier in-depth interviews. Considerations based on the research findings are also summarized in this report.

Single-Family Stretch Code Update Compliance and Potential

Synopsis: Beginning in January of 2017, the Massachusetts stretch code references the 2015 International Energy Conservation Code (IECC) rather than the 2009 IECC that it was previously based on. The new stretch code requirements are being referred to as the 2015 Stretch Code Update. This study calculates projected statewide compliance estimates for stretch code homes and assesses the projected potential savings from compliance enhancement from single family homes that will be built under the 2015 Stretch Code Update. The Program Administrators (PAs) and Energy Efficiency Advisory Council (EEAC) determined that this study was an opportunity to conduct a simplified analysis, based on pre-existing data collection, to determine if projected stretch code compliance and the associated compliance enhancement savings potential justify further evaluation research.

Conclusion: The 2015 Stretch Code Update resulted in a significant increase in efficiency requirements for stretch code homes. While this report is based on homes that were built under the previous stretch code requirements, we project that the 2015 Stretch Code Update will result in decreased compliance statewide for stretch code homes and an increase in the potential for compliance enhancement. Assuming building practices remain the same, statewide compliance using the MA-REC (Massachusetts Residential Evaluation Contract) methodology is projected to be 90% under the 2015 Stretch Code Update requirements. This compares to a statewide compliance rate of 96% under the previous stretch code requirements. Maximum projected potential energy savings from compliance enhancement are 14% statewide using the MA-REC methodology. This compares to 5% under the previous stretch code requirements. Maximum projected potential energy savings from compliance enhancement range from 13% to 34% using methods that compare HERS scores to energy usage. Evaluators recommend that the PAs continue to provide trainings and support for compliance enhancement in stretch code municipalities and that they measure compliance with homes that were permitted and built under the 2015 Stretch Code Update.

Lighting Market Assessment Consumer Survey and On-Site Saturation Study

Synopsis: This report presents results of the 2016-17 Massachusetts Residential Lighting Market Assessment. The study was designed to update estimates of lighting saturation and other critical market indicators, and assess consumers’ knowledge of and interactions with the lighting market in Massachusetts. The data for this study came from randomly sampled web and telephone surreys with residents and on-site lighting inventories of homes in Massachusetts and a comparison area (portions of New York, namely a 40-mile radius around the cities of Albany, Buffalo, Rochester, and Syracuse, as well as all of Westchester County, referred to in this report ass New York) completed between October of 2016 and January of 2017.

Conclusion: Evidence from this study suggests that the Massachusetts programs have had a strong impact on saturation and penetration of LEDs. While consumers in New York are adopting LEDs, LED saturation and penetration rates continue to lag the rates measured in Massachusetts. Further, ENERGY STAR® LEDs (the only type of LEDs supported by program efforts) accounted for seven of the eight-percentage-point difference in LED saturation between the two states. In 2013, Massachusetts and New York had similar levels of CFL saturation (28% and 26%, respectively), but began to diverge between 2013 and 2017; by 2017, CFL saturation in Massachusetts was 25% higher than New York’s saturation 29% vs. 22%). This divergence aligned closely with the New York State Energy Research & Development Authority’s (NYSERDA) decision to cease spiral CFL incentives in 2012 and essentially all upstream lighting incentives in 2014. Over the past three years, we have observed a steady (though not statistically significant) decline in CFL saturation in Massachusetts, down from 33% in 2014 to 29% in 2017. Moving forward, we expect to see this trend accelerate. Additional findings related to socket saturation trends, comparison area trends, ENERGY STAR LEDs, penetrations, and panel trends, as well as recommendations are also included.

RLPNC Sales Data Analysis Task 1f and Task 2c Memo

Synopsis: This memo presents trends in light bulb sales and market share in Massachusetts, New York, the nation, and program and non-program areas. The purpose of the effort is to understand the dynamics and direction of the residential lighting market. Evaluators obtained national- and state-level light bulb sales data from IRI and Nielsen via the CREED Initiative. IRI and Nielsen each offer a point-of-sale (POS) and a panel dataset. The POS datasets are composed of sales data reported by selected retailers, while the panel datasets are composed of purchases scanned by consumer panelists. The analyses in this memo include standard and specialty CFL, halogen, incandescent, and LED bulbs, and exclude other bulb types (e.g., linear fluorescents). The memo also includes a brief update on Second and Third Quarter 2016 Bulb Shipments, as reported by the Association of Electrical Equipment and Medical Imaging Manufacturers.

Conclusion: Key findings included the following. 2015 LED market share in Massachusetts was approximately 19%, comparable to all program states (20%) but higher than New York (15%) and non-program states (9%). Longitudinal data suggest that lighting programs successfully boost energy-efficient bulb share, nationally and in MA. ENERGY STAR LED bulb sales are increasing nationally and in MA and NY, although NY lags behind MA. Point estimates of market share vary by data source, but they generally point to the same national lighting trends. General bulb sales and market share trends are consistent across study areas. Total bulb sales per household have been declining across the nation. LED market share of NEMA shipments lost ground to halogens and incandescents in the second quarter of 2016, then rebounded in the third quarter. Comparison of the NEMA shipment and retailer market shares shows a lag for incandescents. The evaluation team suggests that the PAs purchase the same datasets for the 2016 calendar year, if Nielsen can provide greater detail on the ENERGY STAR qualification of the panel data.

RES 16 – HEHE Condensing Equipment Barriers Report

RES 16 -HEHE Condensing Equipment Barriers Summary

Synopsis: This study is a follow-on study to the 2014 HEHE Impact Evaluation. The Massachusetts High-Efficiency Heating Equipment (HEHE) Rebate Program offers prescriptive rebates for the installation of new high-efficiency natural gas heating and water heating equipment. The primary objective of this follow-on study was to evaluate the energy savings and costs associated with installing outdoor reset controls to determine the extent to which these and other controls could mitigate the non-condensing problem identified in the earlier study. The evaluation team chose to focus the field portion of this study on outdoor reset controls because they are the most common control strategy for modulating boiler supply temperature in the U.S.

Conclusion: See summary document above.

RLPNC 16-6 – Lighting Shelf Stocking Report

Synopsis: This study included a shelf-stocking and price survey to evaluate the impact of the Mass Save® residential lighting program on consumer retail lighting in Massachusetts. The objectives were to: determine the amount of shelf area dedicated to LED, CFL, halogen, and incandescent lamps in retail stores in Massachusetts and New York; examine the characteristics of these lamps, their prices, and how they are displayed by retailers; gauge the impact of program support on shelf-stocking practices and pricing in Massachusetts compared to a non-program area, New York; show how shelf-stocking practices have changed over time in Massachusetts; and explore web scraping as a complementary method to store visits for shelfstocking studies.

Conclusion: The overall story that emerges from the shelf-stocking data is that LEDs seem poised to dominate lighting aisles in both states, especially in those channels that sell the most lighting. LEDs have reached their highest shelf share thus far, 30%. Incandescent lamps, however, are the most common type on shelves in both Massachusetts and New York. Among participating Massachusetts stores, the shares of incandescent and halogen lamps have increased (up to 40% and 18%, respectively) to fill some of the gap left by the rapid decline of CFLs from their 2012 peak of 69% to 12% in 2016. Additional key findings related to the proportion of lamps on shelves in Massachusetts and New York are also presented in the study.

RLPNC 16-11 Task 4b Market Progress Assessment Memo

Synopsis: This Market Progress Assessment summarizes current and historical values for key indicators of progress toward market transformation in the Massachusetts residential lighting market. The indicators are based on outcomes expected to accrue in the market over time as a result of Massachusetts Residential ENERGY STAR® Lighting Program activities. Phase 1 of the study resulted in a logic model for the program. This document is the first deliverable for Phase 2 of the Massachusetts Lighting Logic Model and Market Scan (Study RLPNC 16-11). For this study, evaluators conducted a detailed review of past Massachusetts lighting studies in order to identify appropriate indicators of progress toward the outcomes in the logic model and extracted historical values. In the process, we identified outcomes for which no indicators are being tracked, or for which additional or improved indicators may be helpful, and developed recommendations for tracking these in future.

Conclusion: Figure 1 at the conclusion of this memo displays the outcomes from the finalized logic model with their related indicators. The sections preceding this figure explain the various indicators and their priority level, along with the evaluator’s recommendations for continued tracking.

Multifamily Program Improvement Strategies Preliminary Report

Synopsis: The goal of the overall Multifamily Program Improvement Strategies study is to identify actionable strategies and innovations to improve the multifamily program performance, realization rates, and overall program cost-effectiveness for both the residential and commercial sectors. The focus of this report, prepared as the Task 4 Preliminary Results Memo, is to identify implementation changes which will result in rapid improvements in program performance. As lighting accounts for 92% of program savings, the lighting end use has been a primary focus of the evaluator’s research. A summary of activities incorporated into this report include: Task 2b: Technical Resource Manual Multifamily Measure Characterization, Task 3a: Project Verification, Lighting Baseline and Delta Watts, and Task 3b: Project Verification, On-Site Verification.

Conclusion: The strategies suggested in this report are limited to the results of project Tasks 2b and 3. Additional analysis of cost effectiveness and comprehensive impacts will be incorporated as part of Task 7 to inform and formalize the final project recommendations. Suggested improvement strategies include: Adjust In-Unit lighting hours of use, use site-specific HOU for common area lighting retrofits, continue to explore prioritizing lighting retrofits to high hours of use locations, strengthen project reporting and verification procedures, shift customer expectations to incorporate additional measures into projects, reinforce auditor training to identify and complete additional measure types, incorporate lighting controls into projects, and update Technical Resource Manual.

2016 Annual Report Market Adoption Model

2016 Annual Report Market Adoption Model-Simplified Version

2016 Annual Report Market Adoption Model-Interim Reflector

Stretch Code Market Effects Study

Synopsis: This study conducted historical case studies in six stretch code communities to examine the possible causal mechanisms that can influence adoption and compliance in different types of communities. The study focused on understanding how the programs and initiatives sponsored by the Program Administrators (PAs) could possibly have affected stretch code-related savings both by affecting individual communities’ adoption of the stretch code, and by increasing energy code compliance in stretch code communities. Evaluators reviewed various publicly available documents from these six communities concerning the adoption of the stretch code, including city council or town meeting minutes as well as other relevant documents such as committee agendas or minutes, municipal annual reports, and newspaper articles.

Conclusion: Key findings from this study include the following: The two main reasons for adoption of the stretch code by the communities studied are the desire to be environmentally responsible, and the Green Community designation which provides access to state funding. The new construction programs sponsored by the PAs, particularly for residential homes, also played a role in the adoption of the stretch code by individual communities by offering incentives to partially compensate for increased construction costs. New construction programs have also likely played a role in increasing the base of HERS raters available, thus helping, at least during the early period, some communities to adopt the stretch code, which uses a performance path for compliance, and requires HERS ratings for all newly built homes. Most code officials considered the Code Compliance Support Initiative (CCSI) trainings sponsored by the PAs to be useful in enforcing the stretch code. However, there has been little use of the Mass Save Energy Code Technical Support outside the classroom.

Lighting Distribution Model Report

Heat Pump Water Heater Impact Study-Volume 1

Heat Pump Water Heaters Impact Study Tasks 1 and 2 Summary

Synopsis: See summary document above.

Conclusion: See summary document above.

RES 25 – Heating and Cooling Contractor Interview Findings

RES 25 Study Summary for 2016 Term Year Report

Synopsis: See summary document above.

Conclusion: See summary document above. [/wptab] [end_wptabset]

MA Multifamily High Rise Baseline Study Report

Addendum to MA Multifamily High Rise Baseline Study

Synopsis: This addendum to the Multifamily High Rise Baseline Study report documents the User Defined Reference Home (UDRH) inputs that will be used when calculating savings retrospectively for 2016 program projects and prospectively for 2017 program projects. The tables in the addendum detail the following information: the current baseline inputs; updated inputs that will be used retrospectively to re-run savings for the 2016 program year; updated inputs that will be used prospectively to calculate savings for the 2017 program year; and sources for new baseline inputs.

Conclusion: Table 1 presents the new baseline inputs for residential and common area lighting. For in-unit residential lighting, the working group agreed to use a value 0.75 watts per square foot; this value will be used both retrospectively and prospectively. For common area lighting, the working group agreed to use different values retrospectively and prospectively. Retrospectively, for use in re-running the 2016 program year savings, the working group agreed to reduce the 2012 IECC lighting requirements by 27%. Prospectively, for use in the 2017 program year, the group agreed to apply a 10% reduction to the 2015 IECC lighting tables. Table 2 presents the new baseline inputs for residential and common area heating systems and detail the assumptions associated with each heating system measure. Table 3 presents the new baseline inputs for residential and common area cooling systems. Retrospectively, all inputs other than ductless mini-splits, are based on ASHRAE 90.1-2010 and the 2012 IECC requirements. Prospectively, all inputs other than ductless mini-splits are based on ASHRAE 90.1-2013 and the 2015 IECC requirements. The ductless mini-split values, both retrospective and prospective, were agreed upon by the working group and are based on the recent ductless mini-split heat pump impact evaluation that was previously mentioned. Table 4 presents the new baseline inputs for in-unit water heating technologies. Retrospectively, the program will use the energy factors specified in the commercial section of the 2012 IECC. For the 2017 program year, the program will use new federal standards that went into place on April 16, 2015 as the baseline inputs for the program. The Team felt it was reasonable to use the previous federal standards when re-running savings for the 2016 program year.

RLPNC 16-3 Lighting Decision Making Memo

Synopsis: This memo describes results of a study interned to determine what motivates consumers to purchase efficient lighting and what factors influence consumers to instead purchase inefficient lighting in Massachusetts and New York. The evaluation team analyzed lighting purchase data and administering surveys to recent shoppers in Massachusetts and New York from a panel of lighting purchasers provided by InfoScout, a research company that has a nationwide panel of customers who upload their receipts from retail and restaurant shopping trips in exchange for various rewards.

Conclusion: Massachusetts lighting consumers are buying more efficient bulbs at a lower cost than their counterparts in New York across nearly all demographic categories and retail channels. Massachusetts consumers are, however, still buying a large share of inefficient bulbs (roughly one-half of all purchases). Efficiency and long life are the most important factors to buyers of efficient lighting. Efficiency is especially valued by Massachusetts consumers. Consumers of inefficient lighting are more likely to be attempting to match their purchases to bulbs that have burned out. Decisions about what type of bulbs to purchase are typically made in advance of the shopping trip, especially for LED bulbs. Retail channels like grocery and discount stores sell a relatively small share of lighting overall, but a disproportionately large share of inefficient bulbs. Low-income consumers, compared to other income groups, are not buying a disproportionate share of lighting at hard-to-reach channels.

RLPNC 16-11 Second Quarterly Lighting Market Scan Memo

Synopsis: This is the second of four market scan memos that are part of the RLPNC 16-11 study. The first memo summarized information on the state of the United States (US) residential lighting market as of September 2016 and where this market might be headed. This second market scan memo presents new information on the US residential lighting market published in the fourth quarter of 2016. The market scans focus on information gained from areas outside of Massachusetts.

Conclusion: One-half of 16 lighting suppliers (manufacturers and retailers) interviewed for the 2015-2016 Northwest lighting market tracking study expect EISA 2020 to drive consumers to choose LEDs going forward. The eleven lighting manufacturers interviewed for this study expect their CFL market share to decline and LED market share to rise more quickly than previously reported. On average, these lighting manufacturers estimated their 2015 US market share to comprise 33% LED, 24% CFL, 23% halogen, and 20% incandescent. The proportion of suppliers’ 2015 LED sales that are ENERGY STAR certified ranged from 35% to 100%, with one quarter of suppliers reporting that all the bulbs they manufactured were ENERGY STAR certified. Evaluated identified three new NTG estimates for lighting programs since the first market scan. All three are from 2015 upstream lighting programs in different portions of Pennsylvania, including the service territories of PPL, PECO, and FirstEnergy. The NTG ratio of 0.61 for all LEDs sold through PPL’s program was derived using a demand elasticity model that estimates free-ridership but not spillover. The NTG ratios of 0.40 for CFLs, 0.66 for standard LEDs, and 0.62 for specialty LEDs incentivized through PECO’s lighting program were based on store intercept surveys (n=330) and include both free-ridership and spillover. The NTG ratio of 0.68 for FirstEnergy’s lighting program is based on interviews with corporate retail chain representatives and includes free-ridership only.

2016

Single Family Code Compliance Baseline Study-Volume-1

Single Family Code Compliance Baseline Study-Volume 2

Single-Family Code Compliance Baseline Study-Volume 3

Single Family Code Compliance Baseline Study-Volume-4

Single-Family Code Compliance Baseline Study-Volume 5

Synopsis: This study was designed to provide the Program Administrators (PAs) and EEAC Consultants with a code compliance assessment of newly constructed single-family homes permitted at the end of the 2009 International Energy Conservation Code (IECC) cycle, homes permitted during the beginning of the 2012 IECC cycle, and homes permitted under the stretch code. To provide the information needed to update the UDRH for the low-rise component (residential buildings three stories or lower) of the RNC program. Another primary goal was to provide the information needed to update the UDRH for the low-rise component (residential buildings three stories or lower) of the RNC program. To meet these goals and answer the research questions listed below, the PAs and EEAC Consultants decided to conduct on-site inspections with non-program single-family homes.

Conclusion: Given the amount of information covered in this study, results are presented in five separate report volumes:

Volume 1: Sampling, Recruitment, and On-site Data Collection This report volume presents the sampling, recruitment, and on-site data collection methods.
Volume 2: Data Analysis Findings This report volume presents detailed findings from the on-site inspections.
Volume 3: Baseline Findings This report volume provides a high-level summary of the baseline data and analysis findings to support updating the Residential New Construction (RNC) program User Defined Reference Home (UDRH).
Volume 4: Compliance Findings This report volume focuses on the code compliance assessment, which is founded on the data presented in the data analysis report volume.
Volume 5: UDRH Addendum This report volume presents the updated Residential New Construction (RNC) program UDRH.

CCSI Residential Immediate Training Survey Memo

Synopsis: This memo provides analyses of the immediate survey responses collected through paper surveys, registration data, and Audience Response Systems (ARS) from six CCSI residential trainings on the 2012 IECC code held from January through April of 2016. The CCSI contractor held four trainings on Envelope and Building Science (EBS) on January 13, February 4, March 31, and April 5. Two trainings on HVAC and Indoor Air Quality (HVAC-IAQ) were held on March 10 and April 28. Out of an estimated 144 training attendees, 116 attendees filled out paper survey forms.

Conclusion: Of the survey respondents, 31 were building code officials and the remaining 85 fell into the general category of builders, architects, contractors, equipment suppliers, and others. The memo presents detailed results related to the following topics: Usefulness Ratings for Envelope Building Science Training Components, Whether the Envelope Building Science Training Components Were New, Usefulness Ratings for HVAC Indoor Air Quality Training Components, Whether the HVAC Indoor Air Quality Training Components Were New, Quality of Trainings, ARS Ratings of Training Quality, Housing Units Permitted under 2012 IECC for January through April 2016 Trainees, Housing Units Permitted under 2012 IECC for 2014 through 2016 Trainees, When Expect a Final Inspection on a 2012 IECC Unit, When Expect to First Use Training Information, Most Important New Information Provided by the Trainings (January through April 2016), How Information Provided by the Trainings Will Be Used, Additional Comments and Suggestions to Improve Trainings, Training Attendees, Years in Present Position for January through April 2016 Trainees, Proportion of Retrofit Building Permits, Cities and Towns Represented in the January through April Trainings, Numbers of Unique Training Attendees.

Follow-up Interviews with CCSI Residential Training Attendees

Synopsis: As part of the ongoing evaluation of the Massachusetts Code Compliance Support Initiative (CCSI), evaluators conducted follow-up in-depth interviews (IDIs) with 60 individuals who had attended one or more residential classroom trainings approximately six months earlier. Thirty respondents work as municipal building code employees and 30 work as builders, architects, equipment suppliers, or energy efficiency professionals (referred to as ‘builders and others’). The overall goal of the follow-up interviews was to determine if and how the subjects are using what they learned at the trainings in the field; the interviews also explored how information from the trainings is shared, the changing environment for code compliance and enforcement, and suggestions for improving the trainings.

Conclusion: Nearly three-fourths of respondents had shared some of the information from the trainings with other parties. Close to one-half of the municipal building code employees said that the most useful part of the trainings were related to insulation and envelope areas. The most common reasons for not making any changes to fieldwork after attending the trainings were already knowing the information and working in a stretch code community. Nearly three-fourths of respondents had shared some of the information from the trainings with other parties. Just over one-half of the respondents said they had attended one or more trainings or gatherings discussing building codes since attended the CCSI training. Most builders and others reported increased interest in energy efficiency among both code officials (67 percent) and their customers (80 percent) during the past year. The most frequent suggestion offered by all respondents was for the CCSI to try to get more people to attend the trainings, especially builders (from municipal building code employees) and contractors (from builders and others). More general suggestions for improving code compliance include offering field assistance at construction sites and educating homeowners about the new code through information accessible by the public. All but two of the 60 respondents reported that they would encourage their colleagues to attend the CCSI trainings. More detailed findings are discussed in the report.

Ductless Mini Split Heat Pump Impact Evaluation

Synopsis: This report presents results of an in situ evaluation of ductless mini‐split heat pumps (DMSHPs). The evaluation team initially planned to study 132 Massachusetts homes that participated in the COOL SMART Program. The PAs, however, extended the scope of work to include 20 Rhode Island homes that participated in the High Efficiency Heating and Cooling Rebate Program. The evaluation sought to address many utility and consumer questions about DMSHPs, focusing on power and energy consumption, heat output, efficiency, and interactions with existing HVAC equipment.

Conclusion: The report presents findings related to operating hours, equivalent full load hours, savings, and cold climate performance as well as a discussion of cooling, heating, COP/SEER/HSPF, savings values, and controls and zoning. Evaluators also provide the following recommendations: The evaluation team recommends exploring ways to improve the PAs’ existing lost opportunity program for DMSHPs, such as how best to encourage the installation of multiple DMSHP heads to better match existing zones and displace primary system operation. The study still finds that a modest level of savings are achievable by moving from a standard efficiency DMSHP to a higher efficiency DMSHP. Substantially more savings could be achieved (i.e., the top 25% of savings) if newly installed DMSHPs are operated more regularly and continuously by better matching and integrating them zonally with primary heating systems, through better configuration design and installation and contractor and customer education and training. The evaluation team also recommends exploring methods for targeting homes with electric resistance heating for DMSHP retrofits, targeting propane‐heated homes for DMSHP, exploring methods for addressing oil‐heated home, and examining opportunities for a new construction measure to substitute DMSHPs for central air conditioners. Other future studies could explore the use of interfaces between learning thermostats and ductless systems.

Residential Lighting Hours-of-Use Update

Synopsis: The 2014 Northeast Residential Lighting Hours-of-Use (HOU) Study was designed to allow sponsors in the Northeast to update HOU estimates based on changes in room-by-room saturation collected as part of regular saturation studies. This memo serves as an update for the Program Administrators in Massachusetts for the upstream lighting programs. This update is not applicable to the direct-install programs which operate using a somewhat different HOU estimate. The sources of data for this memo include: the Northeast Lighting Hours-of-Use Study, results of the Massachusetts On-site Lighting Inventory (2013), and the 2015-16 Lighting Market Assessment Consumer Survey and On-site Saturation Study.

Conclusion: The memo provides an overview of the average number of sockets and snapback-adjusted HOU by room; socket saturation for CFLs, LEDs, and all energy-efficient bulbs (CFLs, LEDs, and linear fluorescents) between 2013 and 2016; and changes in saturation. As the data show, with the exception of kitchens, all room types show net gains in energy-efficient bulbs. For kitchens, the gains in LED saturation were offset by losses in CFL and linear fluorescent (not shown) saturation. This is important because kitchens have the second highest HOU after exteriors. To estimate updated HOU, evaluators calculated the proportion of bulbs in each room by bulb type (separately for CFLs and LEDs). The memo provides the results of these calculations for each bulb category type as well as the snapback-adjusted HOU by room and the resulting household HOU estimate. Conclusions drawn by the evaluators suggest that stimated HOU for CFLs should be revised downward (by about 3%) for any retrospective (such as the 2016 Annual Report) or prospective CFL savings calculations. However, because the PAs plan to end supporting CFLs as part of the upstream program in 2017, there may be no prospective CFL savings calculations. Estimated HOU for LEDs should be revised upward (by about 3%) for any retrospective (such as the 2016 Annual Report) and prospective LED savings calculations. However, there should be consideration of the likelihood that LEDs will continue to replace CFLs and may reduce the HOU over time. LED installations should continue to be monitored for potential future shifts. Evaluators do not recommend any changes for direct-install lighting programs. For these programs, the PAs should continue to use the all-bulb HOU estimates provided in the HOU report. For the household level, this estimate is 2.7. However, whenever possible the PAs should track installed bulbs by room type and apply the room-specific HOU estimates.

Multifamily Program Gas and Electric Impact Study

Synopsis: The Mass Save® Multifamily Retrofit Program is a direct installation program that promotes both electric and gas energy efficiency measures. The goal of this study is to provide 2013 program level (Residential Multifamily) realization rates for both electric and gas overall and provide disaggregated results where appropriate. Evaluators used a two-stage, premise-level, difference-of-differences modelling approach for energy consumption analysis using a dataset combining consumption, weather, and participation and other premise and customer-specific characteristics information. This approach estimates gross energy savings and relies on a comparison group consisting of future participants to control for non-program related change. The data in this study relies nearly exclusively on data directly from National Grid and was accompanied by stages of review to ensure the representation and adequacy of the data to support the billing analysis.

Conclusion: The Multifamily Program is producing electric savings, though much less than estimated in the tracking system. The evaluators’ estimate of savings from the 2013 program year is 3,559 MWh with a precision of ±49.3 at the 90% confidence interval. This result provides a realization rate of 24.4%. Evaluators found no substantial difference in results when analyzed at the premise level or when examined by facility size (number of premises in each facility) or housing type (apartment vs condominium), though they found participants with commercial billing rates experienced greater savings in absolute terms as well as percent of consumption than participants with residential rates. The final estimate of program level natural gas impacts is 383,129 Therms, which does not include interactive effects. This result is accompanied by a realization rate of 86.2% and a precision of ±64.1% at 90% confidence interval. The tracking savings estimate for commercial bill rates had a considerably higher realization rate (127%) and savings as a percent of consumption (12.4%) than savings estimated for residential (38% and 1.6%, respectively). Evaluators also found results vary when facilities examined by ownership type, though not as significantly, with apartment and condominium realization rates of 108.8% and 68.5%, respectively.

Massachusetts Behavioral Programs Impact Evaluation

Synopsis: This report summarizes the impact analyses conducted in 2015 of National Grid’s and Eversource Energy’s Home Energy Report (HER) programs. At the end of 2014, there were a total of thirty-one evaluable cohorts in these Program Administrator’s HER programs. This included fifteen electric cohorts, eight gas cohorts, and four dual fuel cohorts. The evaluation team conducted three distinct impact analyses related to these HER programs: cohort-specific impact analysis, mapping analysis, and dual treatment analysis.

Conclusion: The cohort-specific analysis indicated that total net electric savings from the Massachusetts HER programs were 127,855 MWh, and total net gas savings were 643,157 MMBtu. The evaluation team also presents recommended savings estimate ratios for the PAs to adopt in future years when third-party impact evaluations are not completed. From the mapping analysis, the evaluation team concluded that the overlap between different program cohorts is generally small and is not expected to impact cohort-specific savings analysis using standard evaluation methods. The dual analysis suggest that on the electric side, dual-fuel customers saved less than dual-treatment customers at a statistically significant level. On the gas side, the difference was not statistically significant. Cross-PA customers (a subset of dual treatment customers who receive electric reports from one PA and gas reports from another) were satisfied with the frequency at which they currently receive reports and they found the reports just as useful as dual-fuel customers. The evaluation team does not believe that coordination across the PAs is warranted and recommends that the PAs continue implementing the HER program in its current form.

MA 2015-16 Lighting Market Assessment Final Report

Synopsis: This report presents the results of the Massachusetts 2015-16 Residential Lighting Market Assessment, which was designed to update estimates of lighting saturation and other critical market indicators, and to assess consumers’ knowledge of, and interactions with, the lighting market in Massachusetts. The data for this study came from web and telephone surveys and on-site lighting inventories conducted with random samples of homes in Massachusetts and a comparison area completed between December 2015 and March 2016.

Conclusion: While consumers are adopting LEDs in the non-program comparison area, evidence from this study suggests that the Massachusetts programs have had a strong impact on saturation and penetration of energy-efficient bulbs. LED saturation and penetration rates in the comparison area (New York) continued to lag significantly behind the rates measured in Massachusetts. Further, ENERGY STAR® LEDs accounted for the entirety of the difference in LED saturation between the two states. Finally, as the potential energy savings analysis in this report demonstrates, there are substantial savings yet to be realized in the Massachusetts residential lighting market. By 2016, CFL saturation in Massachusetts was significantly higher compared to New York, closely aligned with NYSERDA’s decision to cease standard spiral CFL incentives in 2012 and essentially end all upstream lighting incentives in 2014. In contrast, faced with similar questions about the future of residential lighting, Massachusetts elected to phase out support for standard CFLs in a controlled manner. Recent changes to the ENERGY STAR specifications that preclude CFLs, and expectations that the price for ENERGY STAR LEDs will continue to decrease, mean that the role of CFLs in the upstream program is coming to an end. The report presents other key findings as well as recommendations, considerations, and guidance for future study.

MA Lighting Interactive Effects Results Memo

Synopsis: This memo details the findings of the Lighting Interactive Effects study conducted in order to better understand and report the true impact of energy efficient lighting retrofits. The goal of this study was to determine a statewide average for the heating, ventilation, and air conditioning (HVAC) impacts of residential lighting measures, quantified using interactive effects (IE) factors. To accomplish this, the Residential Evaluation Team developed and calibrated hourly building energy simulation models for single family, low-rise multifamily and high-rise multifamily structures. The team weighted the model results using socket saturations for each building type and Massachusetts-specific housing stock data to arrive at one statewide average.

Conclusion: Table 1 in the memo presents average interactive effects for the Residential Upstream Lighting Initiative in Massachusetts. These values may be updated later in 2016 or early in 2017 as improved housing characteristic information becomes available through the forthcoming residential baseline study.

Ductless Mini-Split Heat Pump Draft Cooling Season Results Memorandum

Synopsis: The Massachusetts and Rhode Island Program Administrators (PAs) commissioned an evaluation team to conduct an in situ study of ductless mini‐split heat pumps (DMSHPs) in Massachusetts and Rhode Island.The PAs and the evaluation team identified several research questions related to the performance, efficiency, and energy savings resulting from the installation of DMSHPs through Massachusetts’ COOL SMART (and through Rhode Island’s High Efficiency Heating and Cooling) Program. This memo addresses the first two questions, which ask: 1. How much energy is being saved with an average DMSHP installation? 2. How does the in situ performance correlate with DMSHP rated cooling capacity, rated SEER, rated EER, and ambient conditions? The final report will address all research questions.

Conclusion: The design and manufacture of DHSHPs is evolving quickly, and much has changed since these units were installed in 2013 and 2014. More makers are offering DMSHPs. Cooling SEERs as high as 33 are advertised, with most makers advertising units approaching SEER 30. These higher SEER systems will likely offer even greater cooling savings than those delivered by the population in this study (which averaged about 22 SEER). Compared to the metered sample and to previous population averages, the average SEER increased in 2015. Overall savings, however declined slightly because the average Tier 2 DMSHP system size was much smaller than the Tier 1 DMSHPs. The savings found in this study are generally aligned with the TRM‐derived values but are lower because the evaluators’ findings show that the metered DMSHP units operate at fewer full load cooling hours (265 actual hours, 259 hours used for savings) than the deemed TRM value (360 hours). Though the weather differences between MA and RI likely result in runtime differences, the variability due to other factors (e.g. user operation) is much higher. The final report will include state‐specific results if we find statistically significant differences in runtime after normalizing by population‐weighted weather data. The final report will also include analytical observations of statistical significance which may help decision makers further refine program design and marketing.

Evaluation of Cape Light Compact’s Creating Awareness for Power Efficiency Initiative

Synopsis: This report presents the results of a process and impact evaluation of Cape Light Compact’s (CLC) Creating Awareness for Power Efficiency (CAPE) behavior feedback initiative. The evaluation covers the period from program launch in August 2014 through December, 2015. The evaluation team conducted a process evaluation consisting of in-depth interviews with 27 customers of varying participation-levels (never installed the equipment, installed the equipment but lost connectivity, installed the equipment and maintained connectivity). In addition, the evaluation team conducted an impact evaluation using econometric analysis to estimate savings.

Conclusion: Customers’ experiences with the CAPE initiative illustrate many of the common benefits and challenges of energy use feedback programs related to equipment installation, participant engagement, and participant action. Results suggest that equipment installation was the primary barrier to participation in the CAPE initiative. Customers who participated in the CAPE initiative expressed high levels of interest in monitoring energy use and motivation to save energy. However, among customers with access to their energy data, monitoring decreased over time with more than half of participants monitoring their energy use once per week or less. Few CAPE participants committed to energy saving actions as a result of their energy use monitoring. Although the savings estimates for the CAPE initiative were not statistically significant, it is clear from the confidence bounds that the savings are not as high as the savings from the SHEMP Legacy program but they may be similar to the SHEMP Energize program.

Final 2015 Report Lighting Market Adoption Model Weighting Scheme Implications Memorandum

Lighting Market Adoption Model Spreadsheet

Synopsis: This memorandum provides a comparison of the weights in the July 30, 2015 Draft Market Adoption Model and in the March 2, 2016 Final Market Adoption Model. It also summarizes the impact of the weighting change on gross delta watts assumptions.

Conclusion: The most substantial change in the weighting scheme is the introduction of unique weights for CFLs and LEDS. Related to this, under the new scheme LED sales tend to be more heavily weighted towards 40W and 60W bins while CFLs are more heavily weighted to the 100W and 60W bins. In terms of gross delta watts assumptions, CFL delta watts all increase by a very small amount while LEDs exhibit a reduction in delta watts. This is because the new weighting scheme moves more CFLs into the 100W category and LEDs into the 40W and 60W categories.

Reducing the Size of the Control Group in the Home Energy Report Program Memorandum

Synopsis: This memorandum addresses the evaluation team’s research and calculations to inform Massachusetts Program Administrators (PAs) and the Energy Efficiency Advisory Council (EEAC) of opportunities to reduce the size of control groups in the Home Energy Report (HER) program. Power analysis results present the PAs and EEAC with optimal numbers of control group customers which can be removed from each HER cohort and assigned to a new cohort, while taking into consideration the statistical confidence of resulting savings estimation, sensitivity analysis around potential deviations from expected savings values, and other key considerations.

Conclusion: Key findings suggest that National Grid has the ability to reduce the control group size in six of the eleven HER program cohorts analyzed, resulting in over 100,000 new treatment customers. All four of the Eversource Energy (formerly NSTAR) HER program cohorts analyzed were found to be reducible, allowing nearly 50,000 new customers to be transitioned into a new treatment group.

Residential Single-Family Building Department Document Review

Synopsis: In conjunction with the ongoing single-family compliance/baseline study, evaluators visited 52 building departments throughout Massachusetts to determine what type of documentation is being filed to show compliance with the energy code for single-family homes.

Conclusion: Evaluators determined that energy-related code requirements are not consistently being documented at building departments. Very few homes had documentation of all energy-related code requirements. Much variation exists between and within towns in terms of energy-related code requirement documentation. Duct leakage and air leakage documentation is lacking, particularly in non-stretch code municipalities. Code officials are not using REScheck checklists. There is very little documentation that Manual J calculations are being conducted. Stretch code homes rarely have documentation filed showing that the ENERGY STAR homes thermal enclosure checklist was completed.

Berkshire Gas Home Energy Report Process and Impact Evaluation

Synopsis: This report present the results of the first‐year process and impact evaluation of Berkshire Gas’ Home Energy Report (HER) program. The primary objective of the program is to provide residential households with information on their gas consumption and tips on how to save energy to prompt them to take action to reduce their natural gas usage. The HER program, implemented by Opower, was launched in October 2014, and has delivered reports to nearly 12,000 customers during its first program year.

Conclusion: Overall, participants are satisfied with the HER reports. Over half of participants (59%) classify the reports as useful, similar to surveys of other Massachusetts gas HER programs. Participants are most satisfied with the personal usage comparison. Total program savings were 7,603 MMBtu, which is 0.7 MMBtu annually per person or 0.49%. This absolute savings value is relatively low given the high baseline usage compared to the first‐year evaluated savings of other gas cohorts in Massachusetts. In percentage terms, savings are the lowest among the MA cohorts. The lower than expected evaluated savings are supported by Opower’s reported savings, and the customer surveys conducted reveal that fewer Berkshire participants report that the reports led to energy‐saving behaviors.

LED Incremental Cost Study

Synopsis: This report presents the methods and findings from a study to understand the incremental bulb purchase cost of light-emitting diodes (LEDs) relative to compact fluorescent lamps (CFLs) and halogen bulbs through 2018 (with additional extrapolations through 2020 included in Appendix A). The three data sources used for various extrapolations were 1) in-depth interviews with suppliers and high-level retail buyers (supplier/retailer interviews), 2) a regression assessment based on historical point-of sale (POS) pricing data, and 3) an exponential regression analysis based on web-scraping data. Preliminary results of the supplier interviews and web-scraping exercises also informed the separate Lighting Net-to-Gross Study, specifically the development of prospective NTG ratios for 2016 to 2018.

Conclusion: All three incremental cost methods predict that LED prices will continue to decline over time. The magnitude of decline varies by method; interviews and POS suggest a 30% decline in estimated prices between 2015 and 2018, and web-scraping estimates a 50% decline in price relative to 2015 by 2018. There was also a large spread in the estimate of starting LED prices ranging from $11.29 to $7.25 (in 2015). The variance in price decline might be attributed to the different distribution channels and associated data that are available within each incremental cost method. The web-scraping approach draws published bulb prices from home improvement and hardware retail channels because grocery stores and similar channels do not offer online retail options. Conversely, the majority of bulbs considered in the POS model were those offered in grocery and drug stores because most big box channels are not includes in the POS data. These factors considered together mean that the data do not provide clear guidance on selecting a single incremental cost for the program years 2016 to 2018. Recommendation and considerations are also included.

2015

Cool Smart DMSHP Heating Results Memo

Synopsis: As part of a large impact metering study, the COOL SMART Impact Evaluation Team sought to determine heating season savings and full load hours associated with DMSHPs installed through the COOL SMART program. The team metered approximately 150 participating systems during the winter of 2014-2015 and extrapolated their usage to a more typical weather year.

Conclusion: As shown in the separate DMSHP baseline memo, the most common heating baseline for DMSHPs was a code-minimum DMSHP. Starting in 2016, all DMSHP units sold in the United States must meet a minimum heating season performance factor (HSPF) of 8.2, up from 7.7. The team calculated equivalent full load hours consistent with a HSPF-based savings calculation and savings against an 8.2 HSPF baseline, shown in a table. While the team found 447 equivalent full load hours and 210 kWh savings on average, the group of participants who purchased systems for both heating and cooling (65% of the total) had 20% higher usage (as measured in annual full load hours) and savings. The very small sample of customers who purchased for heating only shows even higher usage (as measured in annual full load hours) and savings.

Residential Customer Profile Study

Synopsis: The Residential Customer Profile Study provides a summary of energy efficiency program participation, gross savings, and incentive spending for the Residential and Low-Income sectors across all Program Administrators (PAs) in Massachusetts. This report presents findings by sector and program for calendar year 2013. Results are based upon an analysis of program records that have been merged with U.S. Census Bureau data and demographic data from the American Community Survey at a U.S. Census Block Group level.

Conclusion: The study present program participation, gross savings, and incentive payments at the sector, program, and PA levels. Maps and other geographic analysis of program participation, savings, and incentive spending are presented at the U.S. Census Block Group level. All savings reported are for the primary fuel savings—the electric and gas savings that are directly targeted by programs—and do not include secondary fuel savings or savings in oil, propane, or wood.

Saturation Comparison of MA, CA, NY

Synopsis: The purpose of this research was to compare the saturation estimates for MA, CA, and NY, with an eye toward the circumstances and strategies that may have been most responsible for boosting efficient bulb saturation by 8% in CA from 2009 to 2012 – a time period during which MA and NY saw a saturation plateau. The Team attempted to achieve this by interviewing individuals heavily involved in the design, implementation, and evaluation of the CA lighting program, as well as reviewing the evaluation and saturation calculation methodologies from CA and NY to examine any differences from MA.

Conclusion: The literature review of the evaluation methodologies and saturation comparisons between CA, NY, and MA revealed only one substantial difference in how saturation was estimated: CA evaluations did not include empty sockets in the saturation calculation, while MA did. However, evaluators determined that the saturation differences between the states were actual differences and did not result from differences in methods, and continued with further research into why saturation rates diverged. Key findings from the study suggest that earlier implementation of EISA and the CA programmatic move toward grocery, discount, and drug stores were likely to have been the most influential factors in CA’s saturation gains from 2009 to 2012. Furthermore, given the higher initial saturation in MA than CA (i.e., more older CFLs), CFL-to-CFL replacement rates in MA were also likely to have been higher from 2009 to 2012 than in CA, which could explain saturation stagnation over that period. Concurrent research in MA and varying comparison states both with and without programs sheds more light on these relationships. It also makes clear that legislation and programmatic changes are not the only factors influencing increases in efficient bulb saturation.

Residential Lighting Shelf Survey and Pricing Analysis

Synopsis: This study presents findings from a shelf stocking and price survey that targeted two types of stores: those that are currently participating in the program (participant stores) and those stores that participated in the program in a previous year but are no longer participating (former‐participant stores). Evaluators analyzed lighting inventory data from a shelf survey to assess: CFL prices in the state relative to competing lighting products; the amount of shelf area dedicated to CFLs; and the pricing, number of bulb packages, and shelf location of CFLs and LEDs relative to otherlighting types. Evaluators also compared the results of the current survey to those of similar surveys conducted in 2010 and 2012.

Conclusion: Evaluators identified several key findings from the study. Participant stores continue to provide a wider variety, more inventory, and better prices of CFLs and LEDs (including discounts) than former‐participant stores. Participant stores also offered more packages with multiple CFLs than did the former‐participant stores. The majority of LEDs offered in participant stores are discounted through the program. CFL inventory appeared to be losing ground in 2013, compared to 2012, for both participant and former‐participant stores, although the differences are not statistically significant. The proportion of shelf space devoted to CFLs among participant stores has increased since 2010. Inventory shares of 75W and 100W incandescent packages have dropped from 8% in 2012 to 3% in 2013 for participant stores. 60W shares stayed relatively constant as EISA efficiency standards do not affect these bulbs until January 2014. Former‐participant stores appear to have replaced CFL inventory primarily with incandescent bulbs. Average CFL prices for A‐line bulbs in participant stores are almost double those of corresponding halogen bulbs. Even though the EISA phase‐out date had started for each, both 100W and 75W incandescent bulbs are widely available, with 57% of participating stores still stocking 100W incandescents and 63% stocking 75W incandescents. The participant stores stock a wider variety of CFL bulb types and wattages than the former‐participant stores. Overall, average discounted prices are less than non‐discounted prices; however, the average PA incentive is larger than that difference, indicating that the full PA discount may not be passed through to consumers.

Cool Smart Baseline Memo

Synopsis: This memorandum summarizes research regarding the appropriate baselines for estimating gross energy impacts from ductless mini‐split heat pumps (DMSHP) promoted through the electric PA’s COOL SMART program. The challenge with establishing installed baselines for DMSHPs is that there is not a clear standard efficiency technology that is the lone alternative to a DMSHP. The determination of appropriate baselines is based on information gathered from program participants during on‐site visits to install meters measuring the energy usage of the DMSHPs and other heating and cooling equipment found in each home.

Conclusion: The evaluation team presents results based on two scenarios that reflect two ways of interpreting the baseline that should be used for customers who said they would have left the space unconditioned. There is likely some confusion on the part of respondents around what constitutes a conditioned or unconditioned space. The evaluation team’s proposed baselines for determining gross DMSHP impacts for each scenario are summarized in tables.

Multistage Lighting Net to Gross Assessment Overall Report

Synopsis: The present Massachusetts (MA) program cycle included several studies with different research approaches that estimated net‐to‐gross (NTG) or net‐of‐freeridership. The purpose of this report is to describe those approaches briefly and to explain the consensus building process undertaken to develop and finalize NTG estimates, both retrospectively for 2014, and prospectively for 2016‐2018 (as required by the MA Department of Public Utilities). A group consisting of Program Administrators (PAs), Energy Efficiency Advisory Council (EEAC) consultants, and evaluation consultants reviewed the CFL and LED NTG and net‐of‐freeridership estimates derived for 2013 and/or 2014 and on‐going evaluation research regarding prices for lighting currently and in the near‐term future, ultimately working together to achieve agreement on the recommended NTG estimates.

Conclusion: The report presents the consensus retrospective (2014) and prospective (2016 to 2018) NTG values for the most critical products. The discussions involved in developing these recommendations are described in detail in the main body of the report. In all cases but one, the consensus values ended up being the average of the NTG values recommended by the group members for each bulb type and program category. The exception was non‐ hard‐to‐reach (HTR) LEDs, in which the final consensus values of 90% for 2016, 80% for 2017 and 70% for 2018—agreed upon after considerable discussion over multiple meetings and emails— were slightly lower than the simple average of group members’ recommendations (91%, 84%, and 77%, respectively).

Lighting Market Assessment and Saturation Stagnation Overall Report

Synopsis: Recent changes to the Massachusetts Residential Lighting Program have included providing incentives, and other program support for general service Light Emitting Diode (LEDs) bulbs and expanding program activity in retail channels and specific store locations thought to serve “hard‐to‐reach” (HTR) consumers. At the same time, the PAs continued providing substantial support for both standard spiral and specialty Compact Fluorescent Lamps (CFLs) in what could be called the more traditional lighting channels (e.g., home improvement and hardware stores). This report summarizes results from the following three studies that explored market reactions to lighting program interventions: 2014 Market Assessment Study; 2015 Market Assessment Study; and 2014 Saturation Stagnation Study.

Conclusion: All three studies confirm what evaluators have found in over 15 years of evaluation activities: the Massachusetts Residential Lighting Program and its predecessors have had a clear and dramatic impact on the residential lighting market. Over these 15 years, evaluators have observed a continuous trend of positive program performance including: increased LED penetration and saturation; nearly ubiquitous CFL penetration and increased saturation, especially among low‐income households; increased LED market share, with Massachusetts’ share being even higher than many other program states; steady CFL market share, even after the introduction of general service halogen bulbs, due at least in part to Massachusetts’ continued program support for CFLs; and decreased average wattage per bulb for all bulbs between 2013 and 2015. Still, while the program has had a clear impact on the market, the PAs must look to the future and consider the potentially evolving impact that LEDs and other lighting technologies, coupled with legislation that increases lighting efficiency standards, will have on the residential lighting market moving forward. In particular, the Energy Independence and Security Act (EISA) of 2007 impacts the availability of incandescents and the Energy Policy Act (EPAct) of 2005, as implemented through the General Service Fluorescent Lamp (GSFL) Rulemaking, affects linear fluorescent tubes, common in residences in Massachusetts. Consumers’ price sensitivity and overall lighting knowledge (awareness of EISA as well as different bulb types and performance) will continue to play an important role in their adoption of efficient lighting. Future planning should consider the implications of these multiple factors, while remaining forward‐looking and keeping pace with new developments in the market.

Cool Smart Incremental Cost Study

Synopsis: This incremental cost study estimates how manufacturing production costs (MPCs) and purchase prices of residential air conditioning (AC) and heat pump (HP) equipment change as equipment efficiency increases. The results of this study will support Cool Smart program enhancements and cost‐effectiveness analysis, as well as potential upstream residential upstream heating, ventilation and air conditioning (HVAC) incentive programs. The study methodology included examining rebate records, modeling costs and prices, and plotting the efficiency frontier.

Conclusion: The incremental prices presented in the cost‐efficiency plots represent the price increase above the least expensive baseline system in each category, at 13.0 SEER for ACs and 14.0 SEER & 8.2 HSPF for HPs. The Methodology and Results sections of this report provide a detailed breakdown of these results, as well as cost‐efficiency charts based on energy efficiency ratio (EER) and heating seasonal performance factor (HSPF) ratings.

HES and HEAT Loan Program Assessment Final Report

Synopsis: The purpose of the assessment was to: determine the relative strengths and weaknesses of the Home Energy Services (HES) initiative by the lead vendor and home performance contractor delivery channels, with the intent to improve the overall initiative; assess the effectiveness of the current overlap in the HES, HEHE, and COOL SMART programs and the HEAT Loan; and identify any improvements that could increase the number of participants installing the recommended energy‐efficient improvements. The evaluation team calculated 20 key performance indicators (KPIs) for this study and used four years of tracking data to calculate each KPI by delivery channel. The evaluation team supplemented its quantitative KPI analysis with qualitative in‐depth interviews and surveyed 965 HES initiative participants to understand their perspective, in particular their experience and satisfaction with the HES initiative and HEAT Loan.

Conclusion: Evaluators drew several conclusions from the study. HPCs and lead vendors offer complementary approaches, each with their own benefits and drawbacks. Providing services through both channels is likely helping to expand the reach of the HES initiative. HPC customers have lower rates of cross‐program participation than lead vendor customers, which may be a result of lack of HPC understanding regarding program offerings and less proactive promotion of non‐HES initiative offerings. The majority of customers find that the Energy Specialist’s recommendations for making energy‐efficient upgrades in their homes are easy to understand; a smaller majority feel certain about how to navigate Mass Save offerings and incentives. Customers who participated through the lead vendor delivery channel indicated higher overall satisfaction with the HES initiative and found that the lead vendor recommendations were easier to understand; however, all customers reported being at least somewhat satisfied with the initiative regardless of delivery channel. The HEAT Loan has been successful in promoting deeper and broader savings, acting as a motivating factor for customers and a business tool for contractors. Although HEAT Loan awareness is relatively high among HES participants, opportunities remain for increasing HEAT Loan awareness among both HES participants and nonparticipants. Recommendations for improvement are also included.

HES and HEAT Loan Appendix A

Efficient Neighborhoods & Incremental Cost Assessment

Synopsis: This memorandum presents the results of the incremental cost analysis of the Efficient Neighborhoods+® (EN+) initiative. The results presented in this memo cover communities targeted during the first round of the EN+ initiative. Evaluators gathered incremental cost data through a series of data requests and follow-up discussions with Program Administrators (PAs) and implementation contractors.

Conclusion: Combined, the PAs spent an estimated additional $429,790 to administer the EN+ initiative relative to the standard HES program. Marketing and administrative costs represented the largest portion (84% combined) of the incremental costs. Total incremental costs vary considerably by PA because of the differing number of communities targeted by each PA, the scope of the marketing efforts, as well as the differing numbers of targeted customers and resulting participation levels.

Behavior Programs Process Evaluation

Synopsis: This report presents the results of a process evaluation for the Massachusetts Home Energy Report (HER) programs offered through Eversource Energy and National Grid in 2014. The HER program is implemented by Opower, and was first piloted in 2009 by National Grid. In this process evaluation, evaluators sought to verify whether customers are still satisfied with the HER and interested in continuing to receive then reports. To achieve this objective, a telephone survey was administered to both program treatment and control group populations.

Conclusion: Results of the study suggest that overall, customers remain satisfied with the HER reports and the frequency of treatment. The survey results also suggest that the HER programs are enhancing customer satisfaction among electric treatment groups. The HER program is one among many ways customers are prompted to save energy in Massachusetts through various education and information initiatives, yet there is a clear lift in energy‐saving actions among treatment customers above their control counterparts. The survey results also suggest that program measure life may extend beyond one year. Although most energy saving actions appear to be independent of other programs (i.e. received no financial incentives) gas customers showed greater cross‐program participation. Self‐reported data suggests that cross‐fuel effects may be occurring. Cross Program Administrator customers are satisfied with the frequency of their reports, and do not differ significantly from dual fuel customers in their views on report usefulness. In‐depth interviews suggest customers could be better educated on the impact of various end uses on their household consumption. Interviews also suggest that customers may benefit from smarter home appliances and automation. Among automation technologies, Wi‐Fi thermostats are of great interest to some, but many customers are skeptical that this technology will save more energy than their existing programmable thermostats. Recommendations for improvement are also included in the study.

Residential Point of Sale Modeling – Final Report

Synopsis: This report summarizes the findings of the Massachusetts Point‐of‐Sale (POS) modeling research. The research examines the influence of all the input variables—but most importantly program activity—in order to determine whether residential lighting programs have an important impact on CFL and LED sales. The purpose of the POS modeling research was to inform two major studies in the MA 2013 Residential cycle: the Saturation Stagnation investigation and the Multistage Lighting Net‐to‐Gross (NTG) research. The modeling lends itself to the former by investigating potential reasons for the lack of gains in efficient bulb saturation in MA between 2009 and 2012, and to the latter by providing MA‐specific NTG estimates for CFLs and LEDs (and both bulb types combined) by leveraging nationwide sales data.

Conclusion: Program activity, as defined by program budget measures, showed a consistently positive and significant relationship to the proportion of CFLs and LEDs sold across all available states and retail channels. The research also sheds light on the confounding finding in prior MA saturation studies that increases in CFL sales are not necessarily accompanied by increases in CFL saturation. It may be that saturation gains take longer to materialize than previously expected. Being able to observe this relationship, however, relies on the ability to control for intervening factors for which prior methods and exploratory analyses were unable to account.

Preliminary Lighting Demand Elasticity Findings Memo

Synopsis: This memo presents preliminary net‐of‐freeridership findings from a lighting demand elasticity analysis. The evaluators developed a model using program data on prices, the number of lamps purchased, and lamp and retailer characteristics. To determine the net‐to‐gross ratio for the upstream lighting program, the Evaluation Team employed multiple approaches. These approaches included supplier interviews with lighting product manufacturers and retailers, Point‐of‐Sale data modeling, saturation and comparison area analysis, and demand elasticity modeling.

Conclusion: Evaluators were unable to develop reliable LED elasticity estimates from a representative sample of products given potential stocking issues, missing months, negative sales, and long, irregular reporting periods that occurred among the two retailers that accounted for the majority of LED sales with varied prices. However, there were sufficient variation in price and sales data to estimate price elasticities and predict freeridership for standard and specialty CFLs. Estimated CFL freeridership was 44% for specialty CFLs and 61% for standard CFLs. The memo also describes the limitations of the available data and opportunities to improve the data for subsequent analyses.

Upstream Lighting Net-to-Gross Estimates Using Supplier Self-Report Methodology

Synopsis: This report presents estimates of net‐to‐gross (NTG) ratios for CFL and LED bulbs sold through the Massachusetts ENERGY STAR upstream lighting program in 2013 using supplier self‐report methodology. Evaluators calculated separate NTG estimates for bulbs sold through stores characterized as hard‐to‐reach (HTR) and through stores not characterized as such. HTR‐designated stores included the vast majority of stores making up the “discount” channel (such as dollar stores), ethnic grocery stores, and a few home improvement stores targeting low‐income citizens.

Conclusion: The report presents NTG ratios estimated for each bulb type sold through the Upstream Lighting Program. While evaluators aggregated to the bulb level and store type, this report also shows bulb‐type NTG ratios broken down by retail channel. The report compares NTG ratios provided by the 2013 program participants with those reported in previous program years, using the same supplier self‐report methodology. While the 2009–2010 and 2013 estimates included NTG estimates from all three market actor levels (e.g., manufacturers, retail buyers, and store managers), the 2011–2012 estimates only included estimates from store managers. Results suggest much higher NTG ratios for the 2013 program year than for the 2009–2010 program year—the last program year basing NTG ratios on estimates from all three levels of the lighting supply chain.

Supplier & Retailer Perspectives on Residential Lighting Market – Summary of Year 2014 Interviews Final Report

Synopsis: This report summarizes findings from retailer and supplier interviews conducted in 2014. The interviews supported the comprehensive assessment and monitoring of the Massachusetts lighting market and the Massachusetts ENERGY STAR® Lighting Program. Findings are intended to help develop a better understanding of the recent trends, new opportunities, and lingering barriers in the Massachusetts residential lighting market. The findings also serve to gauge lighting market actors’ satisfaction with this program and to identify opportunities for program improvements. [/wptab] [wptab name=’Conclusion’] This report presents findings from lighting manufacturer and retailer interviews. Topics covered include impacts of EISA legislation, LED market assessment, hard to reach customers, program effects on CFL and LED sales, and program satisfaction.

Efficient Bulb Saturation Comparison of Massachusetts, California and New York – Final Report

Synopsis: The purpose of this research was to compare the saturation estimates for MA, CA, and NY, with a focus on the circumstances and strategies that may have been most responsible for boosting efficient bulb saturation by 8% in CA from 2009 to 2012. During this time period, MA and NY saw a saturation plateau. To answer this question, the evaluators interviewed individuals heavily involved in the design, implementation, and evaluation of the CA lighting program, as well as reviewed the evaluation and saturation calculation methodologies from CA and NY to examine any differences from MA.

Conclusion: Results of the study suggest that the earlier implementation of EISA in CA had an important impact on the move toward CFLs in that state, which was reflected in the saturation gains CA experienced relative to MA from 2009 to 2012. In addition, the CA program’s move away from big box stores and toward grocery, drug, dollar, and club channels also likely promoted saturation increases. CFLto‐CFL replacement rates in MA, which had higher CFL saturation than CA in 2009, also likely played a role in the saturation stagnation over that period. Finally, although the saturation point estimates of MA and CA had differed in 2012 (27% and 30%, respectively), when taking the confidence intervals into account, MA rates overlapped with those of CA, suggesting that while CA did demonstrate greater saturation gains over the 2009 to 2012 period, the actual saturation rates in 2012 between the two states were comparable. Other explanations that the Team considered were not supported by the research findings.

On-Site Lighting Inventory – Final Results

Synopsis: This report presents the findings of research related to the market for energy‐efficient light bulbs in Massachusetts. The objective of the study was to track various lighting market indicators—many of which have been measured in Massachusetts for over a decade—and to explore the data for possible impacts (particularly stockpiling of incandescent bulbs) associated with the Energy Independence and Security Act of 2007 (EISA). Specifically, this report presents an analysis of over four hundred residential lighting on‐site inventories collected in Massachusetts, Georgia, and Kansas in 2014. This report also compares findings from 2014 to those from previous on‐site studies conducted in Massachusetts in 2009, 2010, 2012, and 2013 and in Kansas and Georgia in 2009.

Conclusion: The report presents detailed findings from the on-site lighting inventories. The report also offers recommendations and considerations based on these findings. Evaluators recommend repeating the panel study and expanding on the panel size, adding 2014 saturation study participants. Considerations include: revising the retail program to include an inefficient bulb buy‐back program; considering program designs that address less efficient linear fluorescent tubes including T12s; undertaking more focused efforts on this lumen range when selecting and determining incentive levels for LEDs and CFLs to offer through upstream programs; revising the program design to encourage the purchase of multiple LEDs in one shopping trip; directly studying use and purchase behavior among hard‐to‐reach households.

High Efficiency Heating Equipment Impact Evaluation Final Report

Synopsis: The Massachusetts High-Efficiency Heating Equipment Rebate Program (HEHE) offers prescriptive rebates of up to $1,600 for the installation of new high-efficiency natural gas heating and water heating equipment. The objective of this evaluation was to determine gross energy savings for gas furnaces and boilers installed through the HEHE program, and refine the estimates of baseline efficiency and heating consumption. The field portion of the study included spot measurement of baseline and new equipment in situ efficiency as well as long-term metering of post-retrofit high efficiency equipment.

Conclusion: This evaluation provides revised savings estimates for high-efficiency furnace and boiler replacements as well as several program implications and conclusions. Key findings suggest: there are differences in annual heating loads between equipment types, though previous deemed savings used the same annual heating load for both furnaces and boilers; it is important to consider standby and cycling losses in addition to combustion efficiency when evaluating gravity-drafted equipment such as standard and early retirement boilers and furnaces; high-efficiency boilers are not being installed to maximize potential savings; many older gas furnaces and boilers considered “early retirement” equipment have AFUEs of at least 75 percent, even when considering actual instead of rated performance; evaluation research suggests that as many as 80 percent of new combination systems are replacing boilers with indirect water heaters, but the TRM currently assumes a boiler and a standalone water heater as the baseline.

Mass Save Mutlifamily Program Process Evaluation Report

Synopsis: This report presents the results of a process evaluation of the Mass Save Multifamily Building Program. The evaluation sought to assess and monitor the program’s evolution as an integrated offering since the last round of program evaluation was conducted; examine barriers to participation, the effectiveness of program operations, and customer experience; and review PA and vendor tracking data to assess whether these data would sufficiently support a planned future impact evaluation. Evaluation tasks largely included in-depth interviews, focus groups, and surveys that included a range of multifamily stakeholders and program experts. Evaluators also completed on-site visits to assess the comprehensiveness of the energy assessments and the measures installed as well as an extensive comparison of available program participation and energy assessment data.

Conclusion: The report presents the four key recommendations most critical for the PAs and EEAC to focus on in order to improve the program and achieve deeper and broader savings. First, the PAs and EEAC should consider creating a role for a single point of contact for each project. The program tracking systems should also be improved to allow for a holistic view of multifamily properties across PAs, fuels and programs. Improvement of the energy assessment process can be achieved through the training of auditors in completing a comprehensive job including a review of all the systems in common areas and major systems within in unit areas. Lastly, the evaluators believe a billing analysis is a feasible approach to determining savings among participating accounts.

Residential Lighting Consumer Survery – Final Report

Synopsis: The research presented in this report includes the results of a multi‐mode residential lighting consumer survey performed between March 6, 2014 and May 5, 2014. The goal of the consumer survey was to track key indicators of the market for CFLs, LEDs, and halogens as well as the impacts of EISA on the residential lighting market. The report provides comparisons with three similar previous lighting consumer surveys (Winter 2011, Summer 2012, and Winter 2012) where appropriate. In addition, results from comparison area surveys in Georgia and Kansas are included to provide a parallel between consumers’ lighting awareness and bulb use in Massachusetts and two other areas with little or no residential lighting program activity.

Conclusion: Evaluators drew several conclusions based on the results of the study. The Spring 2014 survey findings indicate that satisfaction with LEDs was higher than that of CFLs; particularly among users of both CFLs and LEDs, satisfaction with LEDs relative to CFLs was even more prominent. Although a majority of CFL users remains satisfied with CFLs, the indicator continues the downward trend observed since 2011. Results of the 2014 survey suggest a significant increase over previous waves in the proportion of households reporting familiarity with screw‐in LEDs. While it is likely that PA efforts to raise awareness about LEDs and provide incentives for them has played an important role in increased LED adoption, future studies would be needed to confirm this assumption. This survey differed from previous ones by offering some respondents the opportunity to answer via a web‐based instrument and sending some respondents a pre‐paid $5 incentive. The pre‐paid $5 incentive doubled the response rate. Also, households that actually completed the survey via the web differed from those that completed by phone demographically (more likely to own homes, have higher education, and, among those that reported their income, to have higher incomes) and in terms of their awareness, use, and satisfaction with energy‐efficient lighting. Additional recommendations and considerations are also included in the report.

Tube TV Recycling Program Assessment Memo

Synopsis: The goal of this study was to assess the feasibility of expanding the MassSave Appliance Rebate program to offer rebates for recycling operable but old, inefficient tube televisions (CRT‐TVs). the Evaluation Team collected television data from 150 Massachusetts homes in conjunction with the lighting market assessment study. The team also collected additional self‐reported usage information such as the average hours of daily use, estimated purchase date, and the willingness of the customer to replace the unit if offered with free removal for secondary TVs and a $50 voucher for primary TVs.

Conclusion: The Evaluation Team found that out of the 370 TVs recorded, 20% were CRT‐TVs and targets for replacement. The remaining 80% consisted of LCD/LED, plasma, or rear projection TVs (all considered efficient). While there was a reasonably high saturation of CRT‐TVs, research indicated that the remaining CRT‐TVs were underutilized, small, and being naturally replaced by users. For these reasons, the Evaluation Team does not recommend expanding the existing recycling program.

Behavior Program Impact Evaluation Memo

Synopsis: This memo presents results for the impact evaluation of the Home Energy Report (HER) programs for the Massachusetts Cross‐Cutting Research in Behavior and Education. In particular, this memo presents results for National Grid and Eversource Energy. The evaluation team conducted multiple analyses to determine cohort‐specific savings, including post‐program regression and linear fixed effects regression models. The evaluation also include channeling analysis to document the lift in other program participation due to the behavioral program treatment (participant lift). It also removed the savings co‐generated by behavioral and standard programs in order to avoid double counting savings across the portfolio at the program and measure level (savings adjustment). The evaluation team calculated savings estimate ratios for each of the HER program cohorts to examine the differences in savings as measured by the program implementer and the savings verified by the evaluation.

Conclusion: Total net electric savings from the Massachusetts HER programs are 127,854,643 kWh. Total net gas savings are 643,157 MMBtu. The evaluation team also provides recommended savings estimate ratios for the PAs to use in future years when third‐party impact evaluations are not completed.

2014

Home Energy Report Savings Decay Analysis Final Report

Synopsis: This report presents findings from an evaluation of the National Grid Home Energy Report reduced treatment experiment. The goal of this study was to determine what effect reduced treatment had on program savings – measured as the difference in program savings between participants for whom treatment is reduced, compared to those who continued to receive standard treatment. The study included three main activities: data request and review, implementation contractor interviews, and billing analysis.

Conclusion: “The study resulted in five key findings related to the treatment reduction experiment: 1. A reduction in treatment led to a decline in observed savings for both cohorts. The magnitude of the savings reduction likely depends on the reduced-treatment design, fuel type, and the duration of program intervention before the reduction. 2. The electric and gas cohorts responded differently to the reduction in treatment. Compared to the electric cohort, reduced treatment customers in the gas cohort seem to show a much sharper reduction in savings in response to the reduction in reports they experienced. 3. The electric and gas cohorts experienced a decay in savings at different times relative to the reduction in treatment. For electric, the decline did not set in immediately, whereas for gas, the decline was more immediate and precipitous. 4. Additional research in this area could help inform the design of treatment reduction strategies. We recommend that future experiments plan the timing of treatment reductions to further test the potential impact of the following factors: treatment duration prior to the experiment, seasonality of the reduction, duration of the reduction, and fuel-specific differences. Implications of these findings are also discussed.”

Market Lift Assessment Final Report

Synopsis: “This report summarizes the findings of an evaluation of the Massachusetts (MA) Market Lift Initiative, a new program approach in MA geared toward increasing sales of compact fluorescent lamps (CFLs) by rewarding participating retailers for increasing sales over a pre‐established baseline. In addition, evaluators explored how suppliers felt about Market Lift approaches, and Cadmus performed Quality Control by reviewing the report.”

Conclusion: The net lift achieved for MA stores for the 13W CFL bulbs over the entire lift period was 8,405 bulbs (15.3%), and the net lift achieved for 23W CFLs over the entire lift period was 2,705 bulbs (8%). The net lift achieved for MA stores for both 13W and 23W CFLs over the entire lift period was 11,110 bulbs (or 12.5% over pre‐lift period). Net Market Lift results are also presented under an alternative seasonal research design using a six‐month pre‐lift period. This approach decreases the net lift from 12.5% to 3.2% for the combined 13W and 23W CFLs. The decline is largely due to the effects of seasonality on purchasing. There was substantial disapproval toward Market Lift voiced by manufacturers and retailer buyers (as well as the claim from the one manufacturer who had participated that they would not do so in the future), calling into question the viability of the approach moving forward. While it does make sense that retailers and manufacturers would favor the standard upstream approach, in which incentives are guaranteed, the right Market Lift structure could still be attractive to at least some potential partners.

Northeast Residential Lighting Hours-of-Use Study Final Report

Synopsis: “The purpose of this study was to provide updated information to the Connecticut Energy Efficiency Board, the Massachusetts Program Administrators (Cape Light Compact, National Grid Massachusetts, Northeast Utilities, and Unitil), National Grid Rhode Island, and the New York State Energy Research and Development Authority to assist in the calculations of demand and energy savings for lighting programs. Specifically, this report presents load shapes, coincidence factors (CFs), and daily hours of use (HOU). Based on data collected from 4,462 loggers, the evaluators performed a series of regression models to estimate HOU. “

Conclusion: Evaluators concluded that the region comprising Connecticut, Massachusetts, Rhode Island, and Upstate New York had a household daily HOU of 2.7 hours for all bulbs and 3.0 for efficient bulbs, with HOU by room type varying from a low 1.7 in bathrooms to a high of 6.7 on the exterior of homes. Hours of use for Downstate New York exceeded those for the other areas included in the study, with a daily HOU of 4.1 for all bulbs and 5.2 for efficient bulbs for the household; room-specific estimates varied from 3.2 for bathrooms to 7.7 for kitchens. The evaluators also provide detailed HOU estimates by room type, home type (i.e., single-family or multifamily), and income level for the region overall and for each individual area included in the analysis. Additionally, the report presents load shapes as well as well as coincidence factors for winter and summer peak period and winter and summer peak hours to aid in load planning and the calculation of peak demand savings. Other topics addressed include comparisons of HOU for efficient and inefficient bulb types and comparisons to other existing HOU studies both in the Northeast region and throughout the United States.

Residential Lighting Shelf Survey & Pricing Analysis Final Report

Synopsis: Evaluators conducted a shelf stocking and price survey for stores currently participating in the e Massachusetts ENERGY STAR® Lighting program and those stores that participated in the program in a previous year but are no longer participating. Evaluators analyzed lighting inventory data from a shelf survey to assess CFL prices in the state relative to competing lighting products, the amount of shelf area dedicated to CFLs, the pricing, number of bulb packages, and shelf location of CFLs and LEDs relative to other lighting types. Evaluators also compared the results of the current survey to those of similar surveys conducted in 2010and 2012.

Conclusion: Participant stores continue to provide a wider variety, more inventory, and better prices of CFLs and LEDs (including discounts) than former‐participant stores. The majority of LEDs offered in participant stores are discounted through the program. CFL inventory appears to be losing ground in 2013, compared to 2012, for both participant and former‐participant stores, although the differences are not statistically significant. The proportion of shelf space devoted to CFLs among participant stores has increased since 2010. Inventory shares of 75W and 100W incandescent packages have dropped from 8% in 2012 to 3% in 2013 for participant stores. Shares of 60W bulbs stayed relatively constant as EISA efficiency standards do not affect these bulbs until January 2014. Former‐participant stores appear to have replaced CFL inventory primarily with incandescent bulbs. Average CFL prices for A‐line bulbs in participant stores are almost double those of corresponding halogen bulbs. Even though the EISA phase‐out date had started for each, both 100W and 75W incandescent bulbs are widely available, with 57% of participating stores still stocking 100W incandescents and 63% stocking 75W incandescents. The participant stores stock a wider variety of CFL bulb types and wattages than the former‐participant stores. Overall, average discounted prices are less than non‐discounted prices; however, the average PA incentive is larger than that difference, indicating that the full PA discount may not be passed through to consumers.

Single Family Non-Program Homes Code Compliance Results

Synopsis: Evaluators conducted on-site inspections at 150 recently constructed single-family non-program homes—fifty homes built at the end of the 2006 IECC cycle and 100 homes built at the beginning of the 2009 IECC cycle. The goal of this evaluation is to compare compliance rates between these two samples using a comparable approach for calculating compliance. The study used code compliance checklists developed by Pacific Northwest National Laboratory (PNNL) to develop compliance scores the homes. The results are representative of the average house-level compliance scores using the PNNL checklists. The checklist allows compliance to be assessed depending on which compliance approach the builder used: the prescriptive approach, the UA trade-off approach, or the performance approach.

Conclusion: Homes built at the end of the 2006 IECC cycle show significantly higher overall compliance scores (76%) than homes built at the beginning of the 2009 IECC cycle (63%). These results are not particularly surprising, given that significant changes took place when the 2009 IECC was implemented and the homes in the 2006 IECC sample were built at the end of the code cycle, while the homes in the 2009 IECC sample were built at the beginning of the cycle. One possible reason that homes built at the end of the 2006 IECC cycle show higher compliance is that builders and subcontractors had three years to familiarize themselves with the code before constructing these homes while builders and subcontractors building homes at the start of the 2009 ECC cycle were likely just getting familiar with the new requirements. The 2009 IECC sample homes are actually slightly more efficient that the 2006 IECC sample homes, even though the 2009 IECC homes display lower compliance with the applicable energy code. This finding supports the hypothesis that low compliance among the 2009 IECC homes is likely due to the fact that new requirements were implemented with the new energy code and builders were still familiarizing themselves with those requirements, resulting in lower compliance.

Ductless Min-Split Heat Pump Customer Survey Results

Synopsis: The Massachusetts COOL SMART program offers rebates to customers for installing qualified, high‐efficiency ductless mini‐split heat pump (DMSHP) equipment in their homes. A web survey was conducted of participants in this program to better understand motivations for participating and how the DMSHP equipment is being used. The final survey included responses from 430 participants.

Conclusion: The report identified several key findings for the program. DMSHPs are being installed as retrofit projects. The majority of participants are installing DMSHPs to obtain an improved level of comfort instead of a focus on saving energy. DMSHPs are more often installed for cooling than heating. Also, DMSHPs are being used to supplement, rather than replace existing heating systems.

Efficient Neighborhoods Plus Initiative Evaluation Final Report

Synopsis: “During 2013, the Massachusetts-based energy efficiency program administrators (PAs) conducted an initiative designed to engage hard-to-reach customers with an enhanced version of the Home Energy Services (“HES”) program. This report presents the findings of process and impact evaluations of the initiative. The initiative included three distinct components collectively referred to as the Efficient Neighborhoods+® (EN+®) initiative. These include the Core initiative, Cape Light Compact’s (CLC’s) EN+, and Fall River Neighborhood Energy Contest. The ultimate goal of the evaluation was to understand whether the three initiatives resulted in a lift in participation and energy savings. In addition, the evaluation sought to provide PAs with insights on the successful components of the initiatives and recommendations on possible improvements.

Conclusion: “Overall, the EN+ Core initiative was successful at increasing participation and savings in the target communities among lower to moderate income customers but less so in reaching rental properties. The Fall River Energy Challenge also resulted in increased participation and energy savings, even without increased incentives. The true impacts of CLC’s EN+ initiative are unknown. The process evaluation findings suggest that despite a variety of marketing and outreach efforts, awareness and knowledge can impede participation. Sources of awareness varied but overall results suggest that in-person outreach or phone calls are effective outreach strategies. Word-of-mouth marketing can also be an effective marketing strategy. Exposure to the initiative through more than one outreach method is an effective way of moving customers toward participation and costs appear to be the core barrier to participation. Lack of interest and perceived lack of need for improvements despite recommendations are additional barriers to participation. Pre-weatherization barriers exist, yet participant knowledge about the initiative offerings to mitigate those barriers is somewhat limited. Recommendations for improvement are also provided.

Residential New Construction Net Impacts Report

Synopsis: This study was designed to estimate the net impacts that may be attributed to the Massachusetts Residential New Construction Program for single-family homes. The study used a multi-step methodology consisting of a builder survey, a Delphi study involving a panel of experts in energy-efficient new construction, modeling of home energy usage under the counterfactual assumption that the Program had not existed from 2004 to 2011, and comparing the as-built modeled energy usage to the estimate from the counterfactual models. This approach was developed to account for as many factors as possible that might have influenced the adoption of energy-efficient construction practices.

Conclusion: The study found substantial net savings for the single-family component of the Massachusetts Residential New Construction Program. The Delphi panel estimated that homes completed in 2011 would have been much less efficient in the absense of the program. Combining a free ridership of 0.53 and non- participant spillover of 1.39 gives the program a net-to-gross ratio of 1.87 when assessing savings using the fuel neutral metric of MMBtu. Non-program homes are responsible for 75% of net savings in terms of MMBtu, 68% of electric savings, and 71% of natural gas savings. The Delphi panelists noted that the program has had a particularly strong effect on air infiltration, duct leakage, lighting, insulation installation grades, and some heating system efficiencies. When assessing net savings using the fuel-neutral metric of MMBtu, natural gas is the fuel with the most net savings, followed by propane and then electricity. Lighting is responsible for 61.5% of all electric net savings.

Residential Lighting Controls Initiative Evaluation Final Report

Synopsis: Evaluators conducted an impact evaluation and a limited process evaluation of Cape Light Compact’s (CLC) Residential Lighting Controls Initiative. This report presents Cadmus’ findings and conclusions, which are drawn from pre- and post-installation metering, lab testing of dimming controls to determine lamp energy use at various dimmer control levels, and participant surveys.

Conclusion: Verified savings values are presented based on eight of the sixteen participating sites. Total gross demand impacts were 2.11 kW and the average impact per site was 0.26 kW. Total gross energy impacts were 1,987 kwh/yr and the average impact per site was 248 kWh/yr. Feedback from interviews with the CLC manager and RISE staff substantiated that this technology is challenging to implement as a program at this time. Compatibility of lighting controls with efficient lamp technologies was an issue and participants, the CLC manager, and RISE staff all had difficulty finding combinations of dimmers and bulbs that worked. Widescale efforts to replace lamps controlled by dimmer switches will require further research to identify working combinations of controls and efficient lamps. Survey and metering data revealed that some participants used their lights more after installation of the new dimmers and bulbs; however, substantial savings can still result since any savings are largely driven by the lamp replacement. Evaluators suggest that in order for an impact evaluation of lighting use with dimmer replacements to be successful, it is important to have detailed tracking of implementation activities.

2013

Year 2012 Home Energy Services Pre-Weatherization Initiative Evaluation Final Report

Synopsis: This report summarizes an evaluation of the 2012 Home Energy Services (HES) Pre-Weatherization Initiative (initiative) that was offered by the electric and gas program administrators (PAs) of Massachusetts. The Evaluation Team assessed the effectiveness providing customers with an additional incentive to help reduce their financial burden for addressing common pre-weatherization repairs that are required prior to installing certain HES measures, as well as on assessing the delivery of the initiative itself. The Team conducted interviews with HES PA program managers and lead vendors, analyzed initiative participation data through July 2012, and surveyed participant and non-participant customers. They also followed-up on inquiries with PAs, conducted surveys with participants and non-participants from all PAs, and completed additional analysis of both complete initiative data and historic HES data at the end of September 2012 and in March 2013.

Conclusion: Evaluators drew several conclusions based on evaluation results. Evaluation results did not show a significant change in the measure adoption rate for National Grid and NSTAR customers who faced the knob and tube wiring barrier. While the turnkey option offers customers easy access to approved contractors, the PAs and lead vendors that offered the turnkey option were uncertain of the delivery option’s long-term viability. Non-participants indicated having confusion about what the initiative actually covered for knob and tube wiring. Stakeholders and customers that were subjected to the 30-day requirement indicated that additional time would have helped. However, an analysis of acceptance rates revealed that customers who were given a 30-day deadline had higher acceptance rates than those offered the 90-day deadline. Also, two of the three 30-day PAs (NSTAR and National Grid) offered additional delivery channels (HPC and turnkey) that were not available to some of the customers subjected to the 90-day timeline. Interviews with PAs and lead vendors indicate that elements of the initiative’s design and delivery varied across PAs. The evaluation also provides recommendations for initiative improvements.

Energy Star Lighting Program Early Impacts of EISA

Synopsis: This report summarizes the results of four different research efforts meant, in part, to identify the current and likely effects of EISA on the residential lighting market in Massachusetts. The research activities occurred from December 2011 through April 2013 and were completed for the Energy Efficiency Advisory Council Consultants and the Massachusetts Program Administrators (PAs).

Conclusion: This report provides key findings and recommendations related to consumer and store awareness of EISA, changes in bulb availability, changes in bulb purchases and use, stockpiling of incandescent bulbs, consumer understanding of key lighting concepts, and 60-Watt incandescent phase out. Some key findings are summarized below. Most consumers remained unaware of the increased lighting efficiency standards resulting from EISA and the subsequent phase-out of most general-service incandescent bulbs. Stores that participate in the program carry 23% more packages of CFLs and 4% more packages of LEDs than do stores that formerly participated. One-half of consumers shopping for 100-Watt incandescent bulbs could still find them during the last three months of 2012. Households in Massachusetts did not exhibit substantial changes in their light bulb use between early 2012 and early 2013. When asked what type of bulb they would chose to replace 100-Watt incandescent bulbs, most consumers selected a lower wattage incandescent bulb followed by a CFL. The research found little evidence of widespread, EISA-induced stockpiling of incandescent bulbs. Sixty-Watt incandescent bulbs fill 22% of all sockets in Massachusetts homes, making this the most popular bulb in use in homes. More detailed findings from each of the individual evaluation research activities are provided in the report.

Onsite Lighting Inventory – Results Final Report

Synopsis: This report presents the findings of a lighting inventory conducted to understand use, saturation, and purchases of lighting products in Massachusetts households in support of the Massachusetts ENERGY STAR® Lighting Program (the Program). The study also sought to understand conditions during the implementation of the Energy Independence and Security Act (EISA) of 2007 and search for possible impacts on lighting use and purchase behavior that may be the result of the new lighting standards. To conduct this research, the Team performed 150 onsite lighting inventories in Massachusetts households between December 2012 and March 2013. The onsite respondents were recruited through a prior telephone survey among 600 households in Massachusetts also conducted in Winter 2012 and early-2013.

Conclusion: This report provides key findings and recommendations related to CFL use and storage, socket saturations, saturation potential for CFLs and LEDs; bulb storage; CFL and LED purchases; tracking CFLs over time; additional factors that influence CLF use, saturation, and purchase; and Federal lighting standards. Some key findings are described below. The full report presents these and other findings in more detail. Based on the onsite analysis, the Team concludes that most households in Massachusetts use CFLs. Despite high rates of penetration, the number of CFLs in use and the percentage of sockets in which they are installed appears to have leveled over the past three years, and there is evidence that recently purchased CFLs are largely being used to replace installed CFLs that have burned out. Between 2009 and 2010, statistically significant gains were made in increasing the number of specialty CFLs in homes, but this increase was not repeated between 2010 and 2013. LEDs remain an emerging technology in Massachusetts, with very few homes using any LEDs bulbs. When considering the most energy-efficient bulbs types – CFLs, LEDs, and fluorescent tubes – saturation currently stands at around 40%. Use of incandescent bulbs has decreased, but this trend started well before the implementation of the first phase of EISA. Evidence of incandescent stockpiling was mixed and inconclusive.

Lighting Retailer, Supplier Perspectives on the Massachusetts Energy Star Lighting Program Final Report

Synopsis: This draft report presents findings from retailer and supplier interviews conducted in support of the evaluation of the 2011-2012 Massachusetts ENERGY STAR® Lighting Program.

Conclusion: This report provides key findings and recommendations related to the impact of EISA, the market for LED Bulbs, program activity in the hard-to-reach lighting markets, program satisfaction, and program attribution. Some key findings are described below. The full report presents these and other findings in more detail. Store manager awareness of the EISA legislation has increased significantly since 2010. Fifty-five percent of EISA-aware store managers who said that the EISA legislation had impacted their stocking practices said that their stocks of incandescent bulbs had decreased. Two-thirds of the store managers who changed their stocking practices in response to EISA said their stocking of standard CFLs had increased. A slight majority of store managers who reported EISA-related impacts said that their sales and stocks of LED bulbs had increased. There was some disagreement among lighting manufacturers as to how well the EISA-compliant halogen bulbs were selling. According to the program tracking data, only 41 of the 240 participating stores in our sample (17%) had sold LED products through the program. There were many reasons why store managers were not selling LEDs, the most cited reason being that the bulbs are too expensive. Retailer recommendations for greater program access in the HTR sectors include doing more marketing and reducing prices. Seventy-one percent of store manager respondents were satisfied with the program as a whole.

Residential Lighting Shelf Survey and Pricing Analysis Final Report

Synopsis: This report describes research to assess the effect of the Mass Save® residential lighting program on CFL prices in the state relative to competing lighting products, the amount of shelf area dedicated to CFLs for participating retailers, the pricing, the number of packages of bulbs; and the shelf location of CFLs and LEDs relative to other lighting types in participating and former participating stores. Results of the current survey were compared to results from a similar survey conducted in 2010. To perform this analysis, the Evaluation Team designed and analyzed a shelf-stocking survey and a hedonic pricing regression analysis.

Conclusion: After controlling for differences in wattage, sales channel, manufacturer, and other characteristics between discounted and undiscounted standard CFLs, the impact of the PA discount reduced the register price of program-discounted CFLs by an average of $1.76. The point estimate of $1.76 is 86% of the amount of the PA incentive. This compares to an average impact of $1.46 in 2010, which was 117% of the PA incentive amount in that year. Thus, in 2012, it appears that partners are not passing the full PA incentive onto consumers. The PA incentive was associated with a decrease in A-bulb, reflector, and globe CFLs bulb prices ranging from $0.96 to $2.15. This price reduction was less than what was observed in 2010 when the price decrease ranged from $2.33 to $3.61. Stores selling PA-discounted CFLs tend to have a higher percentage of CFL packages than former participating stores. In participating stores, the percentage of shelf space allocated to CFLs increased across most distribution channels from 2010 to 2012.

2012 Residential Heating, Water Heating, and Cooling Equipment Evaluation Net-to-Gross, Market Effects, and Equipment Replacement Timing

Synopsis: This report summarizes net-to-gross (NTG) values, the timing of equipment replacement (ER), and the net market effects (NME) for the Cool Smart and High-Efficiency Heating and Water Heating Equipment (HEHE) programs for the 2010-2012 period. The Team based evaluation findings, conclusions, and recommendations on results from an array of data collection activities and evaluation tasks, and report NTG, ER and NME results at the measure level.

Conclusion: The measures responsible for the majority of savings due to equipment installations in the HEHE and Cool Smart programs are central HVAC systems: gas boilers, gas furnaces, CAC, and heat pumps. The estimates from both QIV contractors and participants suggest that significant shares of participants are replacing equipment early. A large proportion of customers replacing boilers and furnaces early are also switching from oil to gas. Policy implications of these findings are further discussed in the report. A range of NTG estimates were developed. All of the average NTG estimates are approximately 0.75 or greater with the exception of ductless mini-splits (0.62). Overall, these NTG estimates are supported by distributor NME analysis.

Residential New Construction Program Incremental Cost Final Report

Synopsis: This report provides estimates of the incremental costs per square foot involved in building high efficiency homes that meet the criteria of the 2013 MA Residential New Construction (RNC) Program. Incremental costs above the costs of typical homes being built outside the program are estimated for single family (SF), low-rise multifamily buildings of three or fewer stories (MF 1-3), and mid- to high-rise multifamily buildings of four stories or more (MF 4+) for each of the incentive options offered by the program. The 2013 MA RNC Program offers five incentive options for the SF and MF 1-3 building sectors – two based on prescriptive measures and three performance options based on percent savings – and three incentive options for the MF 4+ building sector.

Conclusion: Estimated incremental costs vary among the incentive options, particularly for homes in the SF attached building sector. The variation is due to both the kinds of measures that were implemented to satisfy the option requirements and the degree to which the measures were enhanced. Incremental cost per square foot for single family attached homes range from $1.03 to $5.61; multifamily with no master meter range from $0.10 to $1.50; multifamily with master meter gas range from $0.08 to $1.48; and MF 103 overall ranged from $0.60 to $3.10.

Winter 2012 Consumer Survey Results Final Report

Synopsis: The research presented in this report compares the results of a telephone survey performed between December 4, 2012, and January 21, 2013, with the results of two similar surveys performed between December 8, 2011 and January 19, 2012, and June 18, 2012 and August 2, 2012, respectively. The Winter 2011 survey sought to establish a baseline at the onset of the changes in lighting standards resulting from the Energy Independence and Security Act of 2007 (EISA) while the mid- and Winter 2012 surveys searched for possible changes in the lighting market since the initial implementation of EISA. When possible, the Team also compares the results to those obtained from lighting surveys conducted in 2009 and 2010.

Conclusion: The Winter 2012 survey provides evidence that customer satisfaction with CFLs remains steady. A persistent concern with CFLs in Winter 2012 relates to their mercury content and disposal issues. Results of the Winter 2012 survey suggest that use, knowledge, and opinions of CFLs differed between those using many CFLs (experts) and those using few or none of the bulbs (novices). A subset of Winter 2012 survey respondents explained that they wanted to prefer LEDs as their lighting source, but were hesitant to spend $20 for a bulb. When the 75-Watt incandescent is no longer available, there was an increase in consumers who indicated they planed to move to a higher watt incandescent. Respondents to all three survey waves reported relatively low levels of familiarity with A-line CFLs, although there was a slight increase in familiarity with the bulb in Winter 2012 . The results presented for Winter 2012 show that one-fifth of respondents will consider stockpiling incandescent bulbs due to of the 75-Watt phase-out. Consumers are becoming more familiar with the term “lumens” and understand that it means light output or brightness, but they still buy bulbs based on wattage or wattage equivalence. The full report presents these and other findings in more detail.

2012

Results of the Massachusetts Onsite Compact Fluorescent Lamp Surveys

Synopsis: This report presents the findings of research conducted to understand use, saturation, and purchases of lighting products in Massachusetts households in support of the ENERGY STAR® Lighting Program. The study also sought to understand baseline conditions early in the implementation of the Energy Independence and Security Act (EISA) of 2007 and search for possible impacts on lighting use and purchase behavior that may be the result of the new lighting standards. To conduct this research, the team performed 150 onsite lighting inventories in Massachusetts households between January and April 2012. The onsite respondents were recruited through a prior telephone survey among 582 households in Massachusetts in 2011.

Conclusion: Although phone survey results suggest that CFL usage has declined, onsite inventories suggest that most households in Massachusetts use CFLs. Despite high rates of penetration, the number of CFLs in use and the percentage of sockets in which they are installed have leveled off over the past two years and there is evidence that recently purchased CFLs are largely being used to replace installed CFLs that have burned out. Between 2009 and 2010, the number of specialty CFLs in homes increased, but the same results did not occur between 2010 and 2012. LED bulbs remain an emerging technology and few homes in MA use them. Use of incandescent bulbs has also decreased, although some stockpiling is occurring.

Massachusetts Residential Retail Products: Consumer Electronics Saturation

Synopsis: This report presents the findings of research conducted in Massachusetts to examine the use of selected consumer electronics and appliances and to explore related plug load concerns. The report focuses on the types of consumer electronics customers use, the characteristics of those products, and the configurations of products that customers use together. It also examines the use of power strips, both conventional and smart strips. Finally, it presents information on the use and characteristics of refrigerators, freezers, clothes washers, clothes dryers, dishwashers, and water heaters. The findings are largely based on the results of visits to 150 households in Massachusetts conducted between February and April 2012; however, the report also summarize responses to questions about consumer electronics asked in an earlier consumer survey.

Conclusion: The results of the saturation study strongly suggest that program potential exists to reduce plug loads by targeting television and home office equipment. The market for consumer electronics is large, diverse, and changing rapidly. Consumers buy electronics at a faster rate than many other ENERGY STAR qualified products (with lighting being an exception), which means consumers have numerous opportunities to choose between more and less efficient models. The analysis demonstrates that 70% of homes have clothes dryers, and most of these are heated with electricity, which are almost always paired with a clothes washer. It is likely that potential exists for a heat pump dryer program when they become available in the US market. Based on the homes included in this sample, some of the barriers that exist in securing a greater number of HPWH installations include access to a condensate drain and having a large enough space into which the larger HPWH could fit.

Massachusetts Consumer Survey Results 2012

Synopsis: This report compares the results of a telephone survey performed between June 18, 2012 and August 2, 2012, with the results of a similar survey performed between December 8, 2011 and January 19, 2012. The 2011 survey sought to establish a baseline at the onset of the changes in lighting standards resulting from the Energy Independence and Security Act of 2007 (EISA) while the 2012 survey searched for possible changes in the lighting market since the initial implementation of EISA.

Conclusion: The 2011 survey provided evidence that customer satisfaction with CFLs was declining, while the onsite study demonstrated that those dissatisfied with CFLs used fewer of them – even though they still typically used at least some CFLs. The current study, in contrast, found that CFL satisfaction was higher in 2012 than in 2011 but remained below 2009 and 2010 levels. Respondents to both the 2011 and 2012 study reported relatively low levels of familiarity with A-line CFLs, and use of these bulbs is still uncommon. In the 2011 survey results, NMR noted that users of dimmable CFLs generally liked them, but the most persistent complaint is that they do not dim consistently or as much as other types of light bulbs. The results presented for both the 2011 and 2012 surveys as well as the onsite saturation effort suggested that about one-fourth of respondents will consider stockpiling incandescent bulbs, and that some respondents have already started to do so. While consumers are becoming more familiar with the term “lumens” and understand that it means light output or brightness, they still buy bulbs based on wattage or wattage equivalence. Although telephone survey respondents reported relatively high rates of familiarity with A-line halogen bulbs, the Team found very few of these bulbs in use in onsite households.

Massachusetts Consumer Electronics Potential Qualitative Research Study

Synopsis: The Massachusetts program administrators commissioned this study in order to (1) identify current factors affecting the energy efficiency of consumer electronics products and related savings opportunities, (2) better understand the current state of the market for these products, and (3) assess opportunities for the Massachusetts ENERGY STAR Appliances and Products Program (“the Program”) to address through program activities. A secondary goal of the study was to identify regulatory approaches to improving the energy efficiency of consumer electronics at the federal level, in California, and in other nations with an eye to how these can inform Massachusetts program planning. The literature review for this study was conducted in April and May, 2012. In-depth interviews were conducted between June and August 2012.

Conclusion: Findings from this research suggest that, while a products-focused program approach may be appropriate for refrigerators, freezers, room air conditioners and cleaners, as well as pool pumps, it may not be appropriate for consumer electronics products. Observations from the literature and interviews suggest 4 options to consider as alternatives to the current approach of the consumer electronics program: 1. Abandoning program support for consumer electronics. 2. Limiting support for consumer electronics to encouraging the development of more rigorous energy efficiency specifications and standards. 3. Changing the focus for the consumer electronics portion of the Program from individual products to overall reduction of miscellaneous plug load energy use intensity (EUI) through market transformation efforts, including but not limited to consumer education and behavior change. 4. Maintaining the current product-focused program, but with modifications to improve effectiveness. Based on an analysis of these options, the report provides several conclusions and recommendations for program consideration. Product-Specific conclusions and related recommendations are also provided.

2012 Home Energy Services Pre-Weatherization Initiative Interim Evaluation Findings Memo

Synopsis: This report summarizes preliminary findings of Phase 1 of the evaluation for the Pre-Weatherization Initiative as part of the Home Energy Services Program. The initiative offers an additional incentive to HES customers who are facing barriers to installing certain assessment-recommended weatherization measures. Phase 1 tasks consisted of: interviews with PA and lead vendor HES program managers; analysis of initiative participation data available to date; and surveys with participant and non-participant NSTAR, National Grid, and Berkshire Gas customers. The Evaluation Team assessed initiative perspectives, assumptions, and processes to determine whether the initiative remains on track to meet its goal of encouraging customers to move forward with weatherization measures by offering additional incentives to remove pre-weatherization barriers.

Conclusion: Based on Phase 1 results, the evaluators drew several initial conclusions. Feedback from interviewed PAs and lead vendors indicated that the 30-day timeframe may be too short. Incentive amounts may be appropriate for clearing barriers from improper dryer venting, combustion safety issues, and identifying whether knob and tube wiring in a home is live. However, the initiative incentives are not enough to cover the cost of rewiring a home if needed. During PA and lead vendor interviews, issues arose about whether a turnkey option made sense. To determine the turnkey option’s success, the Evaluation Team will include questions in Phase 2 surveys to determine acceptance and completion rates of customers utilizing turnkey services, compared to those finding their own contractors.

Home Energy Services Impact Evaluation Report: Part of the Massachusetts 2011 Residential Retrofit and Low Income Program Area Evaluation

Synopsis: This report summarizes the gross impact evaluation findings of the 2010 Home Energy Services (HES) Program. The Evaluation Team performed an array of evaluation tasks to inform the reported estimates of per-unit gross savings for each HES measure, including billing analysis, engineering analyses, and calibrated simulation modeling.

Conclusion: The report presents evaluated gross savings estimates for all HES measures, covering electric, natural gas, and oil fuel types. The results are grouped by measure type and primary heating fuel type, although some measures have savings for more than one fuel type. Results are primarily provided in a series of tables.

Wi-Fi Programmable Controllable Thermostat Pilot Program Evaluation: Part of the Massachusetts 2011 Residential Retrofit & Low Income Program Area Study

Synopsis: This report presents the impact and process evaluation of the 2011 Wi-Fi Programmable Controllable Thermostat Pilot Program. The findings, conclusions, and recommendations have been drawn from data collection activities that included billing analyses, site visits, and interviews with program administrator staff, contractors, and participating customers.

Conclusion: Based on findings from the study, the evaluators drew several conclusions. The gas savings for single thermostat installations are considerably larger than for two thermostat installations. Also, when replacing non-programmable thermostats, the Wi-Fi thermostats achieve larger gas savings than when replacing standard programmable thermostat replacements. The electric savings for non-programmable thermostat replacements are effectively equal to those for programmable thermostat replacements. Additionally, electric savings associated with Wi-Fi enabled thermostats vary significantly from one house to another. For some participants, the energy savings benefits of a Wi-Fi enabled thermostat are similar to those of a standard programmable thermostat; however determining whether a participant saves more energy with a Wi-Fi thermostat than they would with a programmable thermostat is difficult to predict because the savings are reliant on participant behavior. In general, participants expressed satisfaction with the Wi-Fi pilot program processes and only few participants made suggestions for program improvements.

Major Renovations Pilot Memo

Synopsis: This memo summarizes findings from the interviews with homeowners, architects and builders involved with completed projects, focusing on satisfaction with the Pilot and suggestions for how the Pilot could be improved or made more user-friendly. In addition, it summarizes a discussion with a HERS rater who worked with several of the completed projects.

Conclusion: Preliminary interviews with potential participants who decided not to enroll in the Pilot showed the interviewed homeowners and builders were committed to energy efficiency; however, they did not think that participating in the Pilot would be cost-effective. Their suggestions for making the Pilot more user-friendly for homeowners focused on providing a clearer description of what the Pilot offered and the cost of participating in the Pilot, especially the cost of hiring a HERS rater. Interviews with homeowners who enrolled in the Pilot showed they were generally satisfied with their experience, satisfied with the rebates and incentives they received, and satisfied with the HERS rater they worked with. Most interviewed homeowners said working with a HERS rater was valuable and that the cost of working with the HERS rater was reasonable for the services provided. Participants’ basic suggestions for improvement were consistent with the suggestions from potential participants who decided not to enroll in the Pilot.

Massachusetts Consumer Survey Results 2011

Synopsis: The goal of the consumer survey was to track key indicators of the market for compact fluorescent lamps (CFLs), light emitting diodes (LEDs), and halogens as well as the impact of EISA. Evaluators conducted the survey in December 2011 and January 2012. The survey was timed to coincide with the EISA-mandated onset of the phase-out of 100 Watt incandescent bulbs. Therefore, the results provide a baseline understanding of these important indicators at the earliest stages of EISA.

Conclusion: Nearly all respondents in 2011 indicated that they were aware of CFLs before responding to the survey. This result represents an increase in awareness from the 2009 survey, but does not differ widely from awareness rates in 2010. The survey also provides evidence that customer satisfaction with CFLs continues to decline, but the reasons for the decline remain unclear. Respondents reported increased familiarity with all types of specialty bulbs, but the increase in familiarity was smallest for A-line, covered CFLs. These bulbs, however, closely resemble incandescents and can fit into some type of fixtures that standard CFLs cannot. Users of dimmable CFLs generally tend to like them, but the most persistent complaint about them is that they do not dim consistently or as much as other types of light bulbs. Dimmable users also complain about warm-up time. The results presented suggest that about one-fourth of respondents will consider stockpiling incandescent bulbs and evidence suggests that some of these respondents have already started to do so. While consumers are becoming more familiar with the term “lumens” and understand that it means light output or brightness, they still buy bulbs based on wattage or wattage equivalence.

Massachusetts Mini Baseline Study of Homes Built at the end of the 2006 IECC Cycle Final Report

Synopsis: This report presents the results of a Mini Baseline study of residential new construction building practices in Massachusetts at the end of the 2006 IECC (International Energy Conservation Code) cycle. On-site inspections of 50 non-ENERGY STAR® homes completed from late 2009 through early 2011 were conducted in April through June 2011. As part of this process, HERS (Home Energy Rating System) ratings were completed for each home. The report focuses on three main topics: consistency of study findings with current User Defined Reference Home (UDRH) assumptions; comparison to individual prescriptive energy code requirements at the end of the 2006 IEC code cycle; and identifying differences between custom and spec built homes.

Conclusion: Findings suggest that some current UDRH inputs may underestimate the energy efficiency of current building practices or equipment such as heating systems. In other cases, such as with insulation levels, UDRH inputs may overestimate the energy efficiency of current building practices. Inspected homes are most likely to have at least R-19 conditioned/ambient wall insulation (2006 IECC prescriptive requirement) and least likely to have R-38 flat ceiling insulation (2006 IECC prescriptive requirement). The HERS ratings suggest there is little difference between the energy efficiency of custom homes and spec homes. Custom homes tend to have higher R-value conditioned/ambient wall and flat ceiling insulation, while spec homes tend to have higher R-value floor and foundation wall insulation. Custom homes have slightly more efficient heating systems and spec homes have slightly more efficient water heating systems. Spec homes have lower duct leakage and air infiltration. Custom homes have more energy-efficient light bulbs and a slightly higher percentage of ENERGY STAR refrigerators and dishwashers.

Massachusetts 2011 Baseline Study of Single-Family Residential New Construction Final Report

Synopsis: The 2011 baseline study was conducted with several objectives in mind. The primary objective is to provide a baseline study of the characteristics of single-family homes recently completed across Massachusetts and permitted under the 2009 International Energy Conservation Code (IECC) that did not participate in the Massachusetts New Homes with ENERGY STAR® Program (Program), These homes can be used to update User Defined Reference Home (UDRH) assumptions used in calculating Program savings.

Conclusion: The 2011 baseline preliminary estimated UDRH inputs suggest that the current UDRH assumptions may underestimate the efficiency of some current building practices or equipment. In other cases, study findings suggest current UDRH inputs may overestimate the efficiency of current building practices. Almost all inspected homes (93%) have at least one instance where the R-value of installed insulation is below 2009 IECC prescriptive insulation requirements or mandatory duct insulation requirements. REM/Rate was used to assess heating and cooling system sizing. Results show most heating and cooling systems are oversized. Average flat ceiling R-values and U-values, cathedral ceiling U-values, unconditioned basement foundation wall R-values, duct insulation R-values, air infiltration, duct leakage and HERS indices are all significantly less energy efficient in baseline homes. Compared to the Mini Baseline Study homes, 2011 baseline homes are smaller. Also, the average HERS Index is significantly lower (better) in 2011 baseline homes (HERS 79) than in Mini Baseline Study homes (HERS 83). Findings suggest that making potential homebuyers aware of ENERGY STAR homes is not enough to get them to act on that awareness when they look for a home or a builder to build a custom home.

Massachusetts Residential New Construction Four to Eight Story Multifamily Pilot Interview Findings

Synopsis: The Four to Eight Story Multifamily New Construction Pilot (Pilot), was introduced in 2010 to address the needs of mid-rise multifamily buildings. NMR conducted fourteen interviews with the Pilot’s sponsors, implementers, and participants with completed projects, examining the Pilot’s goals and objectives, the process of signing up and completing verification, outreach and the timing of projects served, the measures covered, the measures installed, barriers to energy efficient multifamily new construction, and satisfaction. The importance of getting a more efficient home with lower energy bills has risen from 2002 to 2010.

Conclusion: The Pilot has finished its second year with its process operating fairly smoothly to address the needs of the mid-rise multifamily new construction market with participants reporting high overall satisfaction. From the implementer’s perspective, the initial participation process in a similar program could be improved by providing an interactive project tool that would allow the customer user to enter the measure inputs and see the savings and incentives right away. The Pilot’s implementers and participants both pointed out the advantages of a performance based system. Timing remains an issue. About one-half of the fourteen projects that had completed participation, in terms of square footage, were in an early design stage with the HVAC equipment still under discussion when they learned of and signed up for the Pilot. Ideally, program engagement should occur during the conceptual design phase of a new project if it aims to provide technical assistance in considering all applicable energy efficiency measures. The project manager thus emphasized the need for the sponsors and regulators to take a longer term view.

Massachusetts Multifamily Program Impact Analysis Report & Appendix

Synopsis: This impact analysis has two primary objectives for the multifamily program activities of the MA Program Administrators (PAs). First, the goal of the impact work is to provide a set of savings approaches (i.e., algorithms and deemed values) that can be used by all PAs (statewide) in future program years. Second, the analysis collected information to inform program attribution, including the measurement of installation rates, persistence, freeridership, and spillover. Findings, conclusions, and recommendations are draw from a review of each PA’s multifamily program tracking data, the measure-specific savings estimates used by the PAs, as well as a related review of program audit data not contained in the program tracking databases.

Conclusion: The major deliverable of this work can be found in Appendix A. For each program measure, this appendix provides information similar to that provided in the TRM, detailing the recommended algorithm, required inputs to be collected during implementation, and underlying assumptions (to calculate algorithm-based assumptions) or deemed values. In general, evaluators also conclude that the multifamily program is experiencing excellent installation and persistence rates, though ensuring that programmable thermostats are used to their full capabilities after installation is a challenge. Freeridership also appears to be well under control at a rate of 18% at the program level. “Likeprogram” spillover among property managers has been low. Participant data is tracked differently by each PA and there are significant gaps in the types of program information being collected and tracked.

Brushless Fan Motors Impact Evaluation: Part of the Massachusetts Residential Retrofit & Low Income Program Area Evaluation

Synopsis: This report presents the evaluated gross impact savings of brushless fan motors (BFMs) incentivized as a part of the Cool SMART program for upgrading heating, ventilating, and air conditioning (HVAC) units. This evaluation used on-site spot measurement and long-term metering of motors to determine the savings impact of a statistically significant sample of participants

Conclusion: The evaluated average energy savings of all units in the sample is 246 kWh. The resulting energy realization rate is 41%, which is the ratio of the evaluated savings divided by the deemed savings of 600 kWh. The average energy savings calculated includes interactive savings due to the reduced cooling load from a more efficient motor, assuming an SEER of 8.3. The heating penalty is calculated assuming 78% forced hot air efficiency. The low realization rate is largely the result of a difference between deemed hours of use and logged hours of use. Logging showed that the BFMs operated in Auto mode, cycling to maintain temperature set points, and averaged about 1,493 hours per year in operation, whereas deemed savings value implicitly assumes that the motors operate more than 5,000 hours per year.

Demand Impact Model User Manual

Synopsis: The MS Excel 2007-based model, Demand Impact Model, allows users to estimate seasonal and coincident peak demand savings of energy-efficient measures. Users can trace all inputs and calculations from load shape profiles to peak period definitions to measure level energy and demand savings. This document provides an overview of this model and information about using it.

Conclusion: None available at this time.

Low Income Single Family Program Impact Evaluation: Part of the Massachusetts Residential Retrofit & Low Income Program Area Evaluation

Synopsis: This report summarizes the impact evaluation of the 2010 Low Income Program using an array of data collection activities and evaluation tasks, including billing and engineering analyses, to determine the evaluation findings, conclusions, and recommendations presented.

Conclusion: The primary results of this evaluation were reported as the average number of measures installed per household and the average energy savings per household per year by measure type. Natural gas measures included insulation and air sealing measures, heating system replacements, duct insulation, duct sealing, and DHW measures. Electric measures included insulation and weatherization measures, refrigerator replacements, second refrigerator removals, freezer replacements, window AC replacements, CFLs, torchieres, fixtures, DHW measures, and other measures from TLC kits. Oil measures included insulation and air sealing measures, oil heating system replacement, boiler reset controls, programmable Thermostats, Furnace Fans, duct insulation, duct sealing, and DHW measures.

Solar Hot Water Pilot Program Evaluation Report: Part of the Massachusetts Residential Retrofit & Low Income Program Area Evaluation

Synopsis: This report summarizes the findings of an evaluation of National Grid’s Solar Hot Water (SHW) Pilot Program. Data for this report were obtained through billing analyses, customer surveys, site visits, and engineering reviews of solar hot water systems installed through this program over the past several years.

Conclusion: The SHW Pilot Program gross gas savings, based on engineering estimates and modeling, is predicted to be approximately 701 MMBTU/yr, with average savings of approximately 14.2 MMBTU/yr per program participant. The SHW Pilot Program net gas savings, based on a billing analysis to account for takeback and other effects, is approximately 512 MMBTU/yr, with average savings of approximately 10.9 MMBtu/yr per program participant. Site visits confirmed the quality of SHW installations, with the only consistent problem being the lack of a UV-resistant jacket over the foam insulation on outdoor piping. The most common non-plumbing issue observed was excessive shading of solar collectors. The cost-effectiveness of SHW systems installed through this program is low, with simple post-rebate payback periods to customers of 50 years, on average.

Home Energy Services Net-to-Gross Impact Evaluation: Part of the Massachusetts Residential Retrofit & Low Income Program Area Evaluation

Synopsis: This report summarizes the 2011 Home Energy Services (HES) Program net-to-gross (NTG) impacts. Evaluation findings, conclusions, and recommendations are based on results from several data collection activities and evaluation tasks including participant and non-participant surveys and self-report and discrete choice (DC)-based assessments of measure-level NTG ratios. The Evaluation Team estimated measure-specific NTG participant free-ridership and spillover, as well as non-participant spillover for some measures, to determine NTG ratios.

Conclusion: The program-level NTG ratio was estimated to be to be 113%. For direct install measures, the CFL NTG estimate is 73% and air sealing is 129%. For incentive measures, the insulation NTG ratio estimate is 123% and the refrigerator estimate is 86%. The evaluation does not report NTG components that overlap with other programs, such as CFL NPS and refrigerator PS, which would double count the impacts of the concurrent Upstream Lighting and ENERGY STAR Appliance Rebate programs.

Heat Pump Water heaters Evaluation of Field Installed Performance

Synopsis: This evaluation quantified the in-situ performance of three recently-released HPWH products (General Electric GeoSpring™, AO Smith Voltex®, and Stiebel Eltron Accelera®300) through the installation and monitoring of 14 units in Massachusetts and Rhode Island for over one year.

Conclusion: Though a small sample set, the overall performance of these 14 HPWHs shows great promise for this technology. In general, these HPWHs were more than twice as efficient as a traditional electric resistance tank water heater; though there are large variations in overall efficiency as performance is dependent on ambient temperature/relative humidity, mains temperature, hot water setpoint temperature, and water usage (consumption and concentration). Despite some issues with siting and location, these new units performed with remarkable energy and cost savings over ERWHs. Evaluators suggest that homeowners need to be aware that these integrated HPWHs often are not a direct size-for-size replacement for existing water heaters due to their lower heat recovery rates. Also, manufacturers’ minimum space requirements need to be adhered to. Depending on air conditions in the water heater location, expectations of the efficiency performance of these units need to be adjusted. To ensure proper operation of a HPWH, filters need to be cleaned regularly, condensate drains need to be checked periodically.

Massachusetts 2011 Low Income Program Process Evaluation Final: Part of the Massachusetts Residential Retrofit & Low Income Program Area Evaluation

Synopsis: This report summarizes the process evaluation of the 2011 Low Income Single-Family (LISF) and Low Income Multifamily Retrofit (LIMF) programs (collectively known as the Low Income Program). The evaluation findings, conclusions, and recommendations presented are based on results from an array of data collection activities including interviews with Program Administrator (PA) Low Income program managers, Community Action Program (CAP) agencies, stakeholders from the Low Income Energy Affordability Network (LEAN), program energy auditors, installation contractors, and multifamily building managers.

Conclusion: The Team’s evaluation revealed numerous ways in which the Low Income Program is succeeding and identified opportunities for improving processes and outcomes. For example, program evaluators recommend that PAs provide savings goals for the Single Family program to their lead agencies to improve transparency between PAs and program implementers. For the Multifamily program, the LIMF Advisory Committee should encourage more standardization across PAs by developing standardized project screening criteria or a tool to determine savings and cost effectiveness for both gas and electric projects. Additional recommendations are also described.

Massachusetts Multifamily Market Characterization Potential Study

Synopsis: This study assesses the potential energy-efficiency savings available in Massachusetts’ multifamily buildings. A market characterization was done based on data collected from workshops with multifamily property managers and PA program implementers from MA, as well as on-site visits from 193 multifamily buildings in MA. These data, supplemented with information from PA data tracking systems and secondary sources, provided the foundational elements for estimating the technical, economic, and achievable potentials.

Conclusion: Results of the study for electricity indicate that 2.8 million MMBTU of technically feasible, electric energy-efficiency potential will be available by 2030. Once screened for cost-effectiveness, this technical potential translates into an economic potential of 1.8 million MMBTU. Should all of this cost-effective potential be deployed, the result would be a 14% reduction in 2030 forecast energy consumption. These percentage estimates of electric technical and economic potentials are comparable to those for multifamily buildings in a 2010 Consolidated Edison potential study. For gas, results indicate that 7.0 million MMBTU of technically feasible, natural gas energy-efficiency potential will be available by 2030. This technical potential translates to an economic potential of 5.2 million MMBTU. Summing the totals across fuels, evaluators estimate 34 million MMBTU of multifamily electric and gas energy consumption in 2030 and indicates nearly 10 million MMBTU (29% of baseline sales) of technically feasible energy-efficiency potential and 7.0 million MMBTU (21% of baseline sales) of total economic potential will be available by 2030.

Massachusetts Multifamily Program Process Evaluation: 2011 Energy Efficiency Annual Report

Synopsis: This report summarizes the process evaluation of the 2011 Multifamily Program. The evaluation findings, conclusions, and recommendations presented have been drawn from an array of data collection activities, including interviews with Program Administrator (PA) staff and Implementers, surveys of participating property managers and tenants, a focus group of property managers, a literature review, and a robust data review.

Conclusion: Although streamlining of the Multifamily Program participation process has greatly improved, a lack of awareness of the program and of what the program can offer was reported as a significant barrier to participation in the program. In the coming years, increasing program goals will likely require the Multifamily Program to seek out new opportunities for energy savings. On average, Multifamily Program participants were highly satisfied with almost all aspects of the program. Property managers and tenants expressed a need for follow-up technical assistance on some of the measures installed through the program. Additional findings are also described.

2011 Home Energy Services Packaged Measure Pilot Process Evaluation: 2011 Energy Efficiency Annual Report

Synopsis: This report summarizes an evaluation of the 2011 Home Energy Services (HES) Packaged Measure Pilot (pilot) offered by the electric and gas Program Administrators (PAs) of Massachusetts. In March 2011, the PAs launched the pilot to encourage HES participants to install a greater number of the recommended measures, thereby ensuring the HES program achieves deeper savings within each home. The report focuses on evaluating the effectiveness of the additional incentive in meeting the pilot’s stated goal, as well as assessing the delivery of the pilot itself. Results are based on interviews with PA Program Administrators and Vendors as well as participant and non-participant surveys.

Conclusion: The PAs offered eligible new and past HES Program participants a total of 641 packages; however, only 52 customers decided to participate (8%). Of the 52 who accepted the pilot, 46 were new participants and six were past participants. These results indicate that the pilot was more likely to be accepted if the participant was new and the pilot was offered at the time of their HES audit, rather than if the participant was offered the pilot after having completed the audit. Pilot participants were also more likely to adopt the measures than were the typical HES participants eligible for the measures. This demonstrates that the additional incentive of the pilot may increase the installation of all measures that were packaged in the pilot. Evaluators identified barriers to participation including high upfront costs, timeframe requirements, and extra paperwork. Generally, the PAs and vendors expressed dissatisfaction with the pilot design and results, although they also named a few implementation elements that were satisfactory. Both PAs and vendors suggested improvements such as extending the offer timeframe, re-evaluating packages, and increasing marketing.

Home Energy Services Realization Rate Results Memo

Synopsis: This memo describes results of a completed impact evaluation of two Home Energy Services (HES) program measures: insulation and air sealing. The HES program is available to single-family residential customers living in buildings with up to four units throughout the Commonwealth of Massachusetts. Ex post savings for both measures were determined using a billing analysis and an engineering analysis.

Conclusion: Results report ex post savings determined for insulation and air sealing by PA and all four fuel types. The ex ante and ex post savings as well as realization rates varied across PAs. This variation resulted from multiple factors, including: the profile of customers served, the mix of measures installed, the weather the service territory experiences, and the vendor-specific software used. Comparing Commonwealth-wide realization rates for insulation and air sealing from the August 2012 evaluation (based on data prior to the software updates) and the current evaluation (based on data after the software updates) indicated that the updated software produces ex ante savings estimates that are closer to the ex post estimates determined by the Evaluation Team. Specifically, this evaluation resulted in a Commonwealth-wide realization rate of 76% as compared to the 61% realization rate found in the previous evaluation (for insulation and air sealing only). Additionally, average savings per participating home – using natural gas for heat – also increased for the participant homes included in the 2013 evaluation (139 therms) versus participant homes included in the 2012 evaluation (92 therms) due to the increased adoption of recommended measures.

Residential Program Studies

2023

2022

2021

2020

2019

2018

National 2017 Seasonal Savings Evaluation

2017

2016

2015

2014

2013

2012

2011

Cross-Cutting Behavioral Program Evaluation

Appliance Turn-In Program Evaluation

New Homes with ENERGY STAR® Program

ENERGY STAR Lighting Program

2010 Residential Retrofit Program and Low-Income Programs

2010

Search by Tag

Search by Date Range

Stay Informed