Commercial and Industrial Studies

Synopsis: The annual Commercial and Industrial (C&I) Customer Profile Study (CCPS) report presents analysis of Massachusetts Program Administrators’ (PAs’) billing and tracking data. The analysis and reporting of the statewide data allow the PAs and the Energy Efficiency Advisory Council (EEAC) Consultants to accurately quantify and report on trends and time series evolution in the Massachusetts C&I landscape, develop narratives about these trends and their implications for a variety of stakeholder interests, and help formulate testable hypotheses for future process, market, and impact assessment studies. The CCPS report also allows the PAs to evaluate how their standardized data compares to other PAs’ standardized data, and compares to the state as a whole, while always maintaining PAs’ customer and IT system confidentiality.

Conclusion: The study provides a deep, in-depth, and complex look at the commercial and industrial sector of Massachusetts. To best understand the high level take-aways of the study, the “C&I Customer Profile Study One-Pager” can provide a good understanding of the content and implications of the study.

P86 Lighting Hours of Use Study

Synopsis: This document presents the results of the “Quick Hits” C&I Lighting Hours of Use Study (C&I Lighting HOU Study). The primary objective of this project was to develop building level annual hours of use estimates for estimating savings for the upstream lighting program offering.

Conclusion: Table 3-1 in the report presents the results of this analysis as well as the absolute and relative precision estimates for each building type. All precisions were calculated at the 90% confidence interval. The final list of buildings used in the analysis was pared down from the 458 that were initially identified in the scoping stage down to for various reasons. These mostly included combining unique project analyses that were completed at the same building into one and sites that only included incentivized lighting controls.

P78_C&I_Comprehensive_Lighting_Inventory_Memo

Synopsis: This report covers the results of a series of computer-assisted telephone interviews (CATI surveys) and on-site lighting inventories of Massachusetts commercial and industrial (C&I) facilities. The respondents included customers at sites that have participated in the Massachusetts’ Program Administrators (PAs) lighting programs as well as customers that have never participated in a lighting or other electric-equipment program. The report aggregates the results of two separate, but related research efforts completed in 2018 to develop estimates of the overall market adoption and saturation of LEDs in Massachusetts and adoption and saturation amongst participants and non-participants.

Conclusion: The memo provides updates to a variety of C&I lighting market saturation figures. Figure one on the first page of the memo shows saturation percents for the linear and non linear markets. The rest of the memo provides more detailed results on the C&I lighting market.

Demand Response Connected Solutions Demonstration Report

Synopsis: This study highlights the results of an evaluation of National Grid’s Connected Solutions Commercial and Industrial Demand Respondd Demonstration activity over the summer of 2018. This was the second study of this Demonstration; the first study concluded with a report published in 2018. The Connected Solutions Demonstration is a two year Demonstration project for demand response available to National Grid Massachusetts C&I electric customers on a G2 or G3 rate with a demand of 250 kW or higher. The Demonstration provides day ahead notice and provides a per kW performance incentive without a non-performance penalty.

Conclusion: National Grid uses three curtailment service providers (CSPs) to sign up customers and identify demand response and demand management opportunities. National Grid administers the Demonstration and uses an AutoGrid system to dispatch notification, aggregate data, and complete the settlement process. Participation increased from 99 enrollees with 20.6 MW in enrolled capacity in the 2017 season to 276 particpants with 77.7 MW in enrolled capacity in the 2018 season. The top three sectors in terms of number of participants and enrolled capacity in both seasons of the Demonstration included manufacturing, educational services, and retail trade. There were six events called in 2018. The Demonstration used a hybrid baseline to determine performance. This settlement process used the higher curtailment provided by either an additive adjusted 10 of 10 or an unadjusted 10 of 10 baseline.

TXC49 (C&I NTG) MOU Research Results Memo

Synopsis: This memorandum details the results of the TXC49 net-to-gross (NTG) analysis of commercial and industrial customers who have executed a Memorandum of Understanding (MOU) with
Eversource. The goals of this effort were to explore how MOUs impact program participation, compare MOU NTG survey responses to non-MOU survey responses, and determine whether changes should be considered for the NTG methodology to account for the effect. This research was not meant to be a comprehensive look at MOU customers but rather to get a pulse on any potential changes. The research included only Eversource account managers, customers, and data.

Conclusion: Results from this study indicate that MOUs do influence customer adoption of stretch savings goals via program participation, although the extent of this influence is unclear given the limited
scope of this effort. In the interviews, MOU customers stated they would not have been able to complete the projects they did without the assistance they received from the program through the MOU. Analysis of the TXC49 NTG survey results also show lower free-ridership rates among MOU respondents. (For Eversource electric, the difference was just over 4%, 15.1% compared to 19.0%.) As the TXC49 NTG survey did not specifically ask about the influence of MOUs, it is unclear if the respondents were considering the MOU during survey responses or just their
general interactions with the programs.

P73B Energy Management Systems Baseline Opinion Memo

This memo provides the DNV GL team’s opinions on the baseline event type of energy management system installation in Massachusetts support by the energy efficiency programs. The baseline event types are discussed in detail in the Massachusetts Commercial/Industrial Baseline Framework. The memo begins with a review of the current measure definition. It then provides a proposed approach for baseline type categorization, opinions on new market issues for this measure, a brief summary of the known codes and standards with relevant language, and a discussion of the newer features available to customers. It closes with ideas for future research into this topic.

Synopsis: For the purposes of the memo, DNV GL defined EMS as systems which control and monitor heating, cooling, and any other non-lighting energy consuming systems. The categorization section breaks down when measures should be considered lost opportunity or early replacement, with an additional section on the categorization of EMS expansions. The next section identifies three different market conditions that may result in an increase in the replace on failure installations compared to early replacement installations. To read more on codes & standards, new features, and new avenues for research, refer to the memo.

Enhanced Customer-Level Database Capabilities Memo

Synopsis: This memo describes improvements done to the Massachusetts Commercial and Industrial Evaluation Database. The database captures the annual billing, customer, and installed energy efficiency measure information by program year, including energy consumption, savings, geographic data, and contact information at the account level, to support evaluation, measurement, and verification (EM&V) projects in Massachusetts. This update involved acquiring 3^rd party data to support the ongoing efforts. There were also updates to the logic behind using geographic and string-searching algorithms to match datasets.

Conclusion: This project developed a series of logic models that integrated third-party datasets with the Program Administrators’ (PAs) C&I and Residential customer information systems, to help PAs and evaluation contractors better understand the customer population. Integrating third-party data helped develop additional analysis grains including location, tax parcel, and cross-fuel customers. It also provided a large, documented, and standardized set of new variables that evaluation projects could incorporate into data mining and analytical undertakings.

TXC 53 Analyses of Immediate Code Compliance Support Initiative Commercial Classroom Training Surveys

Synopsis: This memo provides analyses of the immediate survey responses collected through paper surveys, registration data, and poll questions from seven Code Compliance Support Initiative (CCSI) commercial trainings. These are the first commercial training surveys to be collected since implementation of the CCSI passed from CLEAResult to Performance Systems Development (PSD), who took over providing training sessions in 2018. All trainings focused on the commercial 2015 IECC code which became mandatory in Massachusetts at the beginning of 2017. Two trainings pertaining to solar-ready provisions (which became mandatory at the beginning of 2018) administered in June 2018 are not covered in this memo because few surveys were collected at these sessions.

Conclusion: Respondents to the surveys found that the training sessions were of high quality. The survey had 64 total respondents, comprised of 45 from the MA CCSI Commercial Code Training Sessions and 19 from the MA CCSI Commercial Air Barriers Training Sessions. On a 6-to-1 scale, in which 6 is excellent and 1 is poor, mean ratings ranged from 5.0 for the handouts to 5.5 for the presenter’s skills. Over the course of the seven training sessions, the rating for presenter’s skills increased from a 5.0 to a 6.0. Additionally, fewer attendees said they would recommend the first three trainings (79%) compared to the latter four (97%). Both trends indicate that quality of the trainings increased over time, likely attributable to the session holders gaining experience over time. The overall mean usefulness of the trainings was rated at 5.1, compared to 4.9 for the trainings held in 2016 and 2017.

2018

ISP Study Findings Memo – Air Compressors and Compressed Air Dryers

P78 C&I Upstream LED NTG Report

Synopsis: This report summarizes an analysis of the net-to-gross (NTG) ratios associated with the Upstream Lighting Initiative for commercial and industrial (C&I) customers offered by the Massachusetts Program Administrators (PAs). Evaluators conducted this analysis to support a reanalysis of NTG for LED products for the 2019-2021 three-year plan and in response to findings from the 2015 Final Report of Massachusetts LED Spillover Analysis (2015 report) which found high levels of spillover associated with the adoption of LED screw-based products. The methodology used to calculate the NTG ratios for screw-based and linear LEDs relies on buyer’s assessment of how the program influenced the timing, efficiency, and quantity of their lighting purchases. It also considers how the program may have influences the efficiency of some, but not necessarily all, of the purchased units. Evaluators estimated spillover under four scenarios that varied in their treatment of out-of-program savings.

Conclusion: Using the results of Scenario 4, screw-based free-ridership is estimated to be 0.25 and spillover is 0.23, yielding a final NTG value of 0.97. For linear technologies, the free-ridership ratio is 0.23 and spillover is 0.14, yielding a final NTG value of 0.91. Section 4 of this report includes the results of the four spillover scenarios and additional findings related to free-ridership and spillover. The evaluation team is currently conducting additional lighting market research and the Consensus Group postponed developing their final recommendation for the prospective NTG ratios to investigate whether this research could further inform the process. The Consensus Group agreed upon the Prospective NTG ratios ranging from 0.73 in 2019 to 0.53 in 2021 for screw-based technologies and 0.80 in 2019 and 0.66 in 2021 for linear technologies. Evaluators note that as market adoption of these products continues, the NTG should decrease, with the screw-based NTG ratios decreasing at a faster rate because there has been more adoption of screw-based LEDs to date. Other findings are presented related to Participant Characterization of the Initiative’s Influence, Market Development, Lamp Replacement Decisions, and Continued Importance of Market Actors.

P73A C&I TWGA Portfolio Findings and Conclusions

Synopsis: This memo summarizes the findings and conclusions from an analysis of the potential impact on savings of complying with three new evaluation practices outlined in the Baseline Framework. Prior to the Baseline Framework, these practices were present, but not systematically implemented in the evaluation and reporting of C&I portfolio savings. The three analyzed practices are: Industry standard practice (ISP), Lost Opportunity – Retrofit Reclassification (LORR), and Dual baseline.

Conclusion: Table 1 summarizes the estimated impact of the systematic application of the three new practices on the 2016 CI portfolio savings using the assumptions that are described in Attachment A. The table shows the estimated percent change in 2016 portfolio savings estimates if we were to evaluate it today using the Baseline Framework. Findings and conclusions are presented related to ISP Practice Effects on Lost Opportunity Measures, LORR Practice Effects on Retrofit Measures, Dual Baseline on Retrofit Measures, Measure Lives and BCRs, Summer Demand and FCM, and different Scenarios.

P81 C&I Upstream Lighting ISR Analysis Summary

Synopsis: This memorandum summarizes the recommendations and considerations from the In-Service Rate (ISR) Analysis. This analysis was part of Project 81: Process Evaluation and Site Visits for the C&I Upstream Lighting Initiative. Evaluators conducted this analysis following the most recent (PY2015) impact evaluation since the electric Program Administrators (PAs) have recently (2016-2018) have been making improvements to the Initiative. The objective of the ISR analysis was to calculate installation rate alternatives from the prior impact evaluation for use by the PAs in the 2019-2021 Three Year Plan. The ISR analysis reviewed both QC contractor inspection data and evaluation on-site data. In June and July 2018, evaluators completed 233 site visits to 2018 Q1 Initiative participants as part of this effort.

Conclusion: Based on their analysis, evaluators recommend using the revised lighting installation rate of 76.2% and savings factors presented in this memo for 2019-2021 Three Year Planning. This represents an increase from the overall PY2015 installation rate (65%). Recommended savings factors to be used in three-year planning are shown in Table 1 – Table 7. Evaluators also suggest building upon the initial July 2018 site visits with “rolling” data collection. The next phase of the study will involve “rolling” data collection, so that the PAs can replace prospective (PY2018 and beyond) results recommended as part of the PY2015 impact evaluation to reflect Initiative changes made since the PY2015 impact evaluation. Evaluators suggest that the PAs consider targeting a more representative sample of sites for QC inspection. The distribution of QC sites by measure category makes it difficult to use current QC results along with some adjustment to represent the population. Also, the PAs could consider requiring the QC contractor to visit sites from each distributor monthly, which is the protocol mentioned in Initiative’s Distributor Handbook.

TXC 35 C&I Upstream HVAC/Heat Pump and Hot Water NTG and Market Effects Indicators Study

Synopsis: The primary purpose of this study was to measure the retrospective (2016) and estimate the prospective (2019-2021) net-to-gross ratio (NTGR) and market effects indicators for selected equipment types supported by the Upstream HVAC/Heat Pump (HP) Initiative and the Upstream Water Heater Initiative. The equipment, which Program Administrator (PA) staff selected in collaboration with the evaluation team, comprise five types of HVAC/HP and gas-fired water heating equipment:
• Ductless mini-split heat pumps
• Electric water-source heat pumps
• Air-cooled unitary/split central air conditioning (>5 tons)
• Gas-fired storage water heaters between 76,000 and 300,000 BTU/hour
• Gas-fired tankless water heaters between 180,000 and 199,900 BTU/hour

Conclusion: The findings from this study indicate that the 2016 Upstream HVAC/HP and Water Heating Initiatives had minimal impact on changing distributor and buyer behavior and inducing program attributable savings. Overall, the equipment-level NTGRs derived from the Distributor-Reported approach ranges between 13.9% and 43%. In 2016, the Upstream HVAC Initiative alone represented 1.2% of electric savings. Combined with other HVAC programs, it represented 14% of total electric savings. During this same period, the Upstream Hot Water Initiative comprised 10% of total gas savings. To address the observed low levels of attribution in advance of the next three-year plan, and improve the future evaluability of these Initiatives, the evaluation team presents several recommendations in the report.

CIEC Stage 5 Report – Baseline Change Effects on NTG

Synopsis: The Net-to-Gross Revisions track (Track C) of the Baseline Transition Planning project (P73) is an effort to provide additional specificity for the full transition of both evaluation and implementation to the Massachusetts Commercial/Industrial Baseline Framework (Baseline Framework) developed in 2017. The purpose of P73 Track C is to establish net-to-gross ratio (NTGR) methodology that accounts for industry standard practice (ISP) and customer-specific baselines. The Track C study included the following activities:
• Review of the prior participant self-report survey and scoring
• Recommend minor changes for the TXC49 field work conducted in the fall of 2017
• Develop and review alternative approaches
• Empirical analysis of alternatives
• Coordinate with the Track B development of ISPs for selected measures

Conclusion: Evaluators present key findings related to what customers can reasonable answer and how different NTGR results are using different methods. The TXC49 questionnaire included some follow-up questions to understand how customers interpret the intermediate efficiency question. The responses generally confirmed the potential for confusion and response error for this question, which could lead to the biases conceptually identified. The questionnaire also included a test of the measure-specific questions. Results indicated that respondents are generally able to report on the alternative technologies they would have installed instead of what they installed, when the alternatives are specifically enumerated. The TXC49 study produced results using the “Classic” calculation method. For the measures with ISP baselines, NTGR using the “Simple” method was also calculated. From the TXC49 survey results, sufficient data were available for three measures to compare FR results between the Simple and Classic methods. These results indicate the following:
• The Simple method produces consistently lower FR results compared to the classic method.
• Despite moderate to large differences between the methods for the efficiency component of FR, the effect on total FR is much smaller. The difference in total FR between the methods is under 1 percentage point for air compressors and boilers, and approximately 2 percentage points for LEDs. The lesser effect on the total FR score occurred because most of the records where efficiency FR changed already had zero total FR because of the timing component or their total FR score was set based on overrides triggered by answers to open-ended questions.

Memo on P73 Track D: Expected Useful Life Estimation Results to Date EUL

Synopsis: As part of the Project 73: Track D Measure Life Method study (P73 Track D), the evaluation team devised a novel method for estimating air-conditioning (AC) expected useful lives (EULs) using the observed AC age data collected through the recent Massachusetts market characterization efforts (P41, P55). The team presented preliminary results in February 2018. After the Massachusetts Program Administrators (PAs) and Energy Efficiency Advisory Council (EEAC) Consultants suggested additional analysis, primarily around AC units with missing manufacture dates, the team presented revised results in April 2018. This memorandum presents all the results to date for P73 Track D. In it, evaluators provide a high-level description of their approach, followed by a discussion of the key data items from the data collection efforts, and the statistical methods used to estimate the EULs by AC type. Evaluators present results based on the current analysis and conclude with a discussion of potential next steps.

Conclusion: Table 1 in the study shows the average observed unit age before and after age imputation by AC type. The basic imputation increases the average age by about half a year compared to excluding the cases with unknown year. The alternative imputation increases average age by nearly two years compared to excluding the unknown year cases. Table 2 shows the estimated EULs with the different imputation methods. Table 3 shows more complete distributional characteristics of the fitted Weibull curves for each AC type. Several observations can be made from the table: The median lifetimes are mostly in the range of 7 to 9 years across the different AC types. Mean lifetimes are greater than medians, as is expected for age distributions. However, means can take on more extreme values. For AC types with fewer than 100 sites, the confidence bounds tend to be fairly wide. For AC types with 300 or more sites, the confidence bounds are tighter and more similar between the two estimation methods. The fit for all AC types together produces median lifetimes smaller than do any of the individual types alone. Reasons for this are unclear.

TXC 47 Non-Residential Code Compliance Support Initiative Attribution and Net Savings Assessment

Synopsis: Since 2014, the Massachusetts Program Administrators (PAs) have funded the Code Compliance Support Initiative (CCSI) to help improve compliance with residential and non-residential building codes in the state. The CCSI trains code officials and building professionals on the building codes and helps them stay abreast of changes to these codes. The purpose of this study is to estimate the savings attributable to the commercial portion of the CCSI for the 2019-2021 program period. This study used a Delphi panel approach to estimate savings attributable to the CCSI from code compliance enhancement efforts.

Conclusion: The evaluation team developed the 2019–2021 compliance rates as if the CCSI program were to end in 2018, and estimated that an increase of roughly five percentage points in compliance is attributable to the CCSI over the period (see Table 1 in the study). The estimated proportion of Gross Technical Potential (GTP) savings projected to be achieved during 2019–2021 is between 42% and 45%. The evaluation team estimated the savings attributable to CCSI by multiplying the GTP (see Table 2 in the study) by the proportion of GTP savings achieved by the CCSI (see Table 1 in the study). The evaluation team offers two alternatives for the overall net savings (see Table 3 in the study). Based on the findings from this study, evaluators offer the following conclusions: The CCSI is the primary statewide resource for training and support related to commercial building energy codes. The CCSI is, in part, responsible for increased non-residential code compliance rates in Massachusetts. The GTP savings assessment relies on several assumptions that could be improved by future research and coordination. The savings calculator only accounts for new construction projects and does not address potential savings associated with compliance enhancement in renovation projects that trigger code requirements.

TXC 49 Massachusetts Sponsors’ Commercial and Industrial Programs Free-ridership and Spillover Study

Synopsis: The primary purpose of this study was to quantify the net impacts of the commercial and industrial downstream energy-efficiency programs that offered incentives directly to customers in 2016 in order to recommend net-to-gross (NTG) ratios to be used in the 2019-2021 filing. The evaluation team conducted phone surveys with program vendors and customers who received an incentive in calendar year 2016 from one of the C&I programs to calculate free-ridership and spillover estimates to inform the recommended NTG ratios. The surveys were conducted between October and December in 2017 and ran concurrently for all the Sponsors.

Conclusion: The statewide 2018 NTG recommended rates for 2019-2021 were slightly lower than the results of the prior studies for both electric and gas. For electric, statewide NTG results increased between 2011 (94.1%) and 2015 (96.1%), then decreased in 2016 to 92.2%. Gas statewide results also increased between 2012 (78.7%) and 2015 (87.1%), and decreased in 2018 (84.0%), but was still higher than in 2012.

MA Commercial Energy Code Compliance and Baseline for IECC 2012 Final Report

Synopsis: This report provides a review of the results of the 2017 Massachusetts Energy Code Compliance and Baseline Study for IECC 2012 (2017 Study). This project is part of an ongoing effort to support the adoption of energy efficiency equipment, design, and construction practices that address energy and environmental challenges in Massachusetts. The primary objectives of the 2017 Study were to assess compliance with the 2012 International Energy Conservation Code for commercial new construction in Massachusetts, as well as to leverage data collected during the compliance assessment to inform baseline market practices for commercial new construction. The 2017 Study focused on the review of construction documents for a sample of 39 buildings.

Conclusion: Overall compliance, when weighted by building square footage, is estimated to be 94% using the DNV GL team methodology and 88% using the DOE/PNNL methodology. At a 90% confidence level, these results have a relative precision of 2% for the DNV GL methodology and 4% for the DOE/PNNL Methodology. Based on these results and the analysis presented in this report, evaluators draw the following conclusions: The compliance of nonresidential new construction buildings is consistent over time, even as the energy code gets more stringent. The findings of the study are a clear call to action to consider the status of standard practice across the Commonwealth and its relation to code requirements. The energy code requirement for interior lighting power is not reflective of current standard practices. The energy code does not reflect standard practices for many mechanical provisions. Opportunities for improving compliance remain for targeted measures. Energy code determinations continue to be made at the design stage for new construction. While this study cannot determine exact baseline values, the evidence strongly suggests that the PAs should be adjusting their program planning and implementation to account for changing standard practice. Evaluators also recommend focusing future training on energy code changes, field verifications, and specific provisions that are not easily understood and/or complied with. Additional considerations and opportunities for future research are also presented in this report.

Impact Evaluation of PY2016 C&I Small Business Initiative – Phase I Lighting Evaluation – One-page summary

Impact Evalulation of PY2016 Small Business Initiative – Phase I Lighting Evaluation Report

P75 LED Market Monitor Study: Lighting Market Model Summary Memo

Synopsis: This memo summarizes an analysis of the commercial and industrial lighting market in Massachusetts and investigation into future market share forecasts with and without program intervention. As part of this LED Lighting Market Monitor study, evaluators compiled information and data on the LED commercial and industrial lighting market in Massachusetts and other comparable regions. Evaluators utilized this information to develop a topic-level summary memo and a stock adjustment model to forecast LED market share and energy savings associated with the future adoption of efficient lighting.

Conclusion: The memo describes the methodology for developing the stock adjustment model, and the inputs used in the model. The model produces results for two market development scenarios. The “Program Scenario” represents evaluators’ best estimate of what the installed stock and market share of equipment technologies will be from 2015 through the end of forecast period using actual program data from 2016 through 2018 and assumptions about the program for the remainder of the forecast period. The “No Program Scenario “represents evaluators’ best estimate of what the installed stock and market share of equipment technologies will be from 2015 through the end of forecast period in the absence of PA programs to promote efficient C&I lighting. Appendix A provides details on the inputs and sources of data used in the model. The results of the model are available in a Microsoft Power BI dashboard, which allows users to see installed stock of lamps, market share, consumption, and savings over time in both scenarios. The interactive dashboard can show the whole market over time or be filtered to isolate specific market segments at a point in time. In addition to the interactive dashboard, evaluators also provide high-level market share results in Appendix B. Appendix B also provides instructions on how to access the Power BI dashboard, and Appendix C provides instructions for accessing the bibliographic database.

Two Tier Steam Traps Savings Study Memo

Synopsis: This memo presents results of a study designed to generate two prescriptive steam trap repair and replacement deemed savings estimates by leveraging the existing data collected from the Phase 2 Steam Trap Evaluation and establish qualification criteria to be used when assigning the proper savings tier for prescriptive steam trap replacements. A prototype model‐basis approach was used to calculate the deemed savings values, which involves tabulating the various parameters from the custom savings equation and taking median and weighted average values.

Conclusion: Based on the parameter assessment, evaluators concluded that the orifice size, hours of operation, and thermal efficiency have a direct relationship with operating pressure while leak factor (among the other parameters not chosen for assessment) has no relationship with pressure. Evaluators recommend that the PAs adopt a two‐tier approach for prescriptive steam trap savings in 2019 and beyond using the following criteria for applying deemed savings appropriately: If the system operating pressure is ≤15 psig, PAs should claim 8.4 MMBtu/yr for every steam trap repaired or replaced at the facility through the program. If the system operating pressure is >15 psig, PAs should claim 35.6 MMBtu/yr for every steam trap repaired or replaced at the facility through the program.

C&I Impact Evaluation of 2014 Custom CDA Installations

Synopsis: This report presents the results for the Massachusetts Commercial and Industrial (C&I) Impact Evaluation of 2014 Electric and Gas Custom Comprehensive Design Approach (CDA) Installations. CDA is an energy efficiency track within the custom C&I New Construction energy efficiency programs offered in Massachusetts intended to encourage a comprehensive, integrated, whole-building approach. The objective of this impact evaluation is to provide verification or re-estimation of electric energy and demand and natural gas therm savings estimates for a sample of custom CDA projects through site-specific inspection, monitoring, and analysis. The results of this study will be used to determine the gross realization rates for custom CDA energy efficiency projects completed in 2017, as well as prospectively. Realization rates were determined at the statewide level only. The report also presents considerations specific to changes the PAs could make in the delivery of their energy efficiency programs.

Conclusion: Site-level evaluation results of the study were aggregated using the final adjusted case weights. The statewide realization rate is the ratio of the total measured savings to the total tracking savings, each of which is calculated by summing across the PAs. Table 1-1 in the study summarizes the statewide results of this analysis. Based on the data collected, conclusions, results, and process of this impact evaluation, evaluators make several recommendations: Apply the realization rates results from this study (57% electric kWh, 101% gas therms) to adjust the gross savings claimed for CDA projects in program year 2017. Adopt a baseline for LPD that is consistent with current industry standard practice. Adopt the ISPs developed for parking garage exhaust fan, kitchen hood, and laboratory fume hood controls; and foil coil unit EC motors. Re-examine and adjust the realization rate to apply to post-2017 CDA projects to account for the adoption of new baselines/ISPs as well as process improvements made by the program. Improve the commissioning process to ensure ECM installation and intended function.

National Grid Connected Solutions Final Report

Synopsis: This study evaluated National Grid’s Connected Solutions Commercial and Industrial Demand Response demonstration activity over the summer of 2017. The Connected Solutions demonstration is a two year demonstration project for demand response available to Massachusetts C&I customers on a G2 (General Service, Demand) or G3 (Time of Use) rate with demand of 250 kW or higher that are able to curtail at least 50kW of summer demand with day ahead notice. This study had both impact and process research activities intended to provide verification of the proper baseline application and impacts calculated by the AutoGrid system, examine the effectiveness of the Connected Solution baseline, and assess ex-post impacts, and to understand customer acceptance and experience with the intervention, readiness of systems for larger deployment, and PA and vendor success in delivery.

Conclusion: Evaluators found that Connected Solutions is a well-designed demonstration that is developing a qualified contractor network and helping C&I customers control peak demand. The demonstration has a good general design with areas that performed very well. Participants received customized services that fit their unique opportunities. Planned curtailment included a diverse set of end uses and systems. With a few minor exceptions, participant respondents indicated that they curtailed load when asked to do so. There were only a few incidents of employee complaints or disruptions of business operations. Collaboration and communication between National Grid and the CSPs was very good. CSPs appreciated that National Grid staff were open to “on-the-fly tweaks” of demonstration design and delivery. Participants were very satisfied with the Connected Solutions demonstration, though these results are based upon surveys performed prior to participant receipt of incentives. The study also presents recommendations and considerations to improve Connected Solutions’ 2018 demonstration offering.

2016 C&I Customer Profile One-Page Summary

2016 Comprehensive C&I Customer Profile

MA Commercial and Industrial 2011-2016 Mid-size Customer Assessment

Synopsis: This study expands on the Mid-size Customer Needs Assessment completed in 2013, which analyzed mid-size electric accounts in the 2011 C&I Evaluation Database and concluded that mid-size electric accounts presented opportunities for the Massachusetts Program Administrators to increase savings. Since the 2013 study, the PAs have reported that they have taken measures to increase savings for these accounts. The 2017 study documented and evaluated the changes made by the PAs to address mid-size accounts from 2011-2016. The overarching goals of this study were to: provide an evaluation of the impact of steps PAs have taken to improve service to mid-size accounts from 2011-2016, examine changes in performance indicators over that time, and determine whether there is a relationship between changes in PA practices and changes in performance indicators.

Conclusion: Evaluators identified several conclusions based on results of the study. At the statewide population level, mid-size electric and gas accounts are no longer underperforming large and small accounts in terms of contribution ratio, and have outperformed large accounts during some recent program years. However, there are still opportunities for some PAs to improve service to these accounts. PAs have changed their criteria for classifying accounts by size. National Grid, Eversource, and Cape Light Compact have developed specific customer engagement practices based on industry segments, many of which include a lot of mid-size accounts. Other PAs have begun targeting specific industry segments, and have expressed interest in or intent to target additional industry segments. While PA and vendor communication has improved in many ways, some vendors remain confused about PAs’ segmentation approaches and vendor roles. Some vendors could benefit from additional PA support/communication in the areas of engineering, training, and customer engagement. Upstream programs have had a substantial impact on participation of mid-size accounts. The 2013 study recommended that PAs encourage more custom/comprehensive energy efficiency projects to increase depth of savings. The evaluators’ findings indicate a need for further attention to this. In other words, the PAs have a range of opportunities to influence depth of savings by encouraging more custom/comprehensive projects.

Findings of Follow-up Interviews with MA Code Compliance Support Initiative Commercial Training Attendees

Synopsis: In 2017, as part of the ongoing evaluation of the Massachusetts Code Compliance Support Initiative (CCSI), evaluators conducted follow-up in-depth interviews with 40 individuals who had attended one or more commercial code classroom trainings approximately six months earlier. This report summarizes the findings from these interviews. The commercial trainings covered the code based on the 2015 International Energy Conservation Code (IECC) and the stretch code. Half of the forty respondents work as code officials and half work as design professionals, engineers, project managers, and contractors, among others (referred to as “building professionals”). The overall goal of the follow-up interviews was to determine if and how the subjects apply the training to their work.

Conclusion: Based on key findings from the follow-up interviews, evaluators provided several conclusions. They suggest that education is critical to enforcing and complying with the energy code compliance. Lack of education is a key barrier to code compliance. Attendees are spreading knowledge of the energy code by sharing information learned from the training, as well as training handouts, with others, potentially having a positive impact on compliance with the energy code. The CCSI trainings are a valuable means for attendees to stay current with the energy code and industry trends. These trainings are the only energy code education for many attendees. While attendees are generally aware of the CCSI technical support services, including the Mass Save website and hotline (phone/email), few are using them. There is no standard procedure for energy code enforcement throughout Massachusetts. Code officials are eager to gain additional knowledge of onsite inspections and willing to utilize a checklist or follow other procedures. Training attendees are interested in receiving additional handouts and resources from the CCSI program that can be used to achieve compliance with the energy code. Considerations for enhancing the effectiveness of the Initiative are also include in the report.

Deep Dive Report – HVAC Trends

Deep Dive Report – HVAC Trends Addendum – City Level Analysis

Synopsis: This Deep Dive report summarizes trends in HVAC installations from 2011 to 2016. The methodology used throughout this Deep Dive stems from the 2016 Comprehensive C&I Customer Profile report. Based on the analysis plan, the primary objectives of this Deep Dive are to delve more deeply into the details of HVAC, which has been steadily decreasing in gross energy savings and share of savings over the past few years, and to understand the composition of HVAC gross energy savings in terms of measures, customers, and savings year over year.

Conclusion: This analysis yielded the several findings across the electric and gas PAs: HVAC trends in historic participation, savings, and incentives are driven primarily by results from the largest PAs in the electric and gas markets. However, this dynamic plays out slightly differently in the two markets. Across the electric and gas PAs, the Health Care and Social Assistance, Educational Services, and Manufacturing industry sectors play important roles each year. For the electric PAs, it appears that HVAC savings are becoming costlier to achieve (in terms of $/kWh saved); this trend is less apparent for the gas PAs, although 2016 could point toward a similar pattern emerging; while the gas analysis was able to identify where decreases in HVAC participation and savings are occurring across the PAs, industry sectors and end uses, it did not yield answers as to why this trend is occurring. The geographic analysis, provided in the report addendum, allows for the specific targeting of towns where historic participation rates and/or population savings achieved have been low or not identified.

2017

Upstream Lighting Impact Evaluation – One Page Summary

Upstream Lighting Initiative Impact Evaluation Final Report

Synopsis: The primary goal of this impact evaluation is to quantify the electric energy savings and demand reduction attributable to the Massachusetts C&I Upstream Lighting Initiative. This enables the PAs to assess whether the initiative is achieving the expected savings, and to identify any recommendations for improvement. The evaluation also seeks to address the potential issue of over or under-illumination by measuring footcandle levels at the work surface and the general quality of the lighting. This study provides results at the state-wide level using metered data collected from each site. Evaluators have developed savings factors that may be applied retrospectively and to future initiative assumption updates.

Conclusion: See summary document above.

Upstream HVAC Initiative Process Evaluation – One Page Summary

Upstream HVAC Initiative Process Evaluation Final Report

Synopsis: This report presents results of an evaluation of the Commercial & Industrial (C&I) Upstream HVAC/Heat Pump (HP) Initiative (Initiative), which compensates participating HVAC distributors for selling discounted qualifying equipment to C&I customers. The evaluation approach included in-depth interviews with PA program managers and implementation contractors, a best practices literature review and in-depth interviews with non-Massachusetts program managers, market penetration/potential analysis, and in-depth interviews of participating and non-participating market actors.

Conclusion: See summary document above.

Combined Heat and Power Process Evaluation – One Page Summary

Combined Heat and Power Process Evaluation

Synopsis: The overall goals of this evaluation are to identify market barriers facing CHP systems in Massachusetts and to help identify ways that these barriers can be mitigated. The evaluation approach is broken down into three distinct phases: 1. Literature review to identify primary benefits, drivers, and barriers to CHP nationally and to the extent possible within Massachusetts. 2. Interviews with PA implementation staff to better understand the Massachusetts CHP market and to understand the PA perspectives on CHP benefits, drivers, and barriers. 3. Interviews with project developers, participants, and prospective participants (both “drop outs” and “high value candidates”) to better understand their perspectives on CHP benefits, drivers, and barriers.

Conclusion: See summary document above.

2016 PA Differences

2016 PA Differences Summary

Synopsis: Evaluators conducted the PA Differences 2016 project primarily to identify the factors that lead to differences among the Massachusetts Program Administrators (PAs) in the total and depth of savings they achieve. This project is intended to provide information and insights that will be useful for the PAs and Energy Efficiency Advisory Council (EEAC) Consultants (together, stakeholders) to identify best practices. The project builds on a 2015 PA Differences project. The three analyses determined to be of most interest to stakeholders were: analysis of market segment approaches, longitudinal analysis of non-upstream HVAC savings, and longitudinal analysis of new construction.

Conclusion: See summary document above.

C&I Incentivized Share of High Efficiency Equipment Final Memo

Synopsis: This memo presents estimates of the incentivized share of high efficiency equipment purchases. To develop these estimates, evaluators merged data collected during the Massachusetts Commercial and Industrial (C&I) On-site Assessments (P41) and the Market Share and Sales Trends Study (P50) (collectively P41/P50) with the 2011 to 2014 upstream and non-upstream PA-sponsored energy efficiency program tracking data. These data were used to develop estimates of the quantity of certain types of equipment purchased by Massachusetts businesses from 2011 to 2014, the number of rebate-eligible high-efficiency purchases of that same equipment, and the number of rebate-eligible equipment purchases receiving rebates.

Conclusion: Given several data uncertainties, many of the study findings were deemed unreliable. However, select data were found to be credible and should be considered when planning programs. Massachusetts businesses are often buying stand-alone cooling equipment that exceed federal energy efficiency standards and meet or exceed the efficiency requirements of the PA-sponsored programs. Continuing to encourage Massachusetts businesses to purchase higher efficiency through the availability of larger rebates for Tier 2 equipment may help Massachusetts businesses continue to lead the market for cooling equipment. The estimated incentivized share of stand-along cooling, while displaying a high level of uncertainty, implied that most rebate eligible equipment is installed without an incentive. The limited period of the upstream program for stand-along cooling reduces the uncertainty of the program tracking to P41/P50 match and the estimated incentivize share. The program tracking data does imply, however, a substantial increase in rebated stand-alone cooling associated with the upstream program. The increased activity may lead to a higher rebated share of stand-alone cooling in the future.

C&I Gross Impact Evaluation Framework

C&I Gross Impact Evaluation Framework Summary

Synopsis: This report serves as a statewide framework to refine and more fully reexamine and document the approach to be used to determine which Massachusetts commercial and industrial (C&I) impact evaluation studies to undertake, at what level of rigor, and when. This framework represents a fundamental reexamination of traditional impact evaluation methodologies, the changing needs of programs and program administrators, and the role of evaluation in responding to those needs.

Conclusion: See summary document above.

Final Deep Dive Report – Advanced Lighting

Synopsis: Evaluators developed this Deep Dive report as an expansion of the annual Commercial and Industrial (C&I) Customer Profile project. The Deep Dive reporting mechanism is designed to provide more in-depth analysis of questions raised while exploring the data in the C&I Evaluation Database, which currently contains 2011-2015 billing and tracking data. This report provides an in-depth view of the C&I lighting market from 2011-2015. To facilitate this analysis, evaluators added new lighting categories to the C&I Evaluation database; these new categories are used throughout the report.

Conclusion: This report presents advanced lighting analysis results by detailed end-use classifications, project track, and project class.

C&I Injection Molding Machine Market Assessment Baseline Study

Synopsis: This study recommends baseline assumptions and calculation methods for injection molding machines (IMMs)that should be used to compute savings estimates in future gross savings impact evaluations. It identifies industry norms regarding specification practices of hydraulic, all electric, and hybrid IMMs, and it examines how facility type, production process, and product characteristics may influence the baseline assumptions.The study utilized on-site program participant interviews, past survey research, a review of literature, an analysis of previously collected machine consumption data, and series of in-depth interviews to meet its research objectives.

Conclusion: The study identifies key findings related to existing practices, machine selection practice, estimating energy consumption, and market changes. Based on these findings, evaluators make the following recommendations. Future commercial and industrial custom measure impact evaluations should assume that the industry standard practice for the lost opportunity purchase of a new IMM to produce medical parts is an all-electric IMM. Future commercial and industrial custom measure impact evaluations should assume that the industry standard practice for the lost opportunity purchase of a new IMM with less than 200 tons of clamping force is an all-electric IMM. Future commercial and industrial custom measure impact evaluations should assume that the industry standard practice for the lost opportunity purchase for all other new IMMs is a machine that has variable volume hydraulic pumping. Future commercial and industrial impact evaluations should continue the practice of specifying the assumed baseline machine model or models for each project sampled. Future commercial and industrial impact evaluations should utilize the savings calculation framework discussed in section 4.2.5 End use monitoring data to estimate evaluated gross energy savings. Evaluated gross energy savings should continue to be calculated based on the normal production volume and practices found at the time of evaluation. Future evaluations should be prepared for situations where the as-found normal production practice is different than expected at the time of machine selection. This framework should be reviewed and updated if necessary at the conclusion of future impact evaluation studies that include the evaluation of lost opportunity IMM installations.

Steam Trap Evaluation Phase II

Steam Trap Evaluation Phase II Summary

Synopsis: Evaluators completed a Phase 1 steam trap evaluation in 2015, which resulted in an increase in measure effective useful life (EUL) from 3 to 6 years, and a recommendation to initiate a Phase 2 study to develop a methodology for producing more consistent results for custom steam trap savings and a new prescriptive steam trap deemed savings value. This report details findings and conclusions from the Phase 2 Steam Trap Evaluation. The general study approach entailed consulting a variety of industry experts as well as incorporating program participant data and secondary research findings to establish a theoretical basis for a revised savings equation.

Conclusion: See summary document above.

Assessment of the Drivers of Net-to Gross

Assessment of the Drivers of Net-to-Gross Summary

Synopsis: The objectives of the Assessment of the Drivers of Net-to-Gross study were to provide program designers and implementers with an understanding of the factors that drive net-to-gross (NTG) results and to provide recommendations and considerations to inform program design and decisions with the goal of increasing net impacts. The Evaluation Team completed the following two tasks to address these research objectives: literature and prior research review; and re-analysis of prior NTG survey data.

Conclusion: See summary document above.

MA 2013 C&I Custom Process Impact Evaluation

Synopsis: The scope of work for this impact evaluation covered the 2013 Custom Process impact category, which included new equipment and/or control systems and strategies for which energy consumption and savings is primarily driven by a quantified non-weather load, such as tons of production or total hours of operation. The 2013-2015 Plan Massachusetts Technical Reference Manual defines this impact group as “New construction and/or retrofit projects for process system equipment and controls.” The objective of this impact evaluation is to provide verification or re-estimation of electric energy and demand savings estimates for a sample of custom electric projects through: Site-specific inspection, monitoring, and analysis. The results of this study are realization rates for custom process electric energy efficiency measures. Realization rates were determined at the statewide level as well as separately for National Grid, Eversource, Eversource-NSTAR only, and Eversource-WMECO only.

Conclusion: The site level evaluation results were aggregated using the final adjusted case weights. The PA realization rates were estimated and then applied to each PA’s total tracking savings to determine their total measured savings. The statewide realization rate is the ratio of the total measured savings to the total tracking savings, each of which is calculated by summing across the PAs. The results presented include realization rates (and associated precision levels) for annual kWh savings, and on-peak demand (kW) savings at the times of the winter and summer peaks, as defined by the ISO New England Forward Capacity Market (FCM). The evaluation team made the following recommendations based on the data collected, conclusions, results, and process of this impact evaluation. The realization results from this study should be used by the PAs to calculate the adjusted gross savings achieved by program year 2016 projects in this impact category. The results should also be applied to future program years until new realization rates are published. Future commercial and industrial custom measure impact evaluations should include an initial task in each evaluation that reviews and reports what actual project documentation was provided. The results of this review should be provided to the PAs and PAs should be given the opportunity to locate and provide additional files. Future commercial and industrial custom measure impact evaluations should include language in the Stage 3 work plan that clearly defines the PA’s tasks and associated time frame for each task. Future commercial and industrial custom measure impact evaluations should include language in the Stage 3 work plan that clearly defines an agreed standard schedule for each site-specific M&V study.

MA Commercial and Industrial Baseline Framework

MA Commercial and Industrial Baseline Framework Summary

Synopsis: This document articulates a statewide framework for evaluators to consistently characterize the baseline of commercial or industrial (C&I) measure selected for evaluation measurement & verification (EM&V) in an impact evaluation. The guidance applies to both electricity- and natural gas-saving measures, both prescriptive and custom measures, and all Massachusetts (MA) Program Administrator (PA) programs. The focus is on characterization, that is, assessing the appropriate baseline system type as part of an impact evaluation. Judgment regarding hours of use, load factors, and related considerations is not included in this document. The document also does not provide specific technology baselines.

Conclusion: See summary document above.

2015 Comprehensive C&I Customer Profile Report

2015 C&I Customer Profile Summary

The Commercial and Industrial (C&I) Customer Profile report is published on an annual basis to present the analysis of Massachusetts Program Administrators’ (PAs’) billing and tracking data. The analysis and reporting of the statewide data allow the PAs and the Energy Efficiency Advisory Council (EEAC) Consultants to: accurately quantify and report on trends and time series evolution in the Massachusetts C&I landscape, develop narratives about these trends and their implications for a variety of stakeholder interests, and help formulate testable hypotheses for future process, market, and impact assessment studies.

Conclusion: See summary document above.

MA45 Phase 2 Programmable Thermostat Final Memo

Project 45 Report Summary – C&I Programmable Thermostat Evaluation

Synopsis: This memo summarizes the findings of Project 45, the Prescriptive Commercial & Industrial Programmable Thermostat Phase 2 study. The purpose of Project 45 was to determine an accurate natural gas savings estimate associated with programmable thermostats (PTs) installed in commercial and industrial (C&I) buildings in Massachusetts.

Conclusion: See summary document above.

Follow-up with CCSI Commercial Training Attendees

Synopsis: As part of the ongoing evaluation of the Massachusetts Code Compliance Support Initiative (CCSI), evaluators conducted follow-up in-depth interviews (IDIs) with 60 individuals who had attended one or more commercial classroom training sessions approximately six months earlier. The respondents in this study attended training sessions from February 2015 through April 2016. Thirty respondents were municipal building code employees and 30 were architects, engineers, equipment suppliers, or energy efficiency professionals (referred to as “building professionals”). Evaluators conducted the interviews primarily to determine if and how the subjects were applying in the field what they learned in the training. The interviews also explored how the information from the training was being shared, what changes were occurring for code compliance and enforcement, and suggestions provided by the respondents for improving the training.

Conclusion: The report includes interview results related to the most useful information from training and its use in the field, whether participants are sharing information from the training, key sources of information and Mass Save technical assistance, the code compliance and enforcement environment, and considerations for improving the CCSI training. Based on interview responses, the evaluators provide several recommendations. To improve the training and its effectiveness, evaluators make the following recommendations: Improve the quality and content of training handouts and encourage attendees to share training handouts in the workplace; add modules to the training that explore topics beyond basic energy code knowledge and project types; include on-site and hands-on modules; increase marketing of the training to industry groups that are currently underrepresented at the training, such as building contractors and equipment suppliers; and use the training to encourage greater communication between municipal building code employees and building professionals. To improve the technical support services and increase user satisfaction, evaluators make the following recommendation: Implement an effort specifically for marketing the technical support offered by Mass Save; improve the response time of the technical support services; and create a medium for distributing code-related information and topics of interest to industry professionals. Evaluators provide the following recommendations to improve commercial building energy code enforcement throughout Massachusetts: Follow-Up Interviews with CCSI Commercial Training Attendees; develop energy code implementation resources such as checklists for inspections and plan review similar to those available from the U.S. Department of Energy; emphasize to both the code enforcement community and industry the necessity of submitting the documentation required to verify energy code compliance; and document and communicate to code officials the characteristics of code enforcement best practices.

Gas Boiler Market Characterization Study Phase II – Final Report

Gas Boiler Market Characterization Study Phase II Summary

Synopsis: Technological advances in commercial gas boiler equipment continue at a fast pace and have created a complex and quickly evolving market. The objective of the Phase II study was to provide the Program Administrators (PAs) and Energy Efficiency Advisory Council (EEAC) consultants with an understanding of the rate of change in the current baseline efficiency level and provide an analytic comparison to boiler programs in the surrounding Northeast Region. The Evaluation Team (“the Team”) completed the following five tasks to address these research objectives: Task 1-Seek Boiler Manufacturer Input/Comment on Phase I Final Report, Task 2-Comparative Research on Condensing Gas Boiler Programs in the Northeast Region, Task 3-Initiate Boiler Product Line Mapping, Task 4-Provide Overview of US Department of Energy (DOE) Notice of Proposed Rulemaking (NOPR) for Commercial Boiler Standards, Task 5-Conduct Open-Dialog Massachusetts Boiler Roundtable.

Conclusion: See summary document above.

2016

2015 C&I Expedited Customer Profile Report

Synopsis: This report provides results from the exploratory data analysis conducted on the Massachusetts C&I Evaluation Database. To develop the database, the Massachusetts Program Administrators (PAs) provided raw customer billing information and raw energy efficiency program tracking data annually for the years 2011-2015. Evaporators then undertook a data extract, transform, and load process to standardize and integrate each year’s new raw data into the C&I Evaluation Database. These data are continually leveraged, checked, and updated as new material is provided by the PAs or identified by ongoing research projects. As a result of the continual database maintenance and data integration, it is possible numbers provided in this report will shift when new information is captured and integrated.

Conclusion: The study presents combined electric and gas 2015 data tracking tables before the report breaks out into fuel type specific sections. It also presents the tables and figures from previous annual C&I Customer Profiles that break out the details of the electric and gas markets, focusing on the new 2015 program tracking data. The 2015 data is broken down by participation and savings, end uses, industry sector, and within PA summaries. The appendices include the detailed methodology, additional fuel type comparisons, and kW breakdown tables. These detailed sections are provided to give the user greater insight into the numbers presented in the tables and figures throughout this report.

MA C&I Market Characterization Study

MA C&I Market Characterization Study Summary

MA C&I Market Characterization Study Appendices

Synopsis: This study was completed to help the Massachusetts Program Administrators (PAs) and the Energy Efficiency Advisory Council better understand the Commonwealth’s existing C&I building and equipment stock, and to identify the greatest opportunities for expanding the statewide energy efficiency program portfolio and encouraging Massachusetts businesses to make further energy efficiency improvements. To complete the study, the evaluation team conducted comprehensive on-site assessments of C&I buildings associated with 800 electric accounts. The buildings were of 13 different business types, including the most complex facility types in the market—hospitals, college campus buildings, manufacturing facilities, and professional sports venues.

Conclusion: See summary document above. The report summarizes key findings, either as unweighted estimates, estimates weighted by number of businesses (respondent weight), or by kWh usage, and presents high-level results and observations. Findings include those related to C&I customer electric energy usage from lighting, HVAC systems, energy management systems, water heating, refrigeration, kitchen equipment and refrigerated vending machines, and on-site generation.

Commercial Building Department Document Review (Final Report)

Synopsis: This study focused on determining current levels of energy code documentation and identifying factors that may influence documentation levels. The evaluation team conducted a limited scope of data collection to assess the level of documentation associated with energy code compliance for a sample of new construction commercial facilities located in Massachusetts. The evaluation team visited the building departments in six jurisdictions and requested data to determine the type of documentation being filed to show compliance with the Massachusetts energy code for commercial buildings.

Conclusion: Overall, available energy code compliance documentation varied site by site, and jurisdiction by jurisdiction and was limited and incomplete. Based on the limited sample, the buildings in Boston proved to have the most thorough level of documentation overall, with 56 percent of the buildings having COMchecks for the envelope, lighting, and HVAC measures. However, given the small sample size, especially in jurisdictions other than Boston, it is hard to draw a definitive conclusion and extrapolate to all of the new construction within each jurisdiction’s territory, or across the Commonwealth.

2014 C&I Customer Profile

Synopsis: The C&I Customer Profile is an annual publication that analyzes Program Administrators’ (PAs) billing and tracking data in order to identify notable trends and research questions that will help inform and improve the Massachusetts Commercial and Industrial (C&I) energy efficiency programs. The study allows the PAs to evaluate how their standardized data compares to the standardized data for other PAs and the state as a whole, while continuing to preserve PAs’ customer and IT system confidentiality.

Conclusion: Key findings of the study suggest that the availability of project level upstream lighting data had a significant participation ratio impact, particularly for smaller customers. Mid-size electric accounts have contributed a greater share of the savings relative to their share of consumption each year since 2011. Town level consumption-weighted participation over the last four years indicates that PAs have engaged many of the larger customers recently. Electric PA savings increasingly come from a larger population of smaller saving projects. Pre-rinse spray valves are a key, but declining, driver of gas participation. Approximately half of all accounts from 2011–2014 are new accounts to the C&I dataset. Gas PAs engaged 25% of their collective consumption-weighted population in 2014 (up from 20% in 2013). The annual consumption for gas accounts is more deterministic of the total savings that can be achieved for the account than it is for electric accounts. Since 2011, single-served PA towns experience higher gas participation than dual-served PA towns. Large outlier and strategic accounts can have very substantial singe year contributions to meeting goals. PAs all have instances where 2014 savings exceeded the matched accounts 2013 full extrapolated consumption. Custom projects continue to be a key source of gas and electric PA savings. Multi-year participants are small population, but key driver of savings. Evaluators also present recommendations, considerations and potential future research.

Stage 2 Results—Commercial and Industrial New Construction Non-Energy Impacts Study―Final Report

Synopsis: This report presents the methodology and results of the 2015 Non-Energy Impact (NEI) Study of energy efficiency measures supported through the Massachusetts Program Administrators’ (PAs) Commercial and Industrial (C&I) New Construction (NC) Programs. This study was conducted in two separate phases. Findings from Stage 1, which focused on recommending an approach to estimate NEIs, were summarized in a separate document; this current report provides the findings from Stage 2. The purpose of this study was to quantify the dollar value of participant NEIs for C&I NC projects completed in 2013, and to estimate gross NEIs per unit of energy savings resulting from NC electric and gas measures separately. Based on the results of the Stage 1 research, this Stage 2 analysis focuses on the NEIs associated with “true” new construction measures, defined as new buildings/facilities and major renovations.

Conclusion: Results of the study suggest that the total annual value of NEIs for 2013 NC program participants that conducted true NC projects was roughly $488,000 per year, across 957 measures installed in 2013. These results include the Custom – Comprehensive Design Analysis (CDA) performance path-based measure. A breakdown of savings by project track is provided. The report presents recommended unit NEI estimates for each of the measure categories used in the PAs’ BC analysis. For each of the BC measure categories, evaluators show the unit NEI in dollars per kWh or per therm and its statistical significance.

Follow-up Interviews with CCSI Commercial Training Attendees—Revised Draft

Synopsis: As part of the ongoing evaluation of the Massachusetts Code Compliance Support Initiative (CCSI), evaluators conducted follow-up in-depth interviews (IDIs) with 21 individuals who had attended one or more commercial classroom training sessions approximately six months earlier. Four respondents are municipal building code employees and 17 are builders, architects, equipment suppliers, or energy efficiency professionals (referred to as “builders and others”). The interviews were intended primarily to determine if and how the subjects are applying in the field what they learned in the training. The interviews also explored how the information from the training is shared, what changes are occurring for code compliance and enforcement, and any suggestions for improving the training.

Conclusion: The report presents findings related to: use of training information in the field, most useful information from training, sharing of information and recommending training, sources of information in addition to CCSI, code compliance and enforcement environment, suggestions for improving the CCSI training and other comments

2015

Impact Evaluation of Prescriptive Chiller and Compressed Air Installations

Synopsis: This document presents the results of a 2012 and 2013 impact evaluation of Prescriptive Compressed Air (CAIR) and High Efficiency Chiller (HE Chiller) Installations in Massachusetts. The purpose of this study was to provide verification or re-estimation of gross energy and demand savings through site specific inspection, monitoring and analysis. The results of this study will be used to determine the final realization rates for Prescriptive CAIR and HE Chiller energy efficiency measures installed in 2015. Realization rates for CAIR were determined at the statewide level, while HE Chiller results were provided for those PAs that use the TRM methodology and Eversource-NSTAR separately, due to the differences in methods of tracking savings calculations. In addition, the impact evaluation provides new deemed savings estimates, savings algorithms and/or savings factors (such as ELFH) to be used to inform future savings estimates.

Conclusion: Savings from new prescriptive chillers are being realized with a retrospective energy savings realization rate of 104% at the state level. For the larger PAs, retrospective energy savings realization rates were also positive with National Grid at 127% and Eversource at 100%. These realization rates were driven by two factors, increased delta efficiency and an increase in tracking EFLH. In addition, it appears as if the TRM methodology underestimated savings while the Eversource tool provided more accurate savings estimates. Savings from all three prescriptive compressed air measures are being realized. The air compressor measure produced an energy savings retrospective realization rate of 109%. This value was driven by VSD air compressors, which performed much better (113%) than the load/unload air compressors (19%). The high VSD compressor realization rates were primarily the result of higher than anticipated operating hours. The dryer measure yielded an energy savings retrospective realization rate of 316%, which was driven by both a higher average kW reduction per CFM, and higher operating hours. The zero loss condensate drains appear to be installed as expected. Detailed results for prescriptive chillers and compressed air are included in tables in the study as well as a discussion of the application of results and general recommendations.

Impact Evaluation of 2012 Custom HVAC Installations

Synopsis: This document summarizes the work performed between 2013 and 2015 to quantify the actual energy and demand savings due to the installation of 69 custom heating, ventilation and air-conditioning (HVAC) measures installed through the C&I New Construction and Major Renovation and C&I Large Retrofit programs in 2012. The scope of work of this impact evaluation covered the 2012 custom HVAC end use, which includes high efficiency HVAC equipment, HVAC controls as part of Energy Management Systems (EMS), operations and maintenance (O&M) and retrocommissioning of HVAC measures, and building shell improvements that impact HVAC loads. The primary objective of determining realization rates at the statewide and PA end-use level was accomplished by conducting on-site M&V at a statistically selected sample of 69 participant sites from the 2012 program year.

Conclusion: The statewide realization rate for custom HVAC measures was found to be 88%. The relative precision for this estimate was found to be ±7.4% at the 90% level of confidence. The error ratio was found to be 0.47, which is nearly equivalent to the 0.48 found in the prior study, and somewhat better than the estimate of 0.6 used in the sample design for this study. For the on-peak summer kW, the overall realization rate was 88%, with a relative precision of ±12.6% at an 80% confidence level. For on-peak winter kW, the realization rate was a bit lower, at 85%. Summer and winter seasonal peak kW savings realization rates are also provided for use by Eversource-WMECo. Detailed results are included in tables in the study as well as recommendations for improvement.

LED Spillover Analysis

Synopsis: This report summarizes an analysis of potential spillover savings associated with LED lighting programs offered in Massachusetts. Evaluators conducted this analysis in response to high C&I customer adoption of LED products found as part of the LED Market Effects Baseline Characterization study. Building on the baseline study findings, the present analysis sought to: further investigate causes of the higher rate of LED adoption observed in Massachusetts C&I customers; estimate spillover and free-ridership associated with PA programs that support C&I LED products; and research other key market elements such as factors influencing the purchase of LED products, changes in supplier activities to promote LEDs, and customer lighting purchase decision-making processes. The results of this analysis were incorporated into the C&I net-to-gross (NTG) factor used in the 2016-18 three-year plan, after accounting for any possible overlap with other NTG estimates.

Conclusion: Results the customer surveys conducted for this study suggest a high level of spillover, especially among participants. Many of the findings from this study reinforce the conclusion that the non-residential market for LED lamps in Massachusetts was significantly advanced beyond conditions in California and a comparison area with low program activity. Results also suggest that the Massachusetts market continues to develop rapidly. Participant survey results suggest that they were very well aware of LED lamps and were disposed to purchase them, but the program encouraged them to purchase more LEDs and accelerate their purchases. Both participants and non-participants reported that their lighting vendors vigorously promoted LEDs. The customer surveys yielded a number of findings that suggest that participation in energy efficiency programs has a strong cumulative effect on customer response to current programs and to adoption of energy efficiency measures as a whole. Large shares of sample customers identified their vendors and building codes as major influences on lighting selection. Over 70% of all sample customers reported that they purchase the majority of their LED lamps directly from distributors or contractors. By contrast, the sample customers reported that they purchase only 38% of all lamps, such as halogens, CFLs and LEDs, from distributors and contractors. Recommendations are also included in the study report.

Comprehensive Review of Non-Residential Training and Educational Programs with a Focus on Building Operator Certification

Synopsis: This evaluation provides a comprehensive review of education and training programs that may produce savings for the eight Massachusetts program administrators (PAs), with a focus on Building Operator Certification (BOC). The main objectives of the study were to: identify how to increase the number of and share of PA customers who achieve BOC certification with assistance of PA-provided training subsidies; develop a better understanding of the factors influencing BOC savings; and update the MA TRM savings estimate for BOC. To achieve these objectives, evaluation team members analyzed indicators of BOC market penetration, conducted interviews, and reviewed studies reporting BOC savings estimates.

Conclusion: Based on the findings from this study, the evaluation team recommendation that the Massachusetts PAs should: employ multiple channels to promote BOC and the subsidies; craft BOC messaging that conveys the value proposition of certification and maintenance of certification to high-level managers; encourage high-level managers who take the training to also send their operators with day-to-day O&M responsibilities; promote BOC to participants of other energy efficiency programs; claim savings for each subsidized customer for eight years from the initial year of certification; not claim additional savings for an individual’s Level 2 certification beyond those claimed for Level 1 certification; claim two-thirds of the recommended per-operator savings for a second subsidized operator at a given workplace; consider designing and implementing additional adult efficiency education/training programs.

Commercial New Construction Energy Code Compliance Follow up Study

Synopsis: The primary objective of the 2014 Massachusetts Commercial New Construction Energy Code Compliance Follow-up Study was to assist the Massachusetts PAs in the development and implementation of programs that support enhanced code compliance rates and promote “beyond code” design and construction. This study was a follow-up effort to the 2012 Massachusetts Code Compliance Baseline Study (2012 Study), which sought to promote the understanding of the energy code enforcement process, compliance levels, and energy savings potential by evaluating the compliance levels of seventy-five recently constructed buildings in Massachusetts. The research approach for the 2014 ECCF Study consisted of three primary activities: mine the 2012 Study results data; verify energy code compliance at active construction sites across Massachusetts; and support program development for targeted new construction efforts.

Conclusion: Overall code compliance, when weighted by building type and size, is estimated to be 85% using one methodology, or 94% when using another methodology that also assesses the energy impacts of partial code provision compliance. The results of the 2012 and 2014 compliance studies suggest trends in increasing compliance rates over the past 5 years. Additional conclusions regarding commercial energy code compliance rates and practices in Massachusetts are also presented in the study. Recent increases in the stringency of the energy code (especially those in IECC 2015 and proposed for future code versions) suggest that the role of the code in driving building practices is changing. Early energy codes were adopted to prevent poor practices; however, the stringency of energy codes has increased such that it is becoming a tool to move the market towards improving the energy performance of buildings rather than set the minimum standard for buildings. The report also includes recommendations for program development that seek to promote a cooperative effort that increases the effectiveness of both codes and efficiency programs.

C&I Natural Gas Programs Free-ridership and Spillover Study

Synopsis: This study presents findings of the free-ridership and spillover survey conducted for the Massachusetts Program Administrators (PAs) for their 2014–2015 Commercial and Industrial (C&I) natural gas programs. The primary objective of the study was to assist the Massachusetts PAs in quantifying the net impacts of their commercial and industrial natural gas energy efficiency programs that offered incentives directly to customers. These programs include Custom and Prescriptive programs for both new construction and retrofit projects. A secondary objective of the study was to assess how free-ridership varies between Green Communities and non-Green Communities.

Conclusion: The study presents summary tables that include statewide figures for the custom and prescriptive programs and different measure types. Following the summary tables, detailed results are presented for each PA. The detailed results include free-ridership and spillover rates by measure and program type. The statewide free-ridership rate is 17.7 percent, the participant spillover “like” rate is 4.3 percent, and the non-participant spillover rate is less than 1 percent, resulting in a statewide net-to-gross rate (NTGR) of 87.1 percent. Overall, the statewide NTGRs were similar to the 2011 gas study (90 percent) but significantly higher than the 2010 gas study (79 percent). In addition, NTGRs varied dramatically by measure type both between evaluation years and among PAs for a given year.

MA 2013 Prescriptive Gas Impact Evaluation Steam Trap Evaluation Phase 1

Synopsis: As part of its annual scoping effort, the Massachusetts Commercial and Industrial Evaluation Contract (CIEC) Gas Evaluation Team determined that it would be worthwhile to conduct an impact evaluation focused on steam traps in order to benefit the gas program. Savings associated with the steam trap measure, which exists in both the custom and prescriptive gas programs in Massachusetts, has been steadily increasing for three years, and the overall gas savings potential for customers is significant. The primary focus of the research was to identify the best available deemed savings calculation methods and measure lifetime assumptions; however, evaluators also used the opportunity to solicit general feedback on program delivery and other factors. Evaluators supplemented a literature review with information solicited directly from steam trap vendors/manufacturers, and investigated the existence of Massachusetts gas customer facility records that could: 1) provide historical documentation of steam trap replacement, and 2) directly support steam trap measure lifetime conclusions.

Conclusion: Recommendations for immediate implementation include: continue providing prescriptive and customer steam trap programs, and increase measure lifetime from three to six years. Proposed Next Steps for Phase 2 include: Convene a steam trap stakeholder group to coordinate adoption of standardized savings algorithms; develop a new prescriptive steam trap deemed savings value; and leverage the steam trap stakeholder group to identify approaches to increase program participation and savings.

Massachusetts Commercial and Industrial Customer on Site Assessments Interim Report

Synopsis: The Massachusetts Commercial and Industrial (C&I) Customer On-site Assessments and Market Share and Sales Trend Studies (MSST) are intended to collect primary data at non-residential facilities and provide primary research findings to be used by the PAs to inform and expand the Massachusetts energy efficiency programs. The on-site data collection efforts focused primarily on collecting information on the major energy end uses. Lighting, HVAC, and motors and drives remain the dominant sources of savings for the electric programs while HVAC and hot water production are the dominant sources of savings for gas programs. Information on refrigeration systems and compressed air systems was also collected. The on-site data collection effort was divided into two segments. The results for the Final Report will include data from all 800 planned on-sites. Due to the Phase 2 data collection and analysis, the final report results may differ from those presented in this Interim report.

Conclusion: The findings presented in this interim report focus on the highest priority end uses: lighting, HVAC, water heating, and refrigeration. Additional findings are also provided for kitchen equipment, office equipment,
energy management systems (EMS), and on-site generation. Information is presented for the saturation of all of the end uses previously listed while recent purchase information (purchases from 2009-2014) is provided for lighting, HVAC, water heating, and EMS. The document consists largely of tables, graphs, and captions due to the short time frame available to clean and analyze the field data. Additional description and exposition has been provided in numerous sections to highlight some of the more significant and interesting findings. In general, estimates displayed in the tables and graphs are of the following types: business-level weighted estimates; kWh-level weighted estimates, and un-weighted estimates.

Characterization of C&I Supply-Side Market Actors – Scoping Study Final Report

Synopsis: The goal of the Supply-Side Market Actors Project is to characterize the population of supply-side market actors and to develop an infrastructure that facilitates and prioritizes data collection from supply-side market actors for market studies and program process and impact evaluations. Phase I of the project was a Scoping Study that focused on the development of a data collection framework for HVAC supply-side market actors. It assessed the feasibility and efficacy of using Program Administrator (PA) program databases, commercially available business data sources, and targeted in-depth interviews to identify the relevant supply-side market actors, develop market actor sample frames, and implement market actor panels to support ongoing C&I Evaluation Contractor (CIEC) data collection activities.

Conclusion: This report presents specific findings and recommendations from the Phase I Scoping Study with respect to the following issues: building sample frames that effectively “represent” the population of each type of supply-side market actor involved in HVAC equipment in Massachusetts businesses and institutions; identifying the trade allies that are involved in the HVAC supply chain, including production, distribution, specification, and installation of HVAC systems and equipment; identifying data collection strategies that maximize the quality of information collected by coordinating sampling and prioritizing data collection across program sectors and business types; implementing survey panels where appropriate; and furnishing information on the most effective strategies for tracking HVAC equipment sales by type and efficiency level.

2013 C&I Customer Profile Report

Synopsis: The C&I Customer Profile is an annual publication that analyzes Program Administrators’ (PAs) billing and tracking data in order to identify notable trends and research questions that will help to inform and improve the Massachusetts Commercial and Industrial (C&I) energy efficiency programs. These studies seek to identify where C&I energy efficiency savings and participation are occurring, and what segments remain to be served. This 2013 study builds on the analyses of prior Customer Profile studies to identify new trends in the data and to verify patterns over time.

Conclusion: High level results of the study suggest that there are several industry sectors where both the account participation and consumption-weighted participation ratios are low. Gas PA participation continues to increase, but at a faster rate than population savings achieved. Large multi-year participants make up a sizable proportion of electric PA savings. Smaller PAs continue to have greater volatility in participation to savings metrics. Larger PAs have higher consumption-weighted market penetration rates, particularly for gas, than the smaller PAs. There does not appear to be an overall difference in gas participation for towns with the same gas and electric PA versus different gas and electric PAs; however there are sizable variations within the different electric and gas PA combinations. Low load factor customers continued to have the highest participant savings achieved, but at reduced levels from the previous two years. New analysis showed that load factor is not a strong determinant of participation. For smaller PAs, there is a shift in savings contribution relative to consumption when compared to the larger PAs. Evaluators also identify opportunities to go beyond the findings of this report and explore the underlying drivers of these trends.

C&I New Construction Non-Energy Impacts Study – Stage 1 Results and Stage 2 Detailed Research Plan

Synopsis: The purpose of this study was to quantify participant non-energy impacts (NEIs) associated with the 2013 Commercial and Industrial New Construction program. The Evaluation Team is conducting the study in two stages. Stage 1 of the project consisted of data mining of program data and previous NEI research and in-depth interviews with various market actors to assess the most effective means of obtaining information necessary for computing NEIs associated with NC measures. Stage 2 will estimate NEIs associated with new construction.

Conclusion: This report presents the results of Stage 1 of the evaluators’ analysis—which consisted of researching and examining appropriate methods—and describes the detailed research plan for quantifying NEIs in Stage 2. Conclusions drawn from Stage 1 suggest the analysis of NEIs associated with new construction measures should focus on true new construction only. Also, self-reports by end users would not provide an effective means for estimating NEIs associated with most new construction measures. Self-reports by engineering firms will provide valuable insights to estimating NEIs across the range of projects for which they perform engineering services. An engineering-based approach is warranted to estimate NEIs. Various individuals may be able to serve on a Delphi panel to provide valuable information regarding NEI estimates. A limited survey effort may be suitable to select measures. The proposed Stage 2 methodology consists of in-depth interviews and Delphi-panels, analysis of existing data and scenario analysis, and an engineering review to estimate NEIs.

C&I Upstream Lighting Program – In Storage Lamps Follow-up Study

Synopsis: The Massachusetts Bright Opportunities Program uses upstream incentives to buy down the cost of energy efficient lighting technologies at the lighting distributor level. In February 2014, evaluators completed a Year 1 impact evaluation of the program, which included 81 on-site visits. One important finding of this effort was that a significant number of program bulbs were actually in storage, as opposed to being installed. This follow-up study, referred to as the Year 3 impact evaluation, was designed to re-visit sites that were found to have in-storage bulbs to investigate when and whether these bulbs were eventually installed, and to calculate savings from bulbs moved from storage to sockets.

Conclusion: Overall, this Year 3 study resulted in a small increase in program savings as compared to the Year 1 evaluation. The LED kWh realization rate increased from 101.9% to 103.4%, while the fluorescent realization rate increased from 89.0% to 92.4. These increases were the result of installation rate increases in both groups. Evaluators also provided recommendations and conclusions regarding LED and fluorescent savings assumptions, and future impact evaluations of the Bright Opportunities Program. For prospective application of program results, evaluators recommend that the PAs utilize the Year 3 results for annual energy and connected kW realization rates and installation rates for both LEDs and fluorescents. The delta watts estimate resulting from this study was not significantly different from the Year 1 study. Based on lighting logger data at each of the sites, the average hours of use for LED lamps were found to be approximately 3,901 hours per year and 3,410 for fluorescent lamps. The PAs and EEAC may want to consider a follow-up impact evaluation to assess the effectiveness of efforts that have gone into improving the installation rate.

C&I Evaluation – Learning from Successful Projects Final Report

Synopsis: Among the projects incented by the Mass Save C&I initiatives, some may experience higher levels of success than others. While some variance is to be expected, it is important to understand the drivers of success so that these effective practices can be generalized and duplicated elsewhere. This study seeks to increase energy savings across MassSave C&I energy efficiency projects by: developing definition(s) of what constitutes project success; using a variety of data to identify projects that meet the criteria of a successful project; identifying the factors that contributed to a project’s success, trends and indications of replicability or uniqueness; and recommending approaches to generalize and duplicate factors that contribute to project success. Evaluators used in-depth interviews with PAs, and EEAC consultant, and C&I customers; and analysis of customer billing and program tracking data to draw conclusions.

Conclusion: Based on the program success factors identified in the study, the evaluators provided recommendations for expanding the occurrence of successful projects. These recommendations include: leverage trade ally customer relationships to increase customer engagement and communication; increase emphasis on vendor training; promote and leverage incentives; explore ways for customers to build internal expertise and capacity to manage projects; emphasizing the value of NEBs and “Being Green”; ensuring the accuracy of technical review and assistance; leverage the results of EM&V site reports; focus on eliminating project delays and intrusions; and have small PAs adopt a simpler form of the MOUs used successfully by larger PAs.

How PA Differences Affect C&I Outcomes – Phase 2 Final Report

Synopsis: This report examines how differences among PAs affect the outcomes of commercial and industrial energy efficiency programs. Massachusetts has achieved substantial standardization of efficiency programs across the eight participating PAs but with different levels of savings and costs. The purpose of this study was to provide a better understanding of the factors that are driving differences in program performance.

Conclusion: The report summarizes several key findings from specific reach questions related to differences among PAs. Based on these findings, the evaluators offer recommendations: small PAs should consider how to increase technical expertise relevant to their largest customers and strike long-term efficiency deals with their largest customers, perhaps in the form of MOUs; whenever possible, comparisons between PAs should be based on multiple years of data and focus on medium- or long-term trends; large and small PAs should attempt to get greater savings from the small and mid-sized customers; expanded use of subcontractors could increase PA reach to smaller customers; targeted initiatives can make a difference. The report also includes recommendations for future research.

Massachusetts Commercial Real Estate Survey Analysis – Final Report

Synopsis: The primary objectives of the study were to provide a comprehensive understanding of the complex relationships between building owners, property managers, and tenants; and to identify specific program offerings and points in the property sale and leasing processes that offer opportunities to capture energy efficiency savings. The study analyzes data from the C&I Customer Survey, which collected information from 943 C&I customers across 11 building types.

Conclusion: The report summarizes key findings related to differences between CRE and non-CRE businesses, physical building characteristics, equipment characteristics, ownership and tenant structure, and business practices. Based on these findings, the evaluators provided recommendations to improve the energy efficiency program offerings to Massachusetts CRE customers. Recommendations include: increase outreach to building managers and owners; leverage the role of account managers; and target marketing to CRE businesses based on building vintage. The report also includes recommendations for future research.

Small Business Program Process Evaluation Final Report

Synopsis: This report presents findings of a process evaluation for the Small Business program, now a sub-program/initiative of the Retrofit Program, which provides efficiency services and incentives to small commercial and industrial customers. Elements covered by process evaluation include program design, administration, delivery, and outcomes in 2012 and 2013. The evaluators completed several major research activities during this evaluation including staff and vendor interviews, vendor ride alongs, customer surveys, documentation review, and data mining.

Conclusion: The report summarizes key findings related to program processes, recent program changes, comprehensiveness, savings and cost effectiveness, and other findings. Based on these findings, the evaluators provided recommendations for ways to increase program savings through wider participation and greater comprehensiveness, and how to maintain or increase cost effectiveness. Recommendations included improvements to the contracting process; measure list, checklist, and assessment process; and data handling.

C&I Electric Programs Free Ridership and Spillover Study

Synopsis: The purpose of this study was to assess program free-ridership and spillover for the electric programs offered by National Grid, NSTAR, Western Massachusetts Electric Company (WMECO), Unitil, and Cape Light Compact. These programs include both Custom and Prescriptive programs for both new construction and retrofit projects. A secondary objective of the study was to assess how free-ridership varies between Green Communities and non-Green Communities. To accomplish the study objectives, telephone surveys were conducted with 2013 program participants in each of the PA’s C&I electric programs and with design professionals and equipment vendors involved in these 2013 installations.

Conclusion: The statewide free-ridership rate is 11.9 percent, the participant spillover “like” rate is 6.9 percent, and the nonparticipant spillover rate is 1.2 percent, resulting in a statewide net-to-gross rate (NTGR) of 96.1 percent. Overall, the statewide NTGRs were relatively stable among 2010 and 2013 participants (96 and 98 percent, respectively). However, NTGRs varied dramatically by end use both between evaluation years and among PAs for a given year. Two factors driving this variability that we were able to observe were: 1) the categorization of measures into end uses varied between PAs and over time, and 2) some end uses and PAs had a small number of participants that make the estimates more volatile. End uses such as Process, Motors & Drives and Lighting were the most stable across the two evaluation years with NTGRs above 90 percent. PA-specific results are also presented.

2014

Prescriptive Gas Pre-Rinse Spray Valve Measure Impact Evaluation

Synopsis: This report presents an evaluation of the Massachusetts Commercial and Industrial Pre-Rinse Spray Valve (PRSV) measure offered through the prescriptive gas program. The research objectives of this impact evaluation included providing new deemed savings value recommendations and making observations based upon actual pre-post site level monitoring performed and integrating PRSV user surveys.

Conclusion: Overall, the pre-rinse spray valve program that is implemented by direct installation contractors is successfully delivering energy and water savings in Massachusetts. The annual savings associated with the spray valve measure of a sample of 39 sites monitored in Massachusetts (33) and Rhode Island (6 ) was calculated as 114 Therms per year. The average calculated water savings per spray valve change-out is 6,410 gallons per year. Survey responses from interviews conducted with spray valve users and facility owners during the site monitoring were positive for the change-out program as were opinions toward the performance of the new high efficiency valves being utilized in the program. A wide variation of calculated savings stems from dissimilarity in dish/pot washing within the food service population of the commercial sector.

C&I Year 2012 Customer Profile Final Report

Synopsis: This study built on the previous analysis conducted in the 2011 C&I Customer Profile study, and involved the collection, organization and analysis of energy efficiency program participant data and billed usage data for all Massachusetts C&I electric and gas customers served by the PAs. The principal goals of the Enhanced C&I Customer Profile project are to: update the previous customer profile characterization report and associated database to incorporate the 2012 participant and billing data; evaluate previously unanalyzed time-series, geographic, and measure interaction trends within the participant and billing database; explore the feasibility of constructing a statewide customer-level (across fuels and PAs) database of program participant, usage data and third party data.

Conclusion: The key population level findings are presented first at the population level by PA, then at the population level across PAs looking at business type, demand size , project end uses by fuel, year over year repeat customers by PA, and then the geographic trends at the PA’s town level service assignments. Chapters 7 and 8 present within PA snapshots detailing how the PA specific data breaks out. Conclusions from the analysis and reporting phases are offered, along with data considerations encountered in the analysis and recommendations for improving the assessment of data for future customer profiles. Results of the study yielded several conclusions and recommendations including: Investigate the geographic data at more detailed granularity; investigate in greater depth why load factor appears correlated with savings; investigate methods to improve PA specific match rates using PA supplied ID data.

Lighting Retrofit Control Measures Final Report

Synopsis: The goals of this research are to provide the Massachusetts Program Administrators (PAs) with recommendations to improve the lighting controls options through the retrofit program, tracking methods, and to determine how lighting controls measures should be evaluated. The results of this research include the following core objectives: discover why program savings for the retrofit lighting controls market dropped off to about half its size between 2010 and 2011, and whether the program can reverse this decline; determine what kind of impact evaluation to conduct for Large C&I Retrofit Lighting Controls installations under MA-Large Commercial and Industrial Evaluation Contract (LCIEC) -study 22; make recommendations for changes to future lighting controls measures to account for new market conditions, including how to track savings consistently; and make recommendations for adjustments to savings estimation methods currently in use in the Massachusetts Technical Resource Manual (TRM).

Conclusion: Tracking data revealed a decline in retrofit lighting controls savings in the Large C&I program between 2010 and 2011 of about one half. The 2012 program year saw an increase of approximately 8% over the 2011 program year. However, 2013 saw large C&I lighting controls savings decrease to their lowest levels since 2010. Based on the program tracking data, it is not clear whether the reduction in lighting controls savings reflects a market shift, a slowdown in this economic sector, changes in program planning, or other factors. Some hypotheses of what may be driving the decline in savings are presented. Findings from a literature review conducted as part of the study suggests that the current market saturation for lighting controls is low but has more potential, and substantial interest is growing in the market for wireless and integrated controls. Interviews with program implementation staff, and lighting controls vendors highlighted some technologies in which the program may focus on in the future. These include: advanced/networked lighting controls, wireless controls, LED lighting and controls, and daylight dimming. Sectors in which the program may benefit from focusing more in terms of lighting controls opportunities include offices and small business. Recommendations are also provided for future program marketing rebate opportunities, and methods for calculating prescriptive lighting controls savings.

Office Buildings Market Sector Profile Final Report

Synopsis: This market sector profile is provided as part of the Massachusetts Existing Buildings Market Characterization study. It provides an overview of the office building market using industry information and the Massachusetts Commercial and Industrial (C&I) Telephone Survey data collected between September and December 2013. The purpose of the market sector profile is to: provide the Massachusetts Program Administrators (PAs) and Energy Efficiency Advisory Council (EEAC) Consultants with a more detailed understanding of the office building industry; identify key customer data and show the distribution of building, decision making and equipment characteristics across customer size segments and building uses; and identify key potential energy efficiency opportunities or areas that warrant further investigation using primary and secondary data.

Conclusion: The report presents data gathered on building, customer, and equipment characteristics of the office building market. It also provides recommendations for overcoming barriers associated with net leasing agreements for the PA’s to consider. These recommendations include: engaging tenants to make them more aware of energy use and offer suggestions for improving the energy culture; working with owners to incorporate energy efficiency into retrofits or tenants fit-out; and promoting “energy-aligned” leasing structures wherein both parties benefit from energy upgrades. Other strategies to consider when working with office type customers are: target retrofits for major energy-using equipment; continuous improvement through Operations & Maintenance (O&M); adopt a targeted EE approach to reflect the divergence in office building outlooks; During major retrofits and fit-outs work with customers to ensure HVAC equipment is properly sized by providing sizing guidelines as well as O&M plans to ensure the system continues to operate optimally in the future; and speak to real-estate professionals in their own language.

Small & Medium Healthcare Market Sector Profile Final Report

Synopsis: This market sector profile is provided as part of the Massachusetts Existing Buildings Market Characterization study. It provides an overview of the small and medium healthcare industry (not including large hospitals) using industry information and the Massachusetts Commercial and Industrial (C&I) Telephone Survey data collected between September and December 2013. The purpose of the market sector profile is to: provide the Massachusetts Program Administrators (PAs) and Energy Efficiency Advisory Council (EEAC) Consultants with a more detailed understanding of the small/medium healthcare industry excluding large hospitals; identify key customer data and show the distribution of building, decision making and equipment characteristics across customer size segments and building uses; and identify key potential energy efficiency opportunities or areas that warrant further investigation using primary and secondary data.

Conclusion: The report presents data gathered on building, customer, and equipment characteristics of the health care market. It also presents opportunities the PAs may consider to increase savings from energy efficiency programs in small to medium sized healthcare customers. These opportunities include: advanced lighting for assisted living facilities; linking energy cost savings to revenue generation; targeted outreach to – customers on HVAC improvements; targeted outreach of occupancy sensors to medium sized customers; and promoting new lighting technologies to executive and senior level administration.

Small & Medium Sized Food Stores Market Sector Profile Final Report

Synopsis: “This market sector profile is provided as part of the Massachusetts Existing Buildings Market Characterization study. It provides an overview of small and medium-Sized Food Stores using industry information and the Massachusetts Commercial and Industrial (C&I) Telephone Survey data collected between September and December 2013. The purpose of the market sector profile is to: provide the Massachusetts Program Administrators (PAs) and Energy Efficiency Advisory Council (EEAC) Consultants with a more detailed understanding of small and medium size food stores; Identify key customer data and show the distribution of building, decision making and equipment characteristics across the customer size segments and building uses; and identify key potential energy efficiency opportunities or areas that warrant further investigation using primary and secondary data.”

Conclusion: The report presents data gathered on food store building, customer, and equipment characteristics. It also presents opportunities the PAs may consider to increase savings from energy efficiency programs in the small to medium sized food store sector including: Retro-Commissioning for HVAC and refrigeration systems; continue to capture opportunities for LED in refrigerated cases; enhance marketing strategies; Develop marketing materials based on case studies; and take advantage of business expansion plans.

T12 Phase Out Market Research Final Report

Synopsis: This report sought to answer whether lighting manufacturers are still producing T12 lamps despite the phaseout initiated by the EPACT and EISA legislation. If the answer is yes, how are manufacturers producing T12 lamps that meet the new efficacy and how big a market do these continuing T12 sales represent? Of secondary interest was the question of how lighting distributors/contractors are reacting/adjusting to the T12 phaseout. Additional data collection efforts included: collecting information relevant to the T12 phaseout from the C&I customer site visits that are being planned by the Existing Buildings Market Characterization – C&I Customer Survey (Project 21) team and its successor the Existing Buildings On-site Assessments (Project 41) team; and collecting information from Massachusetts retailers who sell linear fluorescent lamps.

Conclusion: The report suggests that the large majority of the Massachusetts lighting distributors and the store managers participating in the Massachusetts residential lighting program reported being aware of the T12 phaseout. All the lighting manufacturers and retail buyers who participated in the Massachusetts residential lighting program and who reported selling linear fluorescent lamps and being familiar with these lamp sales, said that they were also aware of the T12 phaseout. The information on the awareness of the T12 phaseout among the Massachusetts C&I customers is not yet available. Findings are also presented on the nature of the current T12 market, the size of the current T12 market, the type of T12 lamps being sold, and other impacts of the T12 phaseout.

Upstream Lighting Impact Evaluation Final Report

Synopsis: This report documents the impact evaluation of the Massachusetts Upstream Lighting Program, which is known as the Bright Opportunities Program. This report represents the impact component of a broader evaluation effort, which also included a process evaluation completed in June 2013. The research objectives of the impact evaluation involved updating the following assumptions with Massachusetts-specific research: application of purchased lamps by facility and space type; hours of use of purchased lamps; baseline replaced lamps for estimating delta watts; gross savings realization rates to be applied to 2012 results; and estimates of delta watts and hours of use to be applied prospectively. The scope of work of this impact evaluation covered upstream lighting purchases made between November 2011 and April 2012, and also between October 2012 and November 2012.

Conclusion: The adjusted gross energy savings and connected kW demand reduction are presented with their associated realization rate and relative precision for each lighting measure. Adjustments were made related to documentation, technology, quantity, operational, HVAC interaction. Overall, the Bright Opportunities program appears to be successfully delivering energy savings, especially with respect to the LED category. LEDs were found to have a realization rate of 102%, which was driven by several savings adjustments. Fluorescents were found to have a realization rate of 89%, which was primarily driven by the quantity adjustment. Based on the results of this study, it is recommended that realization rates for connected kW and kWh, and adjusted savings estimates for hours of use should be applied at the category level (LED and FLR). This study does not have enough data points to disaggregate results at the building type or LED lamp type level with acceptable estimates of precision. Other conclusions and recommendations for the program, and future evaluations of the program are also provided.

C&I Boiler Market Characterization Study Final Report

Synopsis: This study sought to provide the PAs and EEAC consultants with an in-depth understanding of the current baseline in the C&I boiler market and historical trends regarding equipment efficiencies and sizes as well as assess the remaining savings potential. Specifically, the study sought to: characterize the current boiler market in Massachusetts; compare the MA market data collected to the PA’s tracking data bases to determine market share affected by the PA’s EE programs; and identify differences in market share between Massachusetts and other states in the northeast. The study included a series of qualitative and quantitative tasks to address the research issues identified.

Conclusion: The study yielded several key findings relative to the MA C&I boiler market. While the total size range of boiler sales is relatively wide, additional research tasks can be conducted to provide a more finite estimate of sales. The state-wide small commercial boiler inventory is approximately 121,040 units, with 28% of the inventory installed “side-by-side” with one or more boilers at the same site. The availability of natural gas distribution infrastructure limits the sales of high efficiency boilers which are growing at a faster rate than standard and mid-efficiency boiler types. The boilers that went through the prescriptive program in 2012 represent a participation rate of 28 – 75% of all condensing boilers sold state-wide in 2012. Boiler manufacturers responded favorably about the prescriptive boiler program. There are significant differences between the interview responses of boiler manufacturers, which are largely reflective of the types of boilers that they produce. The study also describes next steps that will enable program administrators to better understand the evolving boiler market in Massachusetts, more closely define program participation rates, calculate market lift and justify boiler incentive continuation.

C&I Existing Buildings Market Characterization Telephone Survey Final Report

C&I Existing Bldg Survery – Appendix OA Office by Size

MA Existing Bldg Survey Results Food Stores Data

MA Existing Bldg Survey Results Healthcare Market Data

MA Existing Bldg Survey Results Office Market Data

Synopsis: The Existing Buildings Market Characterization Commercial and Industrial (C&I) Customer Survey was initiated to obtain a deeper understanding of the existing C&I building market in Massachusetts and to help the Energy Efficiency Program Administrators (PAs) continue to shape and expand their portfolio of electric and gas energy efficiency programs. The MA C&I Customer Survey was administered to 943 customers and was designed to collect information on building uses, owner/tenant relationship, square footage, primary heating fuel, cooling systems, recent energy related improvements, and general purchasing practices.

Conclusion: Analysis of survey responses identified aspects of the PA’s marketing and program offerings that could be modified to increase participation and program savings. Key findings are presented related to customer practices, building improvements, and equipment. Considerations for each of the findings indicated areas for future improvement. Continued engagement with both participating and non-participating customers is critical to increase the percentage of customers engaging in program services when making energy related improvements. PAs should continue to make financing available for interested customers as one of the tools to increase program participation, but realize that program-sourced financing will have limited market acceptance and therefore should not be a primary program focus. The PAs could play a role and should continue to explore opportunities to develop and promotes ‘green’ leases as well as energy use benchmarking that incorporates energy efficiency to evaluate the building. PAs should aggressively pursue opportunities to take full advantage of the increase in major renovations among warehouses, lodging and retail respondents within the past three years. Smaller customers were less likely to be aware of incentives and utility programs in general, and less likely to have received financial incentives. Less awareness of incentives and utility programs among smaller customers highlights opportunities for the PAs to build on existing efforts to strengthen their relationships with smaller C&I customers. Continue to pursue and conduct follow-up with program participants at key intervals to solicit interest in any projects. Opportunities exist for energy efficiency through more efficient cooling strategies. PAs should continue to educate customers and ensure they understand the types of equipment installed in their facility and how to visually distinguish more energy efficient linear fluorescents from less efficient types.

C&I Customer Profiles Report Public Webinar on September 16, 2014

Synopsis: This PowerPoint was presented as part of a public webinar on results from the 2012 C&I Customer Profile Report and offers a high level overview of the findings. The purpose of the report was to provide insight into the compiled electric and gas tracking and billing evaluation datasets. The report builds on the base 2011 Customer Profile Report; evaluates time-series, geographic, and measure interaction trends; and identifies areas of interest for deeper analysis.

Conclusion: Results of the study indicated that mid-size customers represent an opportunity for increased savings. Savings by building type varies year over year, though industrials continue to provide a key source of savings. Lighting projects also remain a key source of electric savings while HVAC remain a key source of gas savings. Within PA territories, there is great variation in savings and participation. Custom projects drive savings, prescriptive offerings participation.

2013

Mid-Size Customer Needs Assessment Public Webinar on April 3, 2014

Synopsis: The research goal of the study was to understand the extent to which current program offerings effectively serve the needs of mid-size C&I customers, and whether new program offerings or variations of existing program offerings would better serve mid-size customers. The study provides an extensive analysis of the segmentation and marketing strategies of C&I customers by customer size, as defined by the electric demand. Evaluators used a combination of in-depth interviews, data mining, and customer surveys.

Conclusion: Results of the study suggests that the PA’s use a variety of techniques to segment and market to C&I customers by size, though demand is the most common criteria. Mid-size customers have neither higher participation rates of large customers nor high savings ratios of small customers, which suggests a possible gap in service. Mid-size customers require more comprehensive solutions but face a number of case flow and technical constraints that limit the effectiveness of existing programs to meet their needs.

Mid-Size Customer Needs Assessment Final Report

Synopsis: This report examines the extent to which current program offerings effectively serve the needs of Mid-size customers, and whether new program offerings or variations of existing program offerings would better serve these customers. Stage I of the research process included interviews with program staff and implementation contractors and combined interview results with results from MA-C&I Customer Profile Project to segment customers for analysis. Stage II included billing and tracking data analysis, a survey of participants and non-participants in the small, mid-size, and large customer size segments, and in-depth interviews with specific customer sub-segments and market actors.

Conclusion: The report describes conclusions and recommendations for PA and EEAC consideration regarding mid-size customer needs. Recommendations include: Increase recruitment and training of energy services firms able to provide comprehensive solutions; Develop a statewide process for qualifying and coordinating energy services firms to provide comprehensive solutions; Lower capital and administrative costs for mid-size customers and/or contractors to improve payback and margin on energy efficiency investment; Increase multi-measure (comprehensive) program offerings; Continue to improve marketing strategies for mid-market; Improve processes for linking multiple accounts to customers and across fuel type; Support energy services firms by obtaining qualifying information; Standardize approaches to classifying and marketing to multi-account customers Link electric and gas customers.

Combined Heat and Power Program Impact Evaluation

Synopsis: The primary purpose of this impact evaluation was to quantify energy savings and realization rates for systems installed under the Massachusetts (MA) Combined Heat and Power (CHP) Program. This evaluation also identified and quantified the drivers of CHP performance. Initiated in 2010, the MA CHP Program is entering its fourth year of operation. Twenty-five CHP systems were deployed and commissioned through the program by the end of November 2012. Generation technologies deployed in the Program consisted of internal combustion (IC) engines, pressure reduction turbines, microturbines and gas turbines.

Conclusion: The primary drivers of the program’s electricity and natural gas impacts were CHP system efficiencies (electrical efficieincy and heat recovery rate) and utilization. Overall, systems produced 90 percent (nearly 30 GWh) of the expected annual electrical energy. However, projects consumed 12 percent more natural gas than expected because heat recovery rates were lower than expected. Peak demand savings were not claimed for some sites.However, actual savings were realized for these sites which increased program level demand realization rates. The report also includes some recommendations for the program moving forward.

Customer Profile Project Revised Final Report

Synopsis: The principle research objectives of this study were to characterize the Massachusetts energy efficiency market by analyzing recent customer usage and program participation data, develop a single database to provide a consistent source of program tracking and billing data to support ongoing evaluation efforts, estimate the extent to which customers of various sizes and types participated in energy efficiency programs during 2011, and document the processes used to consolidate and normalize PA data, and recommend enhancements to tracking systems to improve accuracy of results.

Conclusion: The overall participation rates of all C&I customers in gas and electric programs were about 2.7% and 2.2% respectively. These 2011 projects resulted in a reduction in energy use of about 2% for electric customers and 1% for gas customers. When the population is segmented by size, it becomes clear that larger customers are more likely to participate than smaller. Overall, the average customer saved about 9.3% of their annual consumption; this result was consistent across fuel types. In the electric sector, lighting dominates both the number of projects and savings. Refrigeration and HVAC each represent about 10% of the projects, but HVAC projects are clearly larger, since they account for 18% of the savings. A small number of Combined Heat and Power (CHP) projects contribute 24% of the electric savings. For gas projects, most 2011 savings came from HVAC. While low load factor customers participate at the lowest rate (3.5%), the savings achieved is higher than the other load factor sectors, indicating that their average savings are higher. The participation rate for gas customers with the same electric PA is 2.6%, which is higher than the 1.6% participation rate for those with different electric PAs. More savings may exist in the education and healthcare sectors.

Impact Evaluation of 2010 Prescriptive Lighting Installations Final Report

Synopsis: The objective of this impact evaluation was to provide verification or re-estimation of electric energy and demand savings estimates and new savings factors for Prescriptive Lighting retrofit and new construction projects through site-specific inspection, monitoring, and analysis.

Conclusion: This report presents realization rates for gross energy savings and savings factors at the statewide level using 12 months of metered data collected from each site. It also provides realization rates for on-peak and seasonal summer and winter demand savings. A listing of all results and savings factors with descriptions is presented in Appendix A, which evaluators suggest should be applied to future Technical Reference Manual (TRM) updates. This report also provides a comparison of the 12 month metering results to the interim results that were based on approximately three months of metering in the winter of 2011/2012. Results are listed for Lighting Systems, Lighting Controls, Advanced Lighting Design (Performance Lighting), and Refrigerated Case Lighting (LED).

Impact Evaluation of 2011 Custom Gas Installations

Synopsis: This document summarizes efforts to quantify the actual energy savings due to the installation of Custom Gas measures installed through the Massachusetts Energy Efficiency Program Administrator’s (PAs) Commercial and Industrial (C&I) Lost Opportunity and Large Retrofit programs in 2011. A desk-review of 90 PY2011 projects assessed whether recommended process improvements had taken sufficient root to warrant a third consecutive impact evaluation. M&V was done for a sample size of 16 NSTAR sites.

Conclusion: Overall, the Custom Gas program is successfully providing energy savings and program administrators significantly increased program participation and savings when compared to the 2010 program year. Tracking savings increased from 4.4 to 8.0 million therms, while the number of unique accounts increased from 330 to 458 from 2010 to 2011, respectively. The results of savings estimation benchmarking indicate that only NSTAR project files showed evidence of a sufficient improvement (or degradation) in savings estimates to warrant proceeding to an on-site impact evaluation. The desk review task is described in detail in Section 3 of the report. NSTAR’s realization rate did indeed improve from 47% to 84%, and this difference is statistically significant. Program implementation and evaluation recommendations for improvements are also described.

MA Custom Gas Evaluation Site Specific Measurement and Verification Report

Synopsis: This document is an appendix of the Impact Evaluation of 2011 Custom Gas Installations. It includes detailed site reports for a sample selection of 2011 Custom Gas applications, which were used to draw conclusions in the full report.

Conclusion: This appendix presents site level impact evaluation results for 2011 Custom Gas Installations.

Impact Evaluation of 2011 Custom Refrigeration, Motor, and Other Impact Evaluation Final Report

Synopsis: This document is an appendix of the Impact Evaluation of 2011 Custom Refrigeration, Motor and Other Installations. It includes detailed site reports for sample selection of 2011 Custom RMO applications, which were used to draw conclusions in the full report.

Conclusion: This appendix presents site level impact evaluation results for 2011 Custom Refrigeration, Motor and Other Installations.

Large C&I Yr. 2011 Custom Refrigeration Motor and Other Impact Evaluation Appendix A

Synopsis: The objective of this impact evaluation was to provide verification or re-estimation of electric energy and demand savings estimates for Custom Refrigeration, Motor and Other (RMO) measures installed in 2011 through site-specific inspection, monitoring, and analysis. The Custom RMO measures were installed through the Massachusetts Energy Efficiency Program Administrator’s (PAs) Commercial & Industrial (C&I) New Construction & Major Renovation and C&I Large Retrofit programs in 2011. Results are presented at the statewide level as well as for individual PAs.

Conclusion: In the case of annual MWh savings, the realization rate for Custom Refrigeration measures ranged from 65.8% for Western Massachusetts Electric to 118.8% for National Grid. The statewide realization rate for Annual MWh savings is 110.5% and the demand realization rates are all between 114.7% and 133.1%. For Custom Motors, realization rates for Annual MWh savings range from 77.7% for WMECO to 134.2% for NSTAR, the statewide realization rate was 91.4% for Annual MWh, and realization rates for the demand measurements were all between 74.5% and 90.2%. Realization rates for Custom Other measures Annual MWh savings range from 31.0% for National Grid to 270.7% for WMECO. The statewide realization rate for Custom Other measures was 61.4% for Annual MWh and the realization rates for the demand measurements were all around 63%.

Impact Evaluation of 2011 Prescription Gas Measures

Synopsis: This report presents the results of the impact evaluation of the Program Year 2011 (PY2011) Massachusetts Prescriptive Gas Measures Program. The evaluation consists of on-site monitoring and verification of the savings for a sample of participants for four of the top five measures installed, in terms of savings. The sample sites were monitored for about eight weeks in an attempt to capture seasonally sensitive variations in energy consumption between the winter and swing seasons. The first monitoring equipment was installed in the first week of December 2012 and recovery was completed during the second week of March 2013.

Conclusion: The overall relative performance (defined as the ratio of on-site savings divided by the revised tracking saving) for the four measures was about 102% . Relative performance is s. The condensing furnace and condensing boiler measures both had relative performance greater than 100%, at about 160% and 107 % respectively. Since they represent about 85% of total program savings, their performance offset the lower relative performance observed for the other two measures. Indirect water heater and infrared heating measures had lower relative performance of 79% and 20% respectively. Recommended new TRM measure savings values that should be used for tracking and planning purposes are also included.

Impact Evaluation of 2011-2012 Prescriptive VSDs

Synopsis: This report presents the results of the Impact Evaluation of Prescriptive Variable Speed Drives (VSD) conducted between 2011 and 2012. The objective of this impact evaluation was to begin to quantify how well prescriptive VSD installations are performing and to estimate the energy and demand savings resulting from a sample of 26 VSDs installed in Massachusetts between 2011 and 2012 using both pre- and post-installation metering.

Conclusion: The motor-level evaluation results were averaged to develop overall statewide realization rates. The statewide realization rate is the ratio of the total sample measured savings to the total sample TRM savings. While the overall realization ratio is close to 100%, there is a significant variation at the motor level, and many motors are either close to 0% or much higher than 100%. Failure to install controls or configure manual VSD speeds is the most common reason for a poor realization ratio. In some cases comparable energy savings could have been achieved through proper balancing and without a VSD installation. In most cases, these seasonal motor types were shut down and did not operate during the off seasons. Summer kW reductions were significantly higher than predicted by the TRM due to the post retrofit motor operating with manual controls at constant input kW. Additional conclusions and recommendations are presented in the report.

Large C&I Year 2011-12 Prescriptive VSDs Impact Evaluation Appendix B Summary Pages for Report

Synopsis: This document is an appendix of the Impact Evaluation of 2011-2012 Prescriptive VSDs. It includes detailed site reports for a sample selection of 2011 Prescriptive VSD applications, which were used to draw conclusions in the full report

Conclusion: This appendix presents tables and graphs of site level impact evaluation results for 2011-2012 Prescriptive VSD Installations.

Mid-Sized Customer Needs Assessment – Interim Results Memo

Synopsis: This memo provides a high level review of the preliminary results of analysis completed to date for the mid-sized customer needs assessment for the evaluation of the large commercial and industrial (C&I) programs operated by the Massachusetts Program Administrators (PA). A final report is planned for early fall, 2013. The memo provides study objectives, summarizes the evaluation approach, and presents key preliminary findings and recommendations. Results are based on interviews with program staff and implementation contractors for C&I programs and results from the MA-C&I Customer Profile Project to segment customers for analysis

Conclusion: The findings are organized by three topics: 1.) Approaches for marketing to customers within each size segment, 2.) Effectiveness of existing energy efficiency programs serving mid-sized customer needs, 3.) Estimating participation rates by account size segment. Conclusions drawn suggest there is no standard definition of ‘mid-sized’ customer; current customer data does not match PA account management claims; current customer tracking systems could be improved for better marketing; there is no consensus as to whether mid-sized customers are optimally served; the Gas PAs believe their mid-sized and large gas customers are not optimally served by the small electric Direct Install programs; and mid-sized customers appear underserved based on participation rates. Preliminary recommendations to the PA and EEAC for consideration are also presented.

Process Evaluation of the 2012 Bright Opportunities Program Final Report

Synopsis: This report includes a process evaluation of the Bright Opportunities Program. Bright Opportunities is a relatively new program that provides incentives directly to distributors of specific lighting technologies (LED bulbs and linear fluorescent tubes). Evaluators conducted end user surveys, interviews with participating and nonparticipating distributors, and interviews with participating contractors to gather data for use in this process evaluation.

Conclusion: Evaluators discuss Bright Opportunities program design, barriers to participation, motivations to participation, program delivery, and net-to gross estimates. Most of the evidence indicates that the Bright Opportunities Program is a well-designed and well-run program. However, to improve the program, evaluators recommend: doing more marketing of the program, especially to end users; encouraging participating trade allies to do more to educate their customers about the source and size of the buydown discounts; and doing more consumer education about the use of LED bulbs in dimmer switches.

Standard Boiler Research Plan and Interview Results Memo

Synopsis: This study was commissioned to identify the baseline boiler features and operation for both prescriptive and custom boiler measures for the Massachusetts Energy Efficiency Programs’ Large Commercial & Industrial Evaluation. Boiler distributors were interviewed to determine the characteristic features of new code-compliant boilers and also to provide leads for standard code-compliant boilers. The second stage of research was expected to be M&V of non-program boilers. However, the team was unsuccessful at locating any customer both with a relatively recently installed standard efficiency boiler and willing to participate in the study through distributors, contractors, and other efficiency partners contacted.

Conclusion: There was notable consistency in the stated market share for efficient boilers in Massachusetts. It is clear that efficient boilers are the norm in the commercial boiler market in Massachusetts. When asked what markets were not buying efficient boilers, the respondents consistently answered “low-end multi-family housing units.” The hypothesis that boiler sales would be tiered by features was not supported by the respondents. Almost all commercial installations are multiple units serving a single distribution system. There was general agreement that it was appropriate to characterize the market by size of boiler, due to the relationship between size and pricing. Replacement boilers make up the lion’s share of the market. Respondents indicated that more than 80% of the boilers sold were hydronic boilers, and the rest were steam. With the exception of one contractor, most firms did not target specific vertical markets. Pricing differences were minimal, especially at the lower end of the market. Each respondent reported that no used boilers or non-code compliant boilers are being installed. There was generally high praise for the boiler program in Massachusetts. When asked to compare programs in different states, the respondents gave the unanimous opinion that the Massachusetts programs were, by far, the most effective.

Massachusetts Small Business Direct Install 2010-2012 Impact Evaluations

Synopsis: The Evaluation Team conducted three impact evaluations of projects that were implemented through the SBDI Program in 2010-2011. This evaluation report provides: summaries of the impact evaluations of lighting fixtures and lighting controls installations; a complete report of the program billing analysis; a comparison of the study methods; results of all three evaluations; and a summary of the final set of recommendations for the 2013-2015 program.

Conclusion: Results of the Non-Controls Lighting Evaluation suggest that the SBDI Program accurately tracks savings for lighting fixture retrofits, with gross energy and demand realization rates of 96% (102% if including HVAC interaction) and 99%, respectively. The verified coincidence factors for the summer (66%, or 73% with HVAC interaction) and winter (44% with negligible HVAC interaction) peak periods were similar to previous estimates from the 2011 Plan Version Technical Reference Manual. Based on the findings of the evaluations we performed during 2010-2012. The Lighting Occupancy Sensor study suggested that the statewide program energy realization rate was 43% (42% with HVAC interaction), indicating that energy savings are being achieved by lighting occupancy sensor installations, but that the SBDI Program is over-estimating those energy impacts. The Team offers several recommendations for the SBDI Program. The Evaluation Team concluded that the Billing Analysis could not deliver a defensible estimate of savings, but estimates a true realization rate that is likely to be higher than 66%.

2012

2010 Combined Heat & Power Impact Evaluation Methodology & Analysis Memo

Synopsis: The purpose of this memo was to provide preliminary findings on performance of Combined Heat and Power (CHP) systems deployed in 2010 under the Massachusetts CHP Program and describe the methodology and analyses used in developing the performance estimates. Preliminary performance findings are compared to performance of CHP systems installed under other CHP programs to place the results in perspective. The preliminary performance evaluation findings used realization rate estimates that compare measured performance to expected performance. Preliminary realization rates were estimated for CHP electricity generation; useful thermal energy delivered, and fuel impacts.

Conclusion: The data available for this analysis show that the CHP systems deployed in Massachusetts achieved an average electrical realization rate of 0.93, an average thermal realization rate of 1.01, and an average fuel impacts realization rate of 0.90. There are significant differences in estimated electrical, thermal and fuel realization rates between CHP systems deployed for the two PAs. These differences occur regardless of whether the estimates are for electricity, thermal or gas impact realization rates. In general, realization rate estimates for CHP systems deployed by NSTAR are higher than for CHP systems deployed by NGRID.

Massachusetts Large Commercial & Industrial Process Evaluation

Synopsis: Evaluators used information gathered from in-depth interviews with EEAC consultants, program staff, trade allies, and customers; customer focus groups and telephone surveys; as well as PA program tracking databases to evaluate the processes involved with administering the MA large C&I energy efficiency programs.

Conclusion: Findings are reported on numerous key topics including how to improve integration and coordination, concerns about the adequacy of staffing levels, how to achieve deeper savings, whether medium-sized C&I customers are being adequately served by the programs, the adequacy or program tracking databases, and program satisfaction. The findings indicate that some aspects of the program are functioning well, but suggest that there are also aspects that could be improved.

Prescriptive Gas – Draft Final Program Evaluation Report

Synopsis: The primary objective of the PY10 Prescriptive Gas Evaluation was to complete a retrospective impact evaluation for the selected measures through site inspection, monitoring and analysis. The evaluation was based on the population of measures rebated during PY10 and selected measures were sampled at the site-level. The goal of this gas evaluation was to develop realization rates for program planning and for reporting 2011 and possibly later program results. Recommendations for new TRM measure savings values are also provided.

Conclusion: The overall realization rate for the four measures was about 104%. The Condensing Boiler measures had the highest realization rate of 137%. Since they represent about 64% of total program savings, this increase offset the decreases observed for the other measures. The indirect water heater measure had the second best realization rate of almost 59%. Furnace and Infrared Heating measures had low realization rates of 33% and almost 21% respectively.

Impact Evaluation of 2010 Custom Process and Compressed Air Installations

Synopsis: The objective of this impact evaluation was to provide verification or re-estimation of electric energy and demand savings estimates for 39 Custom Process and Compressed Air projects through site-specific inspection, monitoring, and analysis. The results of this study will be used to determine the final realization rates for Custom Process and Compressed Air energy efficiency measures installed in 2011. This evaluation report presents realization rates for gross energy savings for all PAs. It also provides realization rates for on-peak summer and winter demand savings for all PAs except for Western Massachusetts Electric (WMECO).

Conclusion: Overall, the Custom Process and Compressed Air program appears to be producing results that are lower than expected. The Custom Process end-use was a bit more variable than Compressed Air. In the case of annual MWh savings, the realization rate for Custom Process measures ranged from 51.3% for WMECO to 144.1% for Unitil. The statewide realization rate for Annual MWh savings was 76.2%. The estimated demand realization rates were all between 82.0% and 89.3%. Compressed Air realization rates for Annual MWh savings range from 78.0% for WMECO and NSTAR to 89.1% for National Grid. The statewide realization rate for Compressed Air measures was 85.2% for Annual MWh and the realization rates for the demand measurements were all between 74.1% and 76.3%. Statewide recommendations for improvement include requiring adequate savings documentation, devoting extra review time to calculation of operating hours, and considering additional savings opportunities when conducting TA studies. Individual utility recommendations are also included.

Impact Evaluation of 2010 Custom Lighting Installations

Synopsis: This impact evaluation provides verification or re-estimation of Custom Lighting electric energy and demand savings estimates based on site-specific inspection, monitoring, and analysis of a sample of Custom Lighting projects. Results presented include realization rates for annual MWh savings, on-peak MWh savings, and on-peak and seasonal demand (kW) savings at the times of the winter and summer peaks. Both statewide and PA-specific findings and recommendations for improvement are provided.

Conclusion: Overall, the Custom Lighting projects appear to be successfully providing energy and demand savings in the State of Massachusetts. Statewide results suggest that the realization rate for Custom Lighting measures was greater than 90% for annual MWh savings, on-peak summer kW, on-peak winter kW, and seasonal summer kW. For on-peak winter kW, the realization rate was 87.5%. Statewide recommendations for improvement include a systematic approach for estimating HVAC interactive effects, ensuring final savings documents are stored for future evaluations, and looking for increased savings opportunities from controls.

Non-Controls Lighting Evaluation for the Massachusetts Small Business Direct Install Program: Multi-Season Study

Synopsis: Evaluators conducted this impact evaluation to provide independent estimates of annual energy savings and peak demand impacts for a single type of measure installed as part of the MA Small Business Direct Install Program: replacement of lighting fixtures without controls. Results are based on an extrapolation of eight months of project data from 2010. Data collected from a subset of sites during summer-fall 2011 was integrated into data collected during winter 2010-2011. This report presents the results of the multi-season study, compares the multi-season savings results to the results of the winter study, and discusses the differences in lighting operation observed during the seasons.

Conclusion: Non-controls lighting measures implemented through the SBDI program offered by the MA PAs accounted for 38,938 MWh of energy savings and 10.3 MW of connected kW savings in PY2010, or 81% of the program’s energy impacts and 75% of demand impacts. In addition to statewide results, the evaluation stratified results for two facility demand tiers, as well as for each of the five electric PAs. Results describe the statewide program impact factors based on the multi-season study results, the impact formulas used to apply the impact factors to tracking energy and demand data, and comparisons of the energy and demand realization rates from the multi-season and winter studies. Gross kWh realization rates were at least 100% across the state, across sites with average demand over 50 kW, and for CLC and NSTAR. The statewide gross kWh realization rate is not impacted by the inclusion of summer logging data for sites with expected seasonal variation in their hours of operation. Connected kW realization rates were very close to 100% statewide, for the two kW tiers, and for all PAs. Evaluators estimated the statewide winter demand coincidence factor to be higher than the previous MA TRM winter demand coincidence factor. Based on winter and summer logging data, evaluators estimated a statewide summer demand coincidence factor and HVAC interaction factor similar to the 2011 Plan Version TRM of 37%. No correlation was found between summer coincidence factors estimated from winter data to summer coincidence factors directly measured in the summer.

Impact Evaluation of 2010 Custom Gas Installations

Synopsis: The primary mission of the study was to determine program realization rates. The rates will be used for planning and program reporting, including program year 2011 annual reporting and any 2013-2015 program planning and subsequent year reporting, unless replaced by results from a subsequent study. The evaluation included all the 2010 Custom Gas measures and derived realization rate results from on-site engineering monitoring-based assessments of installed custom gas measures of a statistically representative sample.

Conclusion: When comparing the 2009 and 2010 program years, there were significant increases in program participation and in the breadth of measures implemented. Tracking savings increased from 2.0 to 4.4 million therms, while the number of unique accounts increased from 249 to 330 from 2009 to 2010. The PA’s successfully reached a broader range of customers, particularly in the industrial sector. The final statewide realization rate of 67.7%, however, was lower than the previous year (71%). Overall, the evaluators conclude that the Custom Gas program is successfully providing energy savings in the State of Massachusetts, but provide some recommendations for program implementation and evaluation improvements.

Code Compliance Baseline Study

Synopsis: The principal research objectives of the study were to understand the energy code enforcement process, understand the level of compliance with the energy code in recently constructed commercial buildings through plan reviews and site visits, and understand the future energy savings potential for recently constructed buildings.

Conclusion: Overall, state-wide new construction code compliance was estimated to be 80%. However, this result does not mean that 80% of commercial building comply, as no buildings were fully in compliance. Rather, it means that on average, commercial buildings perform 20% worse than the code requires and use 20% more energy than fully compliant buildings. Evaluators drew several other conclusions based on this study. First, the 20% non-compliance represents a significant increase in the potential savings that could be legitimately claimed. Lighting and lighting controls offer major opportunities for efficiency while new daylighting provisions experience low compliance. Mechanical system efficiency levels are at full compliance, though installation practices are not fully code compliant. Code compliance is slightly better for larger buildings, occurs primarily at the design phase, and is a shared responsibility. Code officials require additional staff resources to share the burden of energy code compliance. Constructing and installing code compliant materials and equipment does not guarantee proper installation and operation. Energy code training and curriculum development is key to increasing knowledge and improving compliance process. Barriers to compliance range from lack of technical expertise to constraints on building officials staff and time. Lack of standardization, rules and metrics also negatively impact compliance.

HVAC Market Characterization and Penetration Analysis

Synopsis: This research was designed to estimate the market penetration of energy-efficient equipment in the MA commercial HVAC market, gauge the level of large C&I program influence on market penetration, and characterize the market for emergency replacement. Results are based on telephone interviews with commercial HVAC contractors and distributors in MA.

Conclusion: Results from contractors and distributors suggest that the market penetration of energy efficient ‘rooftop’ AC units <20 tons in size is approximately 55 percent to 54 percent. However, contractors provided substantially higher market penetration estimates for units > 20 tons than did distributors. Also, results indicate different degrees of program impact on market penetration depending on equipment size. Contractors offer substantially higher estimates of market penetration than do distributors for high efficiency natural gas furnaces with ECM motors (79 percent vs. 33 percent). Participants estimated relatively low market penetrations estimates in the absence of program incentives. Results from contractors suggest that market penetration of high-efficiency models among all condensing gas boilers is higher. Results do not clearly provide evidence of the program’s impact on market penetration. For Dual Enthalpy Economizers, contractor results suggest that market penetration is about 45 percent, while distributors estimate only 2 percent. For Demand Control Ventilation, contractor results suggest that the market penetration is 34 percent, while distributors estimate only 1 percent. For ECM Fan motors, contractor results were 41 percent, while distributors were 9 percent. And for Programmable Thermostats, contractors suggest 48 percent, while distributors estimate 4 percent. For all of these control measures, the impact of the program on market penetration is unclear. Based on study results, evaluators recommend raising efficiency levels for condensing gas boilers and offering stocking incentives to distributors.

Comparison of 2011 Desk Reviews with Decision Criteria Memo

Synopsis: This memo compares the results of the desk reviews of PY2011 Custom Gas participants with the Decision Criteria developed from benchmark sites (evaluated sites from PY2009 and PY2010 impact evaluations). The primary purpose of the desk review is to assist the Evaluation Team in deciding whether to proceed with site M&V for the PY2011 Custom Gas Program or not. The Benchmark Value, No Action Range, and Weighting Factor are the previously agreed upon Decision Criteria values and are summarized in Table 1 of the report.

Conclusion: Findings indicate that another statewide impact evaluation is not warranted. As noted in Table 1, four of the seven criteria fall within the ‘No Action Range’ including the most heavily weighted criterion characterizing the baseline. Results were also examined by PA to indicate whether a PA’s results were within or outside of the No Action range. NSTAR is the only PA where the sum of the weighted criteria indicates further study is warranted.

Revision: Decision Criteria for Proceeding to M&V Memo

Synopsis: This memo presents the benchmark review findings and a straw man proposal for the desk review Decision Criteria used to decide whether to proceed with site M&V for the PY2011 Custom Gas Program or not. A final section in the report describes the process for determining benchmark results.

Conclusion: The Custom Gas desk review Decision Criteria have been revised as follows: Revised figures include the results of all sites from the 2009 and 2010 impact evaluations; the desk review ratio has been dropped as a criterion because it is not an apples-to apples comparison with the evaluated realization rate and may be misleading; and the Evaluation Team proposes adding the hypothesis testing to determine whether the benchmark populations and the PY2011 are statistically different.

Commercial and Industrial Studies

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

Commercial New Construction Market Characterization

Search by Tag

Search by Date Range

Stay Informed