-
Lessons learned from field demonstrations of model predictive control and reinforcement learning for residential and commercial HVAC: A review
Authors:
Arash J. Khabbazi,
Elias N. Pergantis,
Levi D. Reyes Premer,
Panagiotis Papageorgiou,
Alex H. Lee,
James E. Braun,
Gregor P. Henze,
Kevin J. Kircher
Abstract:
A large body of simulation research suggests that model predictive control (MPC) and reinforcement learning (RL) for heating, ventilation, and air-conditioning (HVAC) in residential and commercial buildings could reduce energy costs, pollutant emissions, and strain on power grids. Despite this potential, neither MPC nor RL has seen widespread industry adoption. Field demonstrations could accelerat…
▽ More
A large body of simulation research suggests that model predictive control (MPC) and reinforcement learning (RL) for heating, ventilation, and air-conditioning (HVAC) in residential and commercial buildings could reduce energy costs, pollutant emissions, and strain on power grids. Despite this potential, neither MPC nor RL has seen widespread industry adoption. Field demonstrations could accelerate MPC and RL adoption by providing real-world data that support the business case for deployment. Here we review 24 papers that document field demonstrations of MPC and RL in residential buildings and 80 in commercial buildings. After presenting demographic information -- such as experiment scopes, locations, and durations -- this paper analyzes experiment protocols and their influence on performance estimates. We find that 71% of the reviewed field demonstrations use experiment protocols that may lead to unreliable performance estimates. Over the remaining 29% that we view as reliable, the weighted-average cost savings, weighted by experiment duration, are 16% in residential buildings and 13% in commercial buildings. While these savings are potentially attractive, making the business case for MPC and RL also requires characterizing the costs of deployment, operation, and maintenance. Only 13 of the 104 reviewed papers report these costs or discuss related challenges. Based on these observations, we recommend directions for future field research, including: Improving experiment protocols; reporting deployment, operation, and maintenance costs; designing algorithms and instrumentation to reduce these costs; controlling HVAC equipment alongside other distributed energy resources; and pursuing emerging objectives such as peak shaving, arbitraging wholesale energy prices, and providing power grid reliability services.
△ Less
Submitted 12 June, 2025; v1 submitted 6 March, 2025;
originally announced March 2025.
-
Why has advanced commercial HVAC control not yet achieved its promise?
Authors:
Gregor P. Henze,
Kevin J. Kircher,
James E. Braun
Abstract:
Over the last two decades, research and development efforts have shown that advanced control of heating, ventilation, and air conditioning (HVAC) equipment in commercial buildings can improve energy efficiency, reduce emissions, and turn buildings into active participants in the power grid. Despite these efforts, advanced commercial HVAC control has not yet seen widespread adoption. In this paper,…
▽ More
Over the last two decades, research and development efforts have shown that advanced control of heating, ventilation, and air conditioning (HVAC) equipment in commercial buildings can improve energy efficiency, reduce emissions, and turn buildings into active participants in the power grid. Despite these efforts, advanced commercial HVAC control has not yet seen widespread adoption. In this paper, we argue that the research community can help companies deploy advanced HVAC control at speed and scale by reorienting research efforts toward clearly demonstrating the business case for adoption. To support this argument, we draw on findings from the 2023 Intelligent Building Operations Workshop, which brought together researchers, entrepreneurs, and representatives from industry and government to discuss current business offerings, state-of-the-art field demonstrations, barriers to adoption, and future directions.
△ Less
Submitted 9 November, 2024;
originally announced November 2024.
-
Granger Causality Based Hierarchical Time Series Clustering for State Estimation
Authors:
Sin Yong Tan,
Homagni Saha,
Margarite Jacoby,
Gregor P. Henze,
Soumik Sarkar
Abstract:
Clustering is an unsupervised learning technique that is useful when working with a large volume of unlabeled data. Complex dynamical systems in real life often entail data streaming from a large number of sources. Although it is desirable to use all source variables to form accurate state estimates, it is often impractical due to large computational power requirements, and sufficiently robust alg…
▽ More
Clustering is an unsupervised learning technique that is useful when working with a large volume of unlabeled data. Complex dynamical systems in real life often entail data streaming from a large number of sources. Although it is desirable to use all source variables to form accurate state estimates, it is often impractical due to large computational power requirements, and sufficiently robust algorithms to handle these cases are not common. We propose a hierarchical time series clustering technique based on symbolic dynamic filtering and Granger causality, which serves as a dimensionality reduction and noise-rejection tool. Our process forms a hierarchy of variables in the multivariate time series with clustering of relevant variables at each level, thus separating out noise and less relevant variables. A new distance metric based on Granger causality is proposed and used for the time series clustering, as well as validated on empirical data sets. Experimental results from occupancy detection and building temperature estimation tasks show fidelity to the empirical data sets while maintaining state-prediction accuracy with substantially reduced data dimensionality.
△ Less
Submitted 9 April, 2021;
originally announced April 2021.
-
CityLearn: Standardizing Research in Multi-Agent Reinforcement Learning for Demand Response and Urban Energy Management
Authors:
Jose R Vazquez-Canteli,
Sourav Dey,
Gregor Henze,
Zoltan Nagy
Abstract:
Rapid urbanization, increasing integration of distributed renewable energy resources, energy storage, and electric vehicles introduce new challenges for the power grid. In the US, buildings represent about 70% of the total electricity demand and demand response has the potential for reducing peaks of electricity by about 20%. Unlocking this potential requires control systems that operate on distri…
▽ More
Rapid urbanization, increasing integration of distributed renewable energy resources, energy storage, and electric vehicles introduce new challenges for the power grid. In the US, buildings represent about 70% of the total electricity demand and demand response has the potential for reducing peaks of electricity by about 20%. Unlocking this potential requires control systems that operate on distributed systems, ideally data-driven and model-free. For this, reinforcement learning (RL) algorithms have gained increased interest in the past years. However, research in RL for demand response has been lacking the level of standardization that propelled the enormous progress in RL research in the computer science community. To remedy this, we created CityLearn, an OpenAI Gym Environment which allows researchers to implement, share, replicate, and compare their implementations of RL for demand response. Here, we discuss this environment and The CityLearn Challenge, a RL competition we organized to propel further progress in this field.
△ Less
Submitted 18 December, 2020;
originally announced December 2020.
-
The effect of price-based demand response on carbon emissions in European electricity markets: The importance of adequate carbon prices
Authors:
Markus Fleschutz,
Markus Bohlayer,
Marco Braun,
Gregor Henze,
Michael D. Murphy
Abstract:
Price-based demand response (PBDR) has recently been attributed great economic but also environmental potential. However, the determination of its short-term effects on carbon emissions requires the knowledge of marginal emission factors (MEFs), which compared to grid mix emission factors (XEFs), are cumbersome to calculate due to the complex characteristics of national electricity markets. This s…
▽ More
Price-based demand response (PBDR) has recently been attributed great economic but also environmental potential. However, the determination of its short-term effects on carbon emissions requires the knowledge of marginal emission factors (MEFs), which compared to grid mix emission factors (XEFs), are cumbersome to calculate due to the complex characteristics of national electricity markets. This study, therefore, proposes two merit order-based methods to approximate hourly MEFs and applies it to readily available datasets from 20 European countries for the years 2017-2019. Based on the resulting electricity prices, MEFs, and XEFs, standardized daily load shifts were simulated to quantify their effects on marginal costs and carbon emissions. Finally, by repeating the load shift simulations for different carbon price levels, the impact of the carbon price on the resulting carbon emissions was analyzed. Interestingly, the simulated price-based load shifts led to increases in operational carbon emissions for 8 of the 20 countries and to an average increase of 2.1% across all 20 countries. Switching from price-based to MEF-based load shifts reduced the corresponding carbon emissions to a decrease of 35%, albeit with 56% lower monetary cost savings compared to the price-based load shifts. Under specific circumstances, PBDR leads to an increase in carbon emissions, mainly due to the economic advantage fuel sources such as lignite and coal have in the merit order. However, as the price of carbon is increased, the correlation between the carbon intensity and the marginal cost of the fuels substantially increases. Therefore, with adequate carbon prices, PBDR can be an effective tool for both economical and environmental improvement.
△ Less
Submitted 1 December, 2020;
originally announced December 2020.
-
A Graph Neural Network Framework for Causal Inference in Brain Networks
Authors:
Simon Wein,
Wilhelm Malloni,
Ana Maria Tomé,
Sebastian M. Frank,
Gina-Isabelle Henze,
Stefan Wüst,
Mark W. Greenlee,
Elmar W. Lang
Abstract:
A central question in neuroscience is how self-organizing dynamic interactions in the brain emerge on their relatively static structural backbone. Due to the complexity of spatial and temporal dependencies between different brain areas, fully comprehending the interplay between structure and function is still challenging and an area of intense research. In this paper we present a graph neural netw…
▽ More
A central question in neuroscience is how self-organizing dynamic interactions in the brain emerge on their relatively static structural backbone. Due to the complexity of spatial and temporal dependencies between different brain areas, fully comprehending the interplay between structure and function is still challenging and an area of intense research. In this paper we present a graph neural network (GNN) framework, to describe functional interactions based on the structural anatomical layout. A GNN allows us to process graph-structured spatio-temporal signals, providing a possibility to combine structural information derived from diffusion tensor imaging (DTI) with temporal neural activity profiles, like observed in functional magnetic resonance imaging (fMRI). Moreover, dynamic interactions between different brain regions learned by this data-driven approach can provide a multi-modal measure of causal connectivity strength. We assess the proposed model's accuracy by evaluating its capabilities to replicate empirically observed neural activation profiles, and compare the performance to those of a vector auto regression (VAR), like typically used in Granger causality. We show that GNNs are able to capture long-term dependencies in data and also computationally scale up to the analysis of large-scale networks. Finally we confirm that features learned by a GNN can generalize across MRI scanner types and acquisition protocols, by demonstrating that the performance on small datasets can be improved by pre-training the GNN on data from an earlier and different study. We conclude that the proposed multi-modal GNN framework can provide a novel perspective on the structure-function relationship in the brain. Therewith this approach can be promising for the characterization of the information flow in brain networks.
△ Less
Submitted 14 October, 2020;
originally announced October 2020.
-
Energy Prediction using Spatiotemporal Pattern Networks
Authors:
Zhanhong Jiang,
Chao Liu,
Adedotun Akintayo,
Gregor Henze,
Soumik Sarkar
Abstract:
This paper presents a novel data-driven technique based on the spatiotemporal pattern network (STPN) for energy/power prediction for complex dynamical systems. Built on symbolic dynamic filtering, the STPN framework is used to capture not only the individual system characteristics but also the pair-wise causal dependencies among different sub-systems. For quantifying the causal dependency, a mutua…
▽ More
This paper presents a novel data-driven technique based on the spatiotemporal pattern network (STPN) for energy/power prediction for complex dynamical systems. Built on symbolic dynamic filtering, the STPN framework is used to capture not only the individual system characteristics but also the pair-wise causal dependencies among different sub-systems. For quantifying the causal dependency, a mutual information based metric is presented. An energy prediction approach is subsequently proposed based on the STPN framework. For validating the proposed scheme, two case studies are presented, one involving wind turbine power prediction (supply side energy) using the Western Wind Integration data set generated by the National Renewable Energy Laboratory (NREL) for identifying the spatiotemporal characteristics, and the other, residential electric energy disaggregation (demand side energy) using the Building America 2010 data set from NREL for exploring the temporal features. In the energy disaggregation context, convex programming techniques beyond the STPN framework are developed and applied to achieve improved disaggregation performance.
△ Less
Submitted 3 February, 2017;
originally announced February 2017.
