-
Human-in-the-Loop AI for HVAC Management Enhancing Comfort and Energy Efficiency
Authors:
Xinyu Liang,
Frits de Nijs,
Buser Say,
Hao Wang
Abstract:
Heating, Ventilation, and Air Conditioning (HVAC) systems account for approximately 38% of building energy consumption globally, making them one of the most energy-intensive services. The increasing emphasis on energy efficiency and sustainability, combined with the need for enhanced occupant comfort, presents a significant challenge for traditional HVAC systems. These systems often fail to dynami…
▽ More
Heating, Ventilation, and Air Conditioning (HVAC) systems account for approximately 38% of building energy consumption globally, making them one of the most energy-intensive services. The increasing emphasis on energy efficiency and sustainability, combined with the need for enhanced occupant comfort, presents a significant challenge for traditional HVAC systems. These systems often fail to dynamically adjust to real-time changes in electricity market rates or individual comfort preferences, leading to increased energy costs and reduced comfort. In response, we propose a Human-in-the-Loop (HITL) Artificial Intelligence framework that optimizes HVAC performance by incorporating real-time user feedback and responding to fluctuating electricity prices. Unlike conventional systems that require predefined information about occupancy or comfort levels, our approach learns and adapts based on ongoing user input. By integrating the occupancy prediction model with reinforcement learning, the system improves operational efficiency and reduces energy costs in line with electricity market dynamics, thereby contributing to demand response initiatives. Through simulations, we demonstrate that our method achieves significant cost reductions compared to baseline approaches while maintaining or enhancing occupant comfort. This feedback-driven approach ensures personalized comfort control without the need for predefined settings, offering a scalable solution that balances individual preferences with economic and environmental goals.
△ Less
Submitted 9 May, 2025;
originally announced May 2025.
-
Predict+Optimize Problem in Renewable Energy Scheduling
Authors:
Christoph Bergmeir,
Frits de Nijs,
Evgenii Genov,
Abishek Sriramulu,
Mahdi Abolghasemi,
Richard Bean,
John Betts,
Quang Bui,
Nam Trong Dinh,
Nils Einecke,
Rasul Esmaeilbeigi,
Scott Ferraro,
Priya Galketiya,
Robert Glasgow,
Rakshitha Godahewa,
Yanfei Kang,
Steffen Limmer,
Luis Magdalena,
Pablo Montero-Manso,
Daniel Peralta,
Yogesh Pipada Sunil Kumar,
Alejandro Rosales-Pérez,
Julian Ruddick,
Akylas Stratigakos,
Peter Stuckey
, et al. (3 additional authors not shown)
Abstract:
Predict+Optimize frameworks integrate forecasting and optimization to address real-world challenges such as renewable energy scheduling, where variability and uncertainty are critical factors. This paper benchmarks solutions from the IEEE-CIS Technical Challenge on Predict+Optimize for Renewable Energy Scheduling, focusing on forecasting renewable production and demand and optimizing energy cost.…
▽ More
Predict+Optimize frameworks integrate forecasting and optimization to address real-world challenges such as renewable energy scheduling, where variability and uncertainty are critical factors. This paper benchmarks solutions from the IEEE-CIS Technical Challenge on Predict+Optimize for Renewable Energy Scheduling, focusing on forecasting renewable production and demand and optimizing energy cost. The competition attracted 49 participants in total. The top-ranked method employed stochastic optimization using LightGBM ensembles, and achieved at least a 2% reduction in energy costs compared to deterministic approaches, demonstrating that the most accurate point forecast does not necessarily guarantee the best performance in downstream optimization. The published data and problem setting establish a benchmark for further research into integrated forecasting-optimization methods for energy systems, highlighting the importance of considering forecast uncertainty in optimization models to achieve cost-effective and reliable energy management. The novelty of this work lies in its comprehensive evaluation of Predict+Optimize methodologies applied to a real-world renewable energy scheduling problem, providing insights into the scalability, generalizability, and effectiveness of the proposed solutions. Potential applications extend beyond energy systems to any domain requiring integrated forecasting and optimization, such as supply chain management, transportation planning, and financial portfolio optimization.
△ Less
Submitted 14 April, 2025; v1 submitted 20 December, 2022;
originally announced December 2022.
-
Proximal Policy Optimization Based Reinforcement Learning for Joint Bidding in Energy and Frequency Regulation Markets
Authors:
Muhammad Anwar,
Changlong Wang,
Frits de Nijs,
Hao Wang
Abstract:
Driven by the global decarbonization effort, the rapid integration of renewable energy into the conventional electricity grid presents new challenges and opportunities for the battery energy storage system (BESS) participating in the energy market. Energy arbitrage can be a significant source of revenue for the BESS due to the increasing price volatility in the spot market caused by the mismatch b…
▽ More
Driven by the global decarbonization effort, the rapid integration of renewable energy into the conventional electricity grid presents new challenges and opportunities for the battery energy storage system (BESS) participating in the energy market. Energy arbitrage can be a significant source of revenue for the BESS due to the increasing price volatility in the spot market caused by the mismatch between renewable generation and electricity demand. In addition, the Frequency Control Ancillary Services (FCAS) markets established to stabilize the grid can offer higher returns for the BESS due to their capability to respond within milliseconds. Therefore, it is crucial for the BESS to carefully decide how much capacity to assign to each market to maximize the total profit under uncertain market conditions. This paper formulates the bidding problem of the BESS as a Markov Decision Process, which enables the BESS to participate in both the spot market and the FCAS market to maximize profit. Then, Proximal Policy Optimization, a model-free deep reinforcement learning algorithm, is employed to learn the optimal bidding strategy from the dynamic environment of the energy market under a continuous bidding scale. The proposed model is trained and validated using real-world historical data of the Australian National Electricity Market. The results demonstrate that our developed joint bidding strategy in both markets is significantly profitable compared to individual markets.
△ Less
Submitted 13 December, 2022;
originally announced December 2022.
-
Model-Free Approach to Fair Solar PV Curtailment Using Reinforcement Learning
Authors:
Zhuo Wei,
Frits de Nijs,
Jinhao Li,
Hao Wang
Abstract:
The rapid adoption of residential solar photovoltaics (PV) has resulted in regular overvoltage events, due to correlated reverse power flows. Currently, PV inverters prevent damage to electronics by curtailing energy production in response to overvoltage. However, this disproportionately affects households at the far end of the feeder, leading to an unfair allocation of the potential value of ener…
▽ More
The rapid adoption of residential solar photovoltaics (PV) has resulted in regular overvoltage events, due to correlated reverse power flows. Currently, PV inverters prevent damage to electronics by curtailing energy production in response to overvoltage. However, this disproportionately affects households at the far end of the feeder, leading to an unfair allocation of the potential value of energy produced. Globally optimizing for fair curtailment requires accurate feeder parameters, which are often unknown. This paper investigates reinforcement learning, which gradually optimizes a fair PV curtailment strategy by interacting with the system. We evaluate six fairness metrics on how well they can be learned compared to an optimal solution oracle. We show that all definitions permit efficient learning, suggesting that reinforcement learning is a promising approach to achieving both safe and fair PV coordination.
△ Less
Submitted 13 December, 2022;
originally announced December 2022.
-
Reinforcement Learning for Strategic Recommendations
Authors:
Georgios Theocharous,
Yash Chandak,
Philip S. Thomas,
Frits de Nijs
Abstract:
Strategic recommendations (SR) refer to the problem where an intelligent agent observes the sequential behaviors and activities of users and decides when and how to interact with them to optimize some long-term objectives, both for the user and the business. These systems are in their infancy in the industry and in need of practical solutions to some fundamental research challenges. At Adobe resea…
▽ More
Strategic recommendations (SR) refer to the problem where an intelligent agent observes the sequential behaviors and activities of users and decides when and how to interact with them to optimize some long-term objectives, both for the user and the business. These systems are in their infancy in the industry and in need of practical solutions to some fundamental research challenges. At Adobe research, we have been implementing such systems for various use-cases, including points of interest recommendations, tutorial recommendations, next step guidance in multi-media editing software, and ad recommendation for optimizing lifetime value. There are many research challenges when building these systems, such as modeling the sequential behavior of users, deciding when to intervene and offer recommendations without annoying the user, evaluating policies offline with high confidence, safe deployment, non-stationarity, building systems from passive data that do not contain past recommendations, resource constraint optimization in multi-user systems, scaling to large and dynamic actions spaces, and handling and incorporating human cognitive biases. In this paper we cover various use-cases and research challenges we solved to make these systems practical.
△ Less
Submitted 15 September, 2020;
originally announced September 2020.
-
Evaluation and Improvement of Laruelle-Widgrén Inverse Banzhaf Approximation
Authors:
Frits de Nijs,
Daan Wilmer
Abstract:
The goal of this paper is to critically evaluate a heuristic algorithm for the Inverse Banzhaf Index problem by Laruelle and Widgrén. Few qualitative results are known about the approximation quality of the heuristics for this problem. The intuition behind the operation of this approximation algorithm is analysed and evaluated. We found that the algorithm can not handle general inputs well, and of…
▽ More
The goal of this paper is to critically evaluate a heuristic algorithm for the Inverse Banzhaf Index problem by Laruelle and Widgrén. Few qualitative results are known about the approximation quality of the heuristics for this problem. The intuition behind the operation of this approximation algorithm is analysed and evaluated. We found that the algorithm can not handle general inputs well, and often fails to improve inputs. It is also shown to diverge after only tens of iterations. We present three alternative extensions of the algorithm that do not alter the complexity but can result in up to a factor 6.5 improvement in solution quality.
△ Less
Submitted 6 June, 2012;
originally announced June 2012.
