-
Quantum-Enhanced Reinforcement Learning for Power Grid Security Assessment
Authors:
Benjamin M. Peter,
Mert Korkali
Abstract:
The increasingly challenging task of maintaining power grid security requires innovative solutions. Novel approaches using reinforcement learning (RL) agents have been proposed to help grid operators navigate the massive decision space and nonlinear behavior of these complex networks. However, applying RL to power grid security assessment, specifically for combinatorially troublesome contingency a…
▽ More
The increasingly challenging task of maintaining power grid security requires innovative solutions. Novel approaches using reinforcement learning (RL) agents have been proposed to help grid operators navigate the massive decision space and nonlinear behavior of these complex networks. However, applying RL to power grid security assessment, specifically for combinatorially troublesome contingency analysis problems, has proven difficult to scale. The integration of quantum computing into these RL frameworks helps scale by improving computational efficiency and boosting agent proficiency by leveraging quantum advantages in action exploration and model-based interdependence. To demonstrate a proof-of-concept use of quantum computing for RL agent training and simulation, we propose a hybrid agent that runs on quantum hardware using IBM's Qiskit Runtime. We also provide detailed insight into the construction of parameterized quantum circuits (PQCs) for generating relevant quantum output. This agent's proficiency at maintaining grid stability is demonstrated relative to a benchmark model without quantum enhancement using N-k contingency analysis. Additionally, we offer a comparative assessment of the training procedures for RL models integrated with a quantum backend.
△ Less
Submitted 20 June, 2025; v1 submitted 19 April, 2025;
originally announced April 2025.
-
Robust Defense Against Extreme Grid Events Using Dual-Policy Reinforcement Learning Agents
Authors:
Benjamin M. Peter,
Mert Korkali
Abstract:
Reinforcement learning (RL) agents are powerful tools for managing power grids. They use large amounts of data to inform their actions and receive rewards or penalties as feedback to learn favorable responses for the system. Once trained, these agents can efficiently make decisions that would be too computationally complex for a human operator. This ability is especially valuable in decarbonizing…
▽ More
Reinforcement learning (RL) agents are powerful tools for managing power grids. They use large amounts of data to inform their actions and receive rewards or penalties as feedback to learn favorable responses for the system. Once trained, these agents can efficiently make decisions that would be too computationally complex for a human operator. This ability is especially valuable in decarbonizing power networks, where the demand for RL agents is increasing. These agents are well suited to control grid actions since the action space is constantly growing due to uncertainties in renewable generation, microgrid integration, and cybersecurity threats. To assess the efficacy of RL agents in response to an adverse grid event, we use the Grid2Op platform for agent training. We employ a proximal policy optimization (PPO) algorithm in conjunction with graph neural networks (GNNs). By simulating agents' responses to grid events, we assess their performance in avoiding grid failure for as long as possible. The performance of an agent is expressed concisely through its reward function, which helps the agent learn the most optimal ways to reconfigure a grid's topology amidst certain events. To model multi-actor scenarios that threaten modern power networks, particularly those resulting from cyberattacks, we integrate an opponent that acts iteratively against a given agent. This interplay between the RL agent and opponent is utilized in N-k contingency screening, providing a novel alternative to the traditional security assessment.
△ Less
Submitted 17 November, 2024;
originally announced November 2024.
-
Demonstration of a Real-Time Testbed for D-Band Integrated Sensing and Communication
Authors:
Sven Wittig,
Rodrigo Hernangomez,
Karen Vardanyan,
Ramez Askar,
Amr Haj-Omar,
Michael Peter,
Slawomir Stanczak
Abstract:
The D-band, spanning 110 GHz to 170 GHz, has emerged as a relevant frequency range for future mobile communications and radar sensing applications, particularly in the context of 6G technologies. This demonstration presents a high-bandwidth, real-time integrated sensing and communication (ISAC) platform operating in the upper D-band at 160 GHz. The platform comprises a software-defined intermediat…
▽ More
The D-band, spanning 110 GHz to 170 GHz, has emerged as a relevant frequency range for future mobile communications and radar sensing applications, particularly in the context of 6G technologies. This demonstration presents a high-bandwidth, real-time integrated sensing and communication (ISAC) platform operating in the upper D-band at 160 GHz. The platform comprises a software-defined intermediate frequency transceiver and a D-band radio frequency module. Its flexible software design allows for rapid integration of signal processing algorithms. Highlighting its potential for advanced research in ISAC systems, the platforms's efficacy is showcased through a live demonstration with an algorithm implementing human target tracking and activity classification.
△ Less
Submitted 28 June, 2024;
originally announced July 2024.
-
Reconfigurable Intelligent Surfaces for 6G Mobile Networks: An Industry R&D Perspective
Authors:
Maik Sode,
Michael Ponschab,
Lucas N. Ribeiro,
Sven Haesloop,
Ehsan Tohidi,
Michael Peter,
Sławomir Stańczak,
Bilal H. Mohamed,
Wilhelm Keusgen,
Heinz Mellein,
Eslam Yassin,
Bernd Schroeder
Abstract:
The reconfigurable intelligent surface (RIS) technology is a potential solution to enhance network capacity and coverage without significant investment in additional infrastructure in 6G networks. This work highlights the interest of the mobile communication industry in RIS, and discusses the development of liquid crystal-based RIS for improved energy efficiency and coverage in the millimeter-wave…
▽ More
The reconfigurable intelligent surface (RIS) technology is a potential solution to enhance network capacity and coverage without significant investment in additional infrastructure in 6G networks. This work highlights the interest of the mobile communication industry in RIS, and discusses the development of liquid crystal-based RIS for improved energy efficiency and coverage in the millimeter-wave band. Furthermore, the paper discusses perspectives and insights from an industry R&D point of view, addressing relevant use cases, technical requirements, implementation challenges, and practical considerations for RIS deployment optimization in the context of 6G networks. A hardware design of an RIS with liquid crystal at 28 GHz is presented. A propagation model for RIS as a new part of the system architecture is discussed, with approaches of semi-empirical models, geometric models, and their combination through the application of artificial intelligence/machine learning. Finally, a channel model for deployment optimization and dimensioning is presented, with the findings that a rather large RIS is favorable for coverage improvement, as well as greater attenuation at higher frequencies combined with a smaller RIS size.
△ Less
Submitted 16 November, 2024; v1 submitted 28 June, 2024;
originally announced June 2024.
-
Little or no equalization is needed in energy-efficient sub-THz mobile access
Authors:
Lorenzo Miretti,
Thomas Kühne,
Alper Schultze,
Wilhelm Keusgen,
Giuseppe Caire,
Michael Peter,
Slawomir Stańczak,
Taro Eichler
Abstract:
By trading coverage and hardware complexity for abundance of spectrum, sub-THz mobile access networks are expected to operate under highly directive and relatively spectrally inefficient transmission regimes, while still offering enormous capacity gains over current sub-6GHz alternatives. Building on this assumption, and supported by extensive indoor directional channel measurements at 160 GHz, th…
▽ More
By trading coverage and hardware complexity for abundance of spectrum, sub-THz mobile access networks are expected to operate under highly directive and relatively spectrally inefficient transmission regimes, while still offering enormous capacity gains over current sub-6GHz alternatives. Building on this assumption, and supported by extensive indoor directional channel measurements at 160 GHz, this study advocates the use of very simple modulation and equalization techniques for sub-THz mobile access. Specifically, we demonstrate that, under the aforementioned transmission regimes, little or no equalization is needed for scoring significant capacity gain targets. In particular, we show that single-carrier or low-number-of-subcarriers modulations are very attractive competitors to the dramatically more complex and energy inefficient traditional multi-carrier designs.
△ Less
Submitted 5 December, 2022; v1 submitted 11 October, 2022;
originally announced October 2022.
-
Sub-THz Channel Measurements at 158 GHz and 300 GHz in a Street Canyon Environment
Authors:
Wilhelm Keusgen,
Alper Schultze,
Michael Peter,
Taro Eichler
Abstract:
This paper presents first results of a channel measurement campaign performed in an urban micro (UMi) street canyon scenario at 158 GHz and 300 GHz.
This paper presents first results of a channel measurement campaign performed in an urban micro (UMi) street canyon scenario at 158 GHz and 300 GHz.
△ Less
Submitted 2 March, 2022;
originally announced March 2022.
-
Observations on the Angular Statistics of the Indoor Sub-THz Radio Channel at 158 GHz
Authors:
Alper Schultze,
Wilhelm Keusgen,
Michael Peter,
Taro Eichler
Abstract:
This paper presents selected results from a channel measurement campaign conducted in a shopping mall scenario at 158 GHz. The focus is on the statistical analysis of the collected measurement data in terms of directional channel gain. Although most power is received from the line-of-sight (LOS) direction, significant multipaths arrive from all measured azimuth directions. The median of the sorted…
▽ More
This paper presents selected results from a channel measurement campaign conducted in a shopping mall scenario at 158 GHz. The focus is on the statistical analysis of the collected measurement data in terms of directional channel gain. Although most power is received from the line-of-sight (LOS) direction, significant multipaths arrive from all measured azimuth directions. The median of the sorted directional channel gain can be approximated by a linear curve showing an offset of about 10 dB with respect to the LOS direction.
△ Less
Submitted 2 March, 2022;
originally announced March 2022.
-
On the Dynamic Range of Digital Correlative Time Domain Radio Channel Measurements
Authors:
Sven Wittig,
Wilhelm Keusgen,
Michael Peter,
Taro Eichler
Abstract:
For the concise characterization of radio channel measurement setups, different performance metrics like dynamic range and maximum measurable path loss have been proposed. This work further elaborates on these metrics and discusses practical bounds for the class of digital correlative time domain channel sounders. From a general instrumentation model, relevant nonidealities and the resulting effec…
▽ More
For the concise characterization of radio channel measurement setups, different performance metrics like dynamic range and maximum measurable path loss have been proposed. This work further elaborates on these metrics and discusses practical bounds for the class of digital correlative time domain channel sounders. From a general instrumentation model, relevant nonidealities and the resulting effects on measurement performance are identified. These considerations yield the tools to select appropriate measurement parameters for a given scenario and assess the overall performance of a particular measurement setup. The findings are further illustrated with data acquired from an instrument-based measurement setup.
△ Less
Submitted 18 August, 2020;
originally announced August 2020.
-
Measurement and Characterization of an Indoor Industrial Environment at 3.7 and 28 GHz
Authors:
Mathis Schmieder,
Taro Eichler,
Sven Wittig,
Michael Peter,
Wilhelm Keusgen
Abstract:
Fifth generation (5G) mobile networks are expected to play an increasing role in industrial communication with private mobile communication networks deployed on company premises. For planning, standardization and product development, it is crucial to to thoroughly understand the radio channel characteristics of such environments. Frequencies around 3.7 GHz were already reserved by regulation autho…
▽ More
Fifth generation (5G) mobile networks are expected to play an increasing role in industrial communication with private mobile communication networks deployed on company premises. For planning, standardization and product development, it is crucial to to thoroughly understand the radio channel characteristics of such environments. Frequencies around 3.7 GHz were already reserved by regulation authorities and to meet the increasing demand for higher bandwidths, spectrum in the millimeter wave range around 28 GHz is targeted. This paper presents a wideband channel measurement campaign at both 3.7 and 28 GHz with direction-of-arrival information at 28 GHz. The results are compared to the 3GPP TR 38.901 Indoor Factory model and to two other recent papers. Evaluation of path loss and RMS delay and angle spread show the unique nature of industrial indoor environments.
△ Less
Submitted 24 April, 2020;
originally announced April 2020.
-
THz Channel Sounding: Design and Validation of a High Performance Channel Sounder at 300 GHz
Authors:
Mathis Schmieder,
Wilhelm Keusgen,
Michael Peter,
Sven Wittig,
Thomas Merkle,
Sandrine Wagner,
Michael Kuri,
Taro Eichler
Abstract:
In this paper, a novel instrument-based time-domain THz channel sounder for the 300 GHz band is introduced. The performance is characterized by conducted measurements and verified by over-the-air measurements, which confirm the sensitivity and highlight the temporal resolution of the setup. Dynamic range, maximum measurable path loss and phase stability are evaluated. The results show that the per…
▽ More
In this paper, a novel instrument-based time-domain THz channel sounder for the 300 GHz band is introduced. The performance is characterized by conducted measurements and verified by over-the-air measurements, which confirm the sensitivity and highlight the temporal resolution of the setup. Dynamic range, maximum measurable path loss and phase stability are evaluated. The results show that the performance in terms of dynamic range and maximum measurable path loss is, to the best knowledge of the authors, unprecedented in this frequency range.
△ Less
Submitted 18 January, 2020;
originally announced January 2020.
-
Directional Wideband Channel Measurements at 28 GHz in an Industrial Environment
Authors:
Mathis Schmieder,
Fabian Undi,
Michael Peter,
Ephraim Koenig,
Wilhelm Keusgen
Abstract:
With the expected adoption of millimeter wave technologies for industrial communication, it is fundamentally important to properly understand the radio channel characteristics of such environments. This paper presents the setup, scenario and results of a measurement campaign at 28 GHz in a machine hall. The radio channel was measured with a bandwidth of 2 GHz and both large scale parameters and di…
▽ More
With the expected adoption of millimeter wave technologies for industrial communication, it is fundamentally important to properly understand the radio channel characteristics of such environments. This paper presents the setup, scenario and results of a measurement campaign at 28 GHz in a machine hall. The radio channel was measured with a bandwidth of 2 GHz and both large scale parameters and directional information were extracted. Evaluation of the power delay profiles shows that the channel contains dense multipath components. A path loss model is parameterized and blockage losses, RMS delay and angle spreads are evaluated. Comparison with the 3GPP TR 38.901 channel model shows that none of the currently defined scenarios is a fit for industrial settings. This emphasizes the need for a newly defined scenario with an industry specific parameter set.
△ Less
Submitted 2 July, 2019;
originally announced July 2019.
