-
Deep Reinforcement Learning Based Routing for Heterogeneous Multi-Hop Wireless Networks
Authors:
Brian Kim,
Justin H. Kong,
Terrence J. Moore,
Fikadu T. Dagefu
Abstract:
Routing in multi-hop wireless networks is a complex problem, especially in heterogeneous networks where multiple wireless communication technologies coexist. Reinforcement learning (RL) methods, such as Q-learning, have been introduced for decentralized routing by allowing nodes to make decisions based on local observations. However, Q-learning suffers from scalability issues and poor generalizati…
▽ More
Routing in multi-hop wireless networks is a complex problem, especially in heterogeneous networks where multiple wireless communication technologies coexist. Reinforcement learning (RL) methods, such as Q-learning, have been introduced for decentralized routing by allowing nodes to make decisions based on local observations. However, Q-learning suffers from scalability issues and poor generalization due to the difficulty in managing the Q-table in large or dynamic network topologies, especially in heterogeneous networks (HetNets) with diverse channel characteristics. Thus, in this paper, we propose a novel deep Q-network (DQN)-based routing framework for heterogeneous multi-hop wireless networks to maximize the end-to-end rate of the route by improving scalability and adaptability, where each node uses a deep neural network (DNN) to estimate the Q-values and jointly select the next-hop relay and a communication technology for transmission. To achieve better performance with the DNN, selecting which nodes to exchange information is critical, as it not only defines the state and action spaces but also determines the input to the DNN. To this end, we propose neighbor node selection strategies based on channel gain and rate between nodes rather than a simple distance-based approach for an improved set of states and actions for DQN-based routing. During training, the model experiences diverse network topologies to ensure generalization and robustness, and simulation results show that the proposed neighbor node selection outperforms simple distance-based selection. Further, we observe that the DQN-based approach outperforms various benchmark schemes and performs comparably to the optimal approach.
△ Less
Submitted 20 August, 2025;
originally announced August 2025.
-
Generalizing Biased Backpressure Routing and Scheduling to Wireless Multi-hop Networks with Advanced Air-interfaces
Authors:
Zhongyuan Zhao,
Yujun Ming,
Ananthram Swami,
Kevin Chan,
Fikadu Dagefu,
Santiago Segarra
Abstract:
Backpressure (BP) routing and scheduling is a well-established resource allocation method for wireless multi-hop networks, known for its fully distributed operations and proven maximum queue stability. Recent advances in shortest path-biased BP routing (SP-BP) mitigate shortcomings such as slow startup and random walk, but exclusive link-level commodity selection still suffers from the last-packet…
▽ More
Backpressure (BP) routing and scheduling is a well-established resource allocation method for wireless multi-hop networks, known for its fully distributed operations and proven maximum queue stability. Recent advances in shortest path-biased BP routing (SP-BP) mitigate shortcomings such as slow startup and random walk, but exclusive link-level commodity selection still suffers from the last-packet problem and bandwidth underutilization. Moreover, classic BP routing implicitly assumes single-input-single-output (SISO) transceivers, which can lead to the same packets being scheduled on multiple outgoing links for multiple-input-multiple-output (MIMO) transceivers, causing detouring and looping in MIMO networks. In this paper, we revisit the foundational Lyapunov drift theory underlying BP routing and demonstrate that exclusive commodity selection is unnecessary, and instead propose a Max-Utility link-sharing method. Additionally, we generalize MaxWeight scheduling to MIMO networks by introducing attributed capacity hypergraphs (ACH), an extension of traditional conflict graphs for SISO networks, and by incorporating backlog reassignment into scheduling iterations to prevent redundant packet routing. Numerical evaluations show that our approach substantially mitigates the last-packet problem in state-of-the-art (SOTA) SP-BP under lightweight traffic, and slightly expands the network capacity region for heavier traffic.
△ Less
Submitted 30 April, 2025;
originally announced April 2025.
-
Optimal Routing and Link Configuration for Covert Heterogeneous Wireless Networks
Authors:
Amna Gillani,
Beatriz Lorenzo,
Majid Ghaderi,
Fikadu Dagefu,
Dennis Goeckel
Abstract:
Nodes in contemporary radio networks often have multiple interfaces available for communication: WiFi, cellular, LoRa, Zigbee, etc. This motivates understanding both link and network configuration when multiple communication modalities with vastly different capabilities are available to each node. In conjunction, covertness or the hiding of radio communications is often a significant concern in bo…
▽ More
Nodes in contemporary radio networks often have multiple interfaces available for communication: WiFi, cellular, LoRa, Zigbee, etc. This motivates understanding both link and network configuration when multiple communication modalities with vastly different capabilities are available to each node. In conjunction, covertness or the hiding of radio communications is often a significant concern in both commercial and military wireless networks. We consider the optimal routing problem in wireless networks when nodes have multiple interfaces available and intend to hide the presence of the transmission from attentive and capable adversaries. We first consider the maximization of the route capacity given an end-to-end covertness constraint against a single adversary and we find a polynomial-time algorithm for optimal route selection and link configuration. We further provide optimal polynomial-time algorithms for two important extensions: (i) statistical uncertainty during optimization about the channel state information for channels from system nodes to the adversary; and, (ii) maintaining covertness against multiple adversaries. Numerical results are included to demonstrate the gains of employing heterogeneous radio resources and to compare the performance of the proposed approach versus alternatives.
△ Less
Submitted 8 January, 2025; v1 submitted 9 December, 2024;
originally announced December 2024.
-
Decentralized Covert Routing in Heterogeneous Networks Using Reinforcement Learning
Authors:
Justin Kong,
Terrence J. Moore,
Fikadu T. Dagefu
Abstract:
This letter investigates covert routing communications in a heterogeneous network where a source transmits confidential data to a destination with the aid of relaying nodes where each transmitter judiciously chooses one modality among multiple communication modalities. We develop a novel reinforcement learning-based covert routing algorithm that finds a route from the source to the destination whe…
▽ More
This letter investigates covert routing communications in a heterogeneous network where a source transmits confidential data to a destination with the aid of relaying nodes where each transmitter judiciously chooses one modality among multiple communication modalities. We develop a novel reinforcement learning-based covert routing algorithm that finds a route from the source to the destination where each node identifies its next hop and modality only based on the local feedback information received from its neighboring nodes. We show based on numerical simulations that the proposed covert routing strategy has only negligible performance loss compared to the optimal centralized routing scheme.
△ Less
Submitted 31 January, 2024;
originally announced February 2024.
-
Directional Communication Enabled by Mobile Parasitic Elements
Authors:
Fikadu T. Dagefu,
Jeffrey N. Twigg,
Chirag R. Rao,
Brian M. Sadler
Abstract:
Mobile communications in complex environments such as mega-cities is a challenging problem that limits the ability to deploy autonomous agents in support of operations. Building on recent progress in low frequency networking that utilizes miniature antennas to provide persistent connectivity among agents, we consider the design and collaborative manipulation of a distributed robotic antenna array…
▽ More
Mobile communications in complex environments such as mega-cities is a challenging problem that limits the ability to deploy autonomous agents in support of operations. Building on recent progress in low frequency networking that utilizes miniature antennas to provide persistent connectivity among agents, we consider the design and collaborative manipulation of a distributed robotic antenna array to provide directional communications that will enable enhanced networking, interference rejection, and collaborative control. The use of parasitic elements in a Yagi-Uda type array design avoids the need for synchronization and highly accurate position control among the agents. We utilize physics-based simulations to investigate the feasibility of using mobile agents equipped with an excited antenna element along with a set of support nodes having parasitic elements that adaptively configure to enhance radiation in a desired direction. We take into account mobile node pose uncertainty including element position and angular orientation, as well as ground scattering effects. We pursue an optimal design approach for different types of ground electromagnetic characteristics based on a hybrid full-wave propagation simulation and genetic algorithm optimization. We also present experiment with one mobile node and two static elements. The results demonstrate the ability to achieve directional low frequency communications that is robust to robotic pose error.
△ Less
Submitted 15 August, 2019;
originally announced August 2019.
-
Real-Time Digital Video Streaming at Low-VHF for Compact Autonomous Agents in Complex Scenes
Authors:
Jihun Choi,
Chirag Rao,
Fikadu T. Dagefu
Abstract:
This paper presents an experimental investigation of real-time digital video streaming in physically complex Non-Line-Of-Sight (NLoS) channels using a low-power, low-VHF system integrated on a compact robotic platform. Reliable video streaming in NLoS channels over infrastructure-poor ad-hoc radio networks is challenging due to multipath and shadow fading. In this effort, we focus on exploiting th…
▽ More
This paper presents an experimental investigation of real-time digital video streaming in physically complex Non-Line-Of-Sight (NLoS) channels using a low-power, low-VHF system integrated on a compact robotic platform. Reliable video streaming in NLoS channels over infrastructure-poor ad-hoc radio networks is challenging due to multipath and shadow fading. In this effort, we focus on exploiting the near-ground low-VHF channel which has been shown to have improved penetration, reduced fading, and lower power requirements (which is critical for autonomous agents with limited power) compared to higher frequencies. Specifically, we develop a compact, low-power, low-VHF radio test-bed enabled by recent advances in efficient miniature antennas and off-the-shelf software-defined radios. Our main goal is to carry out an empirical study in realistic environments of how the improved propagation conditions at low-VHF affect the reliability of video-streaming with constraints stemming from the limited available bandwidth with electrically small low-VHF antennas. We show quantitative performance analysis of video streaming from a robotic platform navigating inside a large occupied building received by a node located outdoors: bit error rate (BER) and channel-induced Peak Signal-to-Noise Ratio (PSNR) degradation. The results show channel-effect-free-like video streaming with the low-VHF system in complex NLoS channels.
△ Less
Submitted 28 June, 2019;
originally announced June 2019.
