-
Bounding-Box Inference for Error-Aware Model-Based Reinforcement Learning
Authors:
Erin J. Talvitie,
Zilei Shao,
Huiying Li,
Jinghan Hu,
Jacob Boerma,
Rory Zhao,
Xintong Wang
Abstract:
In model-based reinforcement learning, simulated experiences from the learned model are often treated as equivalent to experience from the real environment. However, when the model is inaccurate, it can catastrophically interfere with policy learning. Alternatively, the agent might learn about the model's accuracy and selectively use it only when it can provide reliable predictions. We empirically…
▽ More
In model-based reinforcement learning, simulated experiences from the learned model are often treated as equivalent to experience from the real environment. However, when the model is inaccurate, it can catastrophically interfere with policy learning. Alternatively, the agent might learn about the model's accuracy and selectively use it only when it can provide reliable predictions. We empirically explore model uncertainty measures for selective planning and show that best results require distribution insensitive inference to estimate the uncertainty over model-based updates. To that end, we propose and evaluate bounding-box inference, which operates on bounding-boxes around sets of possible states and other quantities. We find that bounding-box inference can reliably support effective selective planning.
△ Less
Submitted 23 June, 2024;
originally announced June 2024.
-
A CNN Approach for 5G mmWave Positioning Using Beamformed CSI Measurements
Authors:
Ghazaleh Kia,
Laura Ruotsalainen,
Jukka Talvitie
Abstract:
The advent of Artificial Intelligence (AI) has impacted all aspects of human life. One of the concrete examples of AI impact is visible in radio positioning. In this article, for the first time we utilize the power of AI by training a Convolutional Neural Network (CNN) using 5G New Radio (NR) fingerprints consisting of beamformed Channel State Information (CSI). By observing CSI, it is possible to…
▽ More
The advent of Artificial Intelligence (AI) has impacted all aspects of human life. One of the concrete examples of AI impact is visible in radio positioning. In this article, for the first time we utilize the power of AI by training a Convolutional Neural Network (CNN) using 5G New Radio (NR) fingerprints consisting of beamformed Channel State Information (CSI). By observing CSI, it is possible to characterize the multipath channel between the transmitter and the receiver, and thus provide a good source of spatiotemporal data to find the position of a User Equipment (UE). We collect ray-tracing-based 5G NR CSI from an urban area. The CSI data of the signals from one Base Station (BS) is collected at the reference points with known positions to train a CNN. We evaluate our work by testing: a) the robustness of the trained network for estimating the positions for the new measurements on the same reference points and b) the accuracy of the CNN-based position estimation while the UE is on points other than the reference points. The results prove that our trained network for a specific urban environment can estimate the UE position with a minimum mean error of 0.98 m.
△ Less
Submitted 30 April, 2022;
originally announced May 2022.
-
Iterated Posterior Linearization PMB Filter for 5G SLAM
Authors:
Yu Ge,
Yibo Wu,
Fan Jiang,
Ossi Kaltiokallio,
Jukka Talvitie,
Mikko Valkama,
Lennart Svensson,
Henk Wymeersch
Abstract:
5G millimeter wave (mmWave) signals have inherent geometric connections to the propagation channel and the propagation environment. Thus, they can be used to jointly localize the receiver and map the propagation environment, which is termed as simultaneous localization and mapping (SLAM). One of the most important tasks in the 5G SLAM is to deal with the nonlinearity of the measurement model. To s…
▽ More
5G millimeter wave (mmWave) signals have inherent geometric connections to the propagation channel and the propagation environment. Thus, they can be used to jointly localize the receiver and map the propagation environment, which is termed as simultaneous localization and mapping (SLAM). One of the most important tasks in the 5G SLAM is to deal with the nonlinearity of the measurement model. To solve this problem, existing 5G SLAM approaches rely on sigma-point or extended Kalman filters, linearizing the measurement function with respect to the prior probability density function (PDF). In this paper, we study the linearization of the measurement function with respect to the posterior PDF, and implement the iterated posterior linearization filter into the Poisson multi-Bernoulli SLAM filter. Simulation results demonstrate the accuracy and precision improvements of the resulting SLAM filter.
△ Less
Submitted 5 December, 2021;
originally announced December 2021.
-
HybridDeepRx: Deep Learning Receiver for High-EVM Signals
Authors:
Jaakko Pihlajasalo,
Dani Korpi,
Mikko Honkala,
Janne M. J. Huttunen,
Taneli Riihonen,
Jukka Talvitie,
Alberto Brihuega,
Mikko A. Uusitalo,
Mikko Valkama
Abstract:
In this paper, we propose a machine learning (ML) based physical layer receiver solution for demodulating OFDM signals that are subject to a high level of nonlinear distortion. Specifically, a novel deep learning based convolutional neural network receiver is devised, containing layers in both time- and frequency domains, allowing to demodulate and decode the transmitted bits reliably despite the…
▽ More
In this paper, we propose a machine learning (ML) based physical layer receiver solution for demodulating OFDM signals that are subject to a high level of nonlinear distortion. Specifically, a novel deep learning based convolutional neural network receiver is devised, containing layers in both time- and frequency domains, allowing to demodulate and decode the transmitted bits reliably despite the high error vector magnitude (EVM) in the transmit signal. Extensive set of numerical results is provided, in the context of 5G NR uplink incorporating also measured terminal power amplifier characteristics. The obtained results show that the proposed receiver system is able to clearly outperform classical linear receivers as well as existing ML receiver approaches, especially when the EVM is high in comparison with modulation order. The proposed ML receiver can thus facilitate pushing the terminal power amplifier (PA) systems deeper into saturation, and thereon improve the terminal power-efficiency, radiated power and network coverage.
△ Less
Submitted 30 June, 2021;
originally announced June 2021.
-
Selective Dyna-style Planning Under Limited Model Capacity
Authors:
Zaheer Abbas,
Samuel Sokota,
Erin J. Talvitie,
Martha White
Abstract:
In model-based reinforcement learning, planning with an imperfect model of the environment has the potential to harm learning progress. But even when a model is imperfect, it may still contain information that is useful for planning. In this paper, we investigate the idea of using an imperfect model selectively. The agent should plan in parts of the state space where the model would be helpful but…
▽ More
In model-based reinforcement learning, planning with an imperfect model of the environment has the potential to harm learning progress. But even when a model is imperfect, it may still contain information that is useful for planning. In this paper, we investigate the idea of using an imperfect model selectively. The agent should plan in parts of the state space where the model would be helpful but refrain from using the model where it would be harmful. An effective selective planning mechanism requires estimating predictive uncertainty, which arises out of aleatoric uncertainty, parameter uncertainty, and model inadequacy, among other sources. Prior work has focused on parameter uncertainty for selective planning. In this work, we emphasize the importance of model inadequacy. We show that heteroscedastic regression can signal predictive uncertainty arising from model inadequacy that is complementary to that which is detected by methods designed for parameter uncertainty, indicating that considering both parameter uncertainty and model inadequacy may be a more promising direction for effective selective planning than either in isolation.
△ Less
Submitted 7 March, 2021; v1 submitted 5 July, 2020;
originally announced July 2020.
-
Positioning and Location-Aware Communications for Modern Railways with 5G New Radio
Authors:
Jukka Talvitie,
Toni Levanen,
Mike Koivisto,
Tero Ihalainen,
Kari Pajukoski,
Mikko Valkama
Abstract:
Providing high-capacity radio connectivity for high-speed trains (HSTs) is one of the most important use cases of emerging 5G New Radio (NR) networks. In this article, we show that 5G NR technology can also facilitate high-accuracy continuous localization and tracking of HSTs. Furthermore, we describe and demonstrate how the NR network can utilize the continuous location information for efficient…
▽ More
Providing high-capacity radio connectivity for high-speed trains (HSTs) is one of the most important use cases of emerging 5G New Radio (NR) networks. In this article, we show that 5G NR technology can also facilitate high-accuracy continuous localization and tracking of HSTs. Furthermore, we describe and demonstrate how the NR network can utilize the continuous location information for efficient beam-management and beamforming, as well as for downlink Doppler precompensation in the single-frequency network context. Additionally, with particular focus on millimeter wave networks, novel concepts for low-latency intercarrier interference (ICI) estimation and compensation, due to residual Doppler and oscillator phase noise, are described and demonstrated. The provided numerical results at 30 GHz operating band show that sub-meter positioning and sub-degree beam-direction accuracies can be obtained with very high probabilities in the order of 95-99%. The results also show that the described Doppler precompensation and ICI estimation and cancellation methods substantially improve the throughput of the single-frequency HST network.
△ Less
Submitted 6 May, 2019; v1 submitted 29 April, 2019;
originally announced May 2019.
-
The Effect of Planning Shape on Dyna-style Planning in High-dimensional State Spaces
Authors:
G. Zacharias Holland,
Erin J. Talvitie,
Michael Bowling
Abstract:
Dyna is a fundamental approach to model-based reinforcement learning (MBRL) that interleaves planning, acting, and learning in an online setting. In the most typical application of Dyna, the dynamics model is used to generate one-step transitions from selected start states from the agent's history, which are used to update the agent's value function or policy as if they were real experiences. In t…
▽ More
Dyna is a fundamental approach to model-based reinforcement learning (MBRL) that interleaves planning, acting, and learning in an online setting. In the most typical application of Dyna, the dynamics model is used to generate one-step transitions from selected start states from the agent's history, which are used to update the agent's value function or policy as if they were real experiences. In this work, one-step Dyna was applied to several games from the Arcade Learning Environment (ALE). We found that the model-based updates offered surprisingly little benefit over simply performing more updates with the agent's existing experience, even when using a perfect model. We hypothesize that to get the most from planning, the model must be used to generate unfamiliar experience. To test this, we experimented with the "shape" of planning in multiple different concrete instantiations of Dyna, performing fewer, longer rollouts, rather than many short rollouts. We found that planning shape has a profound impact on the efficacy of Dyna for both perfect and learned models. In addition to these findings regarding Dyna in general, our results represent, to our knowledge, the first time that a learned dynamics model has been successfully used for planning in the ALE, suggesting that Dyna may be a viable approach to MBRL in the ALE and other high-dimensional problems.
△ Less
Submitted 28 March, 2019; v1 submitted 5 June, 2018;
originally announced June 2018.
-
Positioning of High-speed Trains using 5G New Radio Synchronization Signals
Authors:
Jukka Talvitie,
Toni Levanen,
Mike Koivisto,
Kari Pajukoski,
Markku Renfors,
Mikko Valkama
Abstract:
We study positioning of high-speed trains in 5G new radio (NR) networks by utilizing specific NR synchronization signals. The studies are based on simulations with 3GPP-specified radio channel models including path loss, shadowing and fast fading effects. The considered positioning approach exploits measurement of Time-Of-Arrival (TOA) and Angle-Of-Departure (AOD), which are estimated from beamfor…
▽ More
We study positioning of high-speed trains in 5G new radio (NR) networks by utilizing specific NR synchronization signals. The studies are based on simulations with 3GPP-specified radio channel models including path loss, shadowing and fast fading effects. The considered positioning approach exploits measurement of Time-Of-Arrival (TOA) and Angle-Of-Departure (AOD), which are estimated from beamformed NR synchronization signals. Based on the given measurements and the assumed train movement model, the train position is tracked by using an Extended Kalman Filter (EKF), which is able to handle the non-linear relationship between the TOA and AOD measurements, and the estimated train position parameters. It is shown that in the considered scenario the TOA measurements are able to achieve better accuracy compared to the AOD measurements. However, as shown by the results, the best tracking performance is achieved, when both of the measurements are considered. In this case, a very high, sub-meter, tracking accuracy can be achieved for most (>75%) of the tracking time, thus achieving the positioning accuracy requirements envisioned for the 5G NR. The pursued high-accuracy and high-availability positioning technology is considered to be in a key role in several envisioned HST use cases, such as mission-critical autonomous train systems.
△ Less
Submitted 4 May, 2018;
originally announced May 2018.
-
Joint Device Positioning and Clock Synchronization in 5G Ultra-Dense Networks
Authors:
Mike Koivisto,
Mário Costa,
Janis Werner,
Kari Heiska,
Jukka Talvitie,
Kari Leppänen,
Visa Koivunen,
Mikko Valkama
Abstract:
In this article, we address the prospects and key enabling technologies for highly efficient and accurate device positioning and tracking in 5G radio access networks. Building on the premises of ultra-dense networks as well as on the adoption of multicarrier waveforms and antenna arrays in the access nodes (ANs), we first formulate extended Kalman filter (EKF)-based solutions for computationally e…
▽ More
In this article, we address the prospects and key enabling technologies for highly efficient and accurate device positioning and tracking in 5G radio access networks. Building on the premises of ultra-dense networks as well as on the adoption of multicarrier waveforms and antenna arrays in the access nodes (ANs), we first formulate extended Kalman filter (EKF)-based solutions for computationally efficient joint estimation and tracking of the time of arrival (ToA) and direction of arrival (DoA) of the user nodes (UNs) using uplink reference signals. Then, a second EKF stage is proposed in order to fuse the individual DoA/ToA estimates from one or several ANs into a UN position estimate. Since all the processing takes place at the network side, the computing complexity and energy consumption at the UN side are kept to a minimum. The cascaded EKFs proposed in this article also take into account the unavoidable relative clock offsets between UNs and ANs, such that reliable clock synchronization of the access-link is obtained as a valuable by-product. The proposed cascaded EKF scheme is then revised and extended to more general and challenging scenarios where not only the UNs have clock offsets against the network time, but also the ANs themselves are not mutually synchronized in time. Finally, comprehensive performance evaluations of the proposed solutions on a realistic 5G network setup, building on the METIS project based outdoor Madrid map model together with complete ray tracing based propagation modeling, are provided. The obtained results clearly demonstrate that by using the developed methods, sub-meter scale positioning and tracking accuracy of moving devices is indeed technically feasible in future 5G radio access networks operating at sub-6GHz frequencies, despite the realistic assumptions related to clock offsets and potentially even under unsynchronized network elements.
△ Less
Submitted 24 November, 2016; v1 submitted 12 April, 2016;
originally announced April 2016.
