-
Testing for directed information graphs
Authors:
Sina Molavipour,
Germán Bassi,
Mikael Skoglund
Abstract:
In this paper, we study a hypothesis test to determine the underlying directed graph structure of nodes in a network, where the nodes represent random processes and the direction of the links indicate a causal relationship between said processes. Specifically, a k-th order Markov structure is considered for them, and the chosen metric to determine a connection between nodes is the directed informa…
▽ More
In this paper, we study a hypothesis test to determine the underlying directed graph structure of nodes in a network, where the nodes represent random processes and the direction of the links indicate a causal relationship between said processes. Specifically, a k-th order Markov structure is considered for them, and the chosen metric to determine a connection between nodes is the directed information. The hypothesis test is based on the empirically calculated transition probabilities which are used to estimate the directed information. For a single edge, it is proven that the detection probability can be chosen arbitrarily close to one, while the false alarm probability remains negligible. When the test is performed on the whole graph, we derive bounds for the false alarm and detection probabilities, which show that the test is asymptotically optimal by properly setting the threshold test and using a large number of samples. Furthermore, we study how the convergence of the measures relies on the existence of links in the true graph.
△ Less
Submitted 25 August, 2021;
originally announced August 2021.
-
Neural Estimators for Conditional Mutual Information Using Nearest Neighbors Sampling
Authors:
Sina Molavipour,
Germán Bassi,
Mikael Skoglund
Abstract:
The estimation of mutual information (MI) or conditional mutual information (CMI) from a set of samples is a long-standing problem. A recent line of work in this area has leveraged the approximation power of artificial neural networks and has shown improvements over conventional methods. One important challenge in this new approach is the need to obtain, given the original dataset, a different set…
▽ More
The estimation of mutual information (MI) or conditional mutual information (CMI) from a set of samples is a long-standing problem. A recent line of work in this area has leveraged the approximation power of artificial neural networks and has shown improvements over conventional methods. One important challenge in this new approach is the need to obtain, given the original dataset, a different set where the samples are distributed according to a specific product density function. This is particularly challenging when estimating CMI.
In this paper, we introduce a new technique, based on k nearest neighbors (k-NN), to perform the resampling and derive high-confidence concentration bounds for the sample average. Then the technique is employed to train a neural network classifier and the CMI is estimated accordingly. We propose three estimators using this technique and prove their consistency, make a comparison between them and similar approaches in the literature, and experimentally show improvements in estimating the CMI in terms of accuracy and variance of the estimators.
△ Less
Submitted 30 December, 2020; v1 submitted 12 June, 2020;
originally announced June 2020.
-
Conditional Mutual Information Neural Estimator
Authors:
Sina Molavipour,
Germán Bassi,
Mikael Skoglund
Abstract:
Several recent works in communication systems have proposed to leverage the power of neural networks in the design of encoders and decoders. In this approach, these blocks can be tailored to maximize the transmission rate based on aggregated samples from the channel. Motivated by the fact that, in many communication schemes, the achievable transmission rate is determined by a conditional mutual in…
▽ More
Several recent works in communication systems have proposed to leverage the power of neural networks in the design of encoders and decoders. In this approach, these blocks can be tailored to maximize the transmission rate based on aggregated samples from the channel. Motivated by the fact that, in many communication schemes, the achievable transmission rate is determined by a conditional mutual information term, this paper focuses on neural-based estimators for this information-theoretic quantity. Our results are based on variational bounds for the KL-divergence and, in contrast to some previous works, we provide a mathematically rigorous lower bound. However, additional challenges with respect to the unconditional mutual information emerge due to the presence of a conditional density function which we address here.
△ Less
Submitted 14 February, 2020; v1 submitted 6 November, 2019;
originally announced November 2019.
-
Multirate Packet Delivery In Heterogeneous Broadcast Networks
Authors:
Mohammad A. Zarrabian,
Foroogh S. Tabataba,
Sina Molavipour,
Parastoo Sadeghi
Abstract:
In this paper, we study the problem of multirate packet delivery in heterogeneous packet erasure broadcast networks. The technical challenge is to enable users receive packets at different rates, as dictated by the quality of their individual channel. We present a new analytical framework for characterizing the delivery rate and delivery delay performance of a previously proposed non-block-based n…
▽ More
In this paper, we study the problem of multirate packet delivery in heterogeneous packet erasure broadcast networks. The technical challenge is to enable users receive packets at different rates, as dictated by the quality of their individual channel. We present a new analytical framework for characterizing the delivery rate and delivery delay performance of a previously proposed non-block-based network coding scheme in the literature. This scheme was studied in homogeneous network settings. We show for the first time, via new theoretical analysis and simulations that it can actually achieve multirate packet delivery. Using acknowledgments from each user, we show that the user with the highest link capacity achieves the maximum possible throughput. Also, a non-zero packet delivery rate is possible for other users, and the delivery rate depends on the difference between the packet arrival rate at the sender and the link capacity of each user. The accuracy of our analytical framework is confirmed by comparing the results with simulations for different settings of packet arrival rate at the sender and link capacities.
△ Less
Submitted 8 August, 2019; v1 submitted 18 June, 2017;
originally announced June 2017.
