Showing 1–14 of 14 results for author: Molkaraie, M

On the Equivalence of Gaussian Graphical Models Defined on Complete Bipartite Graphs

Authors: Mehdi Molkaraie

Abstract: This paper introduces two Gaussian graphical models defined on complete bipartite graphs. We show that the determinants of the precision matrices associated with the models are equal up to scale, where the scale factor only depends on model parameters. In this context, we will introduce a notion of ``equivalence" between the two Gaussian graphical models. This equivalence has two key applications:… ▽ More This paper introduces two Gaussian graphical models defined on complete bipartite graphs. We show that the determinants of the precision matrices associated with the models are equal up to scale, where the scale factor only depends on model parameters. In this context, we will introduce a notion of ``equivalence" between the two Gaussian graphical models. This equivalence has two key applications: first, it can significantly reduce the complexity of computing the exact value of the determinant, and second, it enables the derivation of closed-form expressions for the determinants in certain special cases. △ Less

Submitted 16 June, 2025; v1 submitted 5 May, 2025; originally announced May 2025.

Comments: IEEE International Symposium on Information Theory (ISIT), Ann Arbor, USA, June 2025

The Exact Determinant of a Specific Class of Sparse Positive Definite Matrices

Authors: Mehdi Molkaraie

Abstract: For a specific class of sparse Gaussian graphical models, we provide a closed-form solution for the determinant of the covariance matrix. In our framework, the graphical interaction model (i.e., the covariance selection model) is equal to replacement product of $\mathcal{K}_{n}$ and $\mathcal{K}_{n-1}$, where $\mathcal{K}_n$ is the complete graph with $n$ vertices. Our analysis is based on taking… ▽ More For a specific class of sparse Gaussian graphical models, we provide a closed-form solution for the determinant of the covariance matrix. In our framework, the graphical interaction model (i.e., the covariance selection model) is equal to replacement product of $\mathcal{K}_{n}$ and $\mathcal{K}_{n-1}$, where $\mathcal{K}_n$ is the complete graph with $n$ vertices. Our analysis is based on taking the Fourier transform of the local factors of the model, which can be viewed as an application of the Normal Factor Graph Duality Theorem and holographic algorithms. The closed-form expression is obtained by applying the Matrix Determinant Lemma on the transformed graphical model. In this context, we will also define a notion of equivalence between two Gaussian graphical models. △ Less

Submitted 11 November, 2023; originally announced November 2023.

Comments: Proc. of the 2023 IEEE International Symposium on Information Theory (ISIT), Taipei, Taiwan

Mappings for Marginal Probabilities with Applications to Models in Statistical Physics

Authors: Mehdi Molkaraie

Abstract: We present local mappings that relate the marginal probabilities of a global probability mass function represented by its primal normal factor graph to the corresponding marginal probabilities in its dual normal factor graph. The mapping is based on the Fourier transform of the local factors of the models. Details of the mapping are provided for the Ising model, where it is proved that the local e… ▽ More We present local mappings that relate the marginal probabilities of a global probability mass function represented by its primal normal factor graph to the corresponding marginal probabilities in its dual normal factor graph. The mapping is based on the Fourier transform of the local factors of the models. Details of the mapping are provided for the Ising model, where it is proved that the local extrema of the fixed points are attained at the phase transition of the two-dimensional nearest-neighbor Ising model. The results are further extended to the Potts model, to the clock model, and to Gaussian Markov random fields. By employing the mapping, we can transform simultaneously all the estimated marginal probabilities from the dual domain to the primal domain (and vice versa), which is advantageous if estimating the marginals can be carried out more efficiently in the dual domain. An example of particular significance is the ferromagnetic Ising model in a positive external magnetic field. For this model, there exists a rapidly mixing Markov chain (called the subgraphs--world process) to generate configurations in the dual normal factor graph of the model. Our numerical experiments illustrate that the proposed procedure can provide more accurate estimates of marginal probabilities of a global probability mass function in various settings. △ Less

Submitted 10 August, 2022; originally announced August 2022.

Comments: 36 pages

Journal ref: The Journal of Machine Learning Research (JMLR), August 2022

Duality for Continuous Graphical Models

Authors: Mehdi Molkaraie

Abstract: The dual normal factor graph and the factor graph duality theorem have been considered for discrete graphical models. In this paper, we show an application of the factor graph duality theorem to continuous graphical models. Specifically, we propose a method to solve exactly the Gaussian graphical models defined on the ladder graph if certain conditions on the local covariance matrices are satisfie… ▽ More The dual normal factor graph and the factor graph duality theorem have been considered for discrete graphical models. In this paper, we show an application of the factor graph duality theorem to continuous graphical models. Specifically, we propose a method to solve exactly the Gaussian graphical models defined on the ladder graph if certain conditions on the local covariance matrices are satisfied. Unlike the conventional approaches, the efficiency of the method depends on the position of the zeros in the local covariance matrices. The method and details of the dualization are illustrated on two toy examples. △ Less

Submitted 2 November, 2021; originally announced November 2021.

Comments: Proc. of the 2021 IEEE Information Theory Workshop (ITW2021), Kanazawa, Japan

Marginal Densities, Factor Graph Duality, and High-Temperature Series Expansions

Authors: Mehdi Molkaraie

Abstract: We prove that the marginal densities of a global probability mass function in a primal normal factor graph and the corresponding marginal densities in the dual normal factor graph are related via local mappings. The mapping depends on the Fourier transform of the local factors of the models. Details of the mapping, including its fixed points, are derived for the Ising model, and then extended to t… ▽ More We prove that the marginal densities of a global probability mass function in a primal normal factor graph and the corresponding marginal densities in the dual normal factor graph are related via local mappings. The mapping depends on the Fourier transform of the local factors of the models. Details of the mapping, including its fixed points, are derived for the Ising model, and then extended to the Potts model. By employing the mapping, we can transform simultaneously all the estimated marginal densities from one domain to the other, which is advantageous if estimating the marginals can be carried out more efficiently in the dual domain. An example of particular significance is the ferromagnetic Ising model in a positive external field, for which there is a rapidly mixing Markov chain (called the subgraphs-world process) to generate configurations in the dual normal factor graph of the model. Our numerical experiments illustrate that the proposed procedure can provide more accurate estimates of marginal densities in various settings. △ Less

Submitted 13 July, 2020; v1 submitted 6 January, 2019; originally announced January 2019.

Comments: Proc. of the 23rd International Conference on Artificial Intelligence and Statistics (AISTATS) 2020, Palermo, Italy. Extended version

The Primal versus the Dual Ising Model

Authors: Mehdi Molkaraie

Abstract: We represent the Ising model of statistical physics by normal factor graphs in the primal and in the dual domains. By analogy with Kirchhoff's voltage and current laws, we show that in the primal normal factor graphs, the dependency among the variables is along the cycles, whereas in the dual normal factor graphs, the dependency is on the cutsets. In the primal (resp. dual) domain, dependent varia… ▽ More We represent the Ising model of statistical physics by normal factor graphs in the primal and in the dual domains. By analogy with Kirchhoff's voltage and current laws, we show that in the primal normal factor graphs, the dependency among the variables is along the cycles, whereas in the dual normal factor graphs, the dependency is on the cutsets. In the primal (resp. dual) domain, dependent variables can be computed via their fundamental cycles (resp. fundamental cutsets). Using Onsager's closed form solution, we illustrate the opposite behavior of the uniform sampling estimator for estimating the partition function in the primal and in the dual of the homogeneous Ising model on a two-dimensional torus. △ Less

Submitted 22 July, 2018; v1 submitted 4 July, 2016; originally announced July 2016.

Comments: Proc. 55th Annual Allerton Conference on Communication, Control, and Computing, Monticello, USA, Oct 3-6, 2017

An Importance Sampling Scheme for Models in a Strong External Field

Authors: Mehdi Molkaraie

Abstract: We propose Monte Carlo methods to estimate the partition function of the two-dimensional Ising model in the presence of an external magnetic field. The estimation is done in the dual of the Forney factor graph representing the model. The proposed methods can efficiently compute an estimate of the partition function in a wide range of model parameters. As an example, we consider models that are in… ▽ More We propose Monte Carlo methods to estimate the partition function of the two-dimensional Ising model in the presence of an external magnetic field. The estimation is done in the dual of the Forney factor graph representing the model. The proposed methods can efficiently compute an estimate of the partition function in a wide range of model parameters. As an example, we consider models that are in a strong external field. △ Less

Submitted 29 December, 2015; originally announced December 2015.

Comments: Proc. IEEE Int. Symp. on Information Theory (ISIT), Hong Kong, June 14-19, 2015. arXiv admin note: text overlap with arXiv:1401.4912

An Importance Sampling Algorithm for the Ising Model with Strong Couplings

Authors: Mehdi Molkaraie

Abstract: We consider the problem of estimating the partition function of the ferromagnetic Ising model in a consistent external magnetic field. The estimation is done via importance sampling in the dual of the Forney factor graph representing the model. Emphasis is on models at low temperature (corresponding to models with strong couplings) and on models with a mixture of strong and weak coupling parameter… ▽ More We consider the problem of estimating the partition function of the ferromagnetic Ising model in a consistent external magnetic field. The estimation is done via importance sampling in the dual of the Forney factor graph representing the model. Emphasis is on models at low temperature (corresponding to models with strong couplings) and on models with a mixture of strong and weak coupling parameters. △ Less

Submitted 26 January, 2017; v1 submitted 22 April, 2014; originally announced April 2014.

Comments: Proc. 2016 Int. Zurich Seminar on Communications (IZS), Zurich, Switzerland, March 2-4, 2016, pp. 180-184

An Importance Sampling Scheme on Dual Factor Graphs. I. Models in a Strong External Field

Authors: Mehdi Molkaraie

Abstract: We propose an importance sampling scheme to estimate the partition function of the two-dimensional ferromagnetic Ising model and the two-dimensional ferromagnetic $q$-state Potts model, both in the presence of an external magnetic field. The proposed scheme operates in the dual Forney factor graph and is capable of efficiently computing an estimate of the partition function under a wide range of m… ▽ More We propose an importance sampling scheme to estimate the partition function of the two-dimensional ferromagnetic Ising model and the two-dimensional ferromagnetic $q$-state Potts model, both in the presence of an external magnetic field. The proposed scheme operates in the dual Forney factor graph and is capable of efficiently computing an estimate of the partition function under a wide range of model parameters. In particular, we consider models that are in a strong external magnetic field. △ Less

Submitted 5 February, 2015; v1 submitted 20 January, 2014; originally announced January 2014.

Comments: 9 pages

Partition Function of the Ising Model via Factor Graph Duality

Authors: Mehdi Molkaraie, Hans-Andrea Loeliger

Abstract: The partition function of a factor graph and the partition function of the dual factor graph are related to each other by the normal factor graph duality theorem. We apply this result to the classical problem of computing the partition function of the Ising model. In the one-dimensional case, we thus obtain an alternative derivation of the (well-known) analytical solution. In the two-dimensional c… ▽ More The partition function of a factor graph and the partition function of the dual factor graph are related to each other by the normal factor graph duality theorem. We apply this result to the classical problem of computing the partition function of the Ising model. In the one-dimensional case, we thus obtain an alternative derivation of the (well-known) analytical solution. In the two-dimensional case, we find that Monte Carlo methods are much more efficient on the dual graph than on the original graph, especially at low temperature. △ Less

Submitted 13 July, 2013; originally announced July 2013.

Comments: Proc. IEEE Int. Symp. on Information Theory (ISIT), Istanbul, Turkey, July 7-12, 2013

Extending Monte Carlo Methods to Factor Graphs with Negative and Complex Factors

Authors: Mehdi Molkaraie, Hans-Andrea Loeliger

Abstract: The partition function of a factor graph can sometimes be accurately estimated by Monte Carlo methods. In this paper, such methods are extended to factor graphs with negative and complex factors. The partition function of a factor graph can sometimes be accurately estimated by Monte Carlo methods. In this paper, such methods are extended to factor graphs with negative and complex factors. △ Less

Submitted 7 October, 2012; v1 submitted 22 June, 2012; originally announced June 2012.

Comments: Proc. IEEE Information Theory Workshop (ITW), Lausanne, Switzerland, Sept. 3-7, 2012

Monte Carlo Algorithms for the Partition Function and Information Rates of Two-Dimensional Channels

Authors: Mehdi Molkaraie, Hans-Andrea Loeliger

Abstract: The paper proposes Monte Carlo algorithms for the computation of the information rate of two-dimensional source/channel models. The focus of the paper is on binary-input channels with constraints on the allowed input configurations. The problem of numerically computing the information rate, and even the noiseless capacity, of such channels has so far remained largely unsolved. Both problems can be… ▽ More The paper proposes Monte Carlo algorithms for the computation of the information rate of two-dimensional source/channel models. The focus of the paper is on binary-input channels with constraints on the allowed input configurations. The problem of numerically computing the information rate, and even the noiseless capacity, of such channels has so far remained largely unsolved. Both problems can be reduced to computing a Monte Carlo estimate of a partition function. The proposed algorithms use tree-based Gibbs sampling and multilayer (multitemperature) importance sampling. The viability of the proposed algorithms is demonstrated by simulation results. △ Less

Submitted 6 August, 2012; v1 submitted 27 May, 2011; originally announced May 2011.

Journal ref: IEEE Trans. on Information Theory, Volume 59, Jan. 2013, pp. 495-503

On Properties of the Minimum Entropy Sub-tree to Compute Lower Bounds on the Partition Function

Authors: Mehdi Molkaraie, Payam Pakzad

Abstract: Computing the partition function and the marginals of a global probability distribution are two important issues in any probabilistic inference problem. In a previous work, we presented sub-tree based upper and lower bounds on the partition function of a given probabilistic inference problem. Using the entropies of the sub-trees we proved an inequality that compares the lower bounds obtained from… ▽ More Computing the partition function and the marginals of a global probability distribution are two important issues in any probabilistic inference problem. In a previous work, we presented sub-tree based upper and lower bounds on the partition function of a given probabilistic inference problem. Using the entropies of the sub-trees we proved an inequality that compares the lower bounds obtained from different sub-trees. In this paper we investigate the properties of one specific lower bound, namely the lower bound computed by the minimum entropy sub-tree. We also investigate the relationship between the minimum entropy sub-tree and the sub-tree that gives the best lower bound. △ Less

Submitted 2 March, 2011; originally announced March 2011.

Generalized Belief Propagation for the Noiseless Capacity and Information Rates of Run-Length Limited Constraints

Authors: Giovanni Sabato, Mehdi Molkaraie

Abstract: The performance of the generalized belief propagation algorithm for computing the noiseless capacity and mutual information rates of finite-size two-dimensional and three-dimensional run-length limited constraints is investigated. For each constraint, a method is proposed to choose the basic regions and to construct the region graph. Simulation results for the capacity of different constraints as… ▽ More The performance of the generalized belief propagation algorithm for computing the noiseless capacity and mutual information rates of finite-size two-dimensional and three-dimensional run-length limited constraints is investigated. For each constraint, a method is proposed to choose the basic regions and to construct the region graph. Simulation results for the capacity of different constraints as a function of the size of the channel and mutual information rates of different constraints as a function of signal-to-noise ratio are reported. Convergence to the Shannon capacity is also discussed. △ Less

Submitted 27 May, 2011; v1 submitted 17 January, 2011; originally announced January 2011.

Comments: 8 pages, 11 figures

Journal ref: IEEE Transactions on Communications, Volume 60, March 2012, pages 669 - 675