Title:Learning Loosely Connected Markov Random Fields

Abstract:We consider the structure learning problem for graphical models that we call loosely connected Markov random fields, in which the number of short paths between any pair of nodes is small. We point out that many previously studied models are examples of this family. However, due to the existence of short cycles, some previous methods fail to detect all the edges in some of these graphical models. We present a new algorithm for learning the structure of loosely connected Markov random fields from i.i.d. samples. The key step for the algorithm is a max-min conditional independence test, in which the maximization step is to detect the edges while the minimization step is to detect non-edges. The maximization step explicitly breaks the short cycles that can cause problems in edge detection. We show that, under certain non-degeneracy conditions, our algorithm learns the graph correctly with high probability using n = O(log p) samples, where p is the size of the graph. For models with at most D_1 short paths between non-neighbor nodes and D_2 non-direct paths between neighboring nodes, the running time of our algorithm is O(np^{D_1+D_2+2}). If the Markov random field has correlation decay and satisfies pairwise non-degeneracy condition, an extended algorithm can be applied and the running time is reduced to O(np^2) with a preprocessing step. If the MRF is a ferromagnetic Ising model, we can remove the maximization step, which gives running time O(np^{D_1+2}), and the extended algorithm can be applied. In several special cases of loosely connected Markov random fields, our algorithm achieves the same or lower computational complexity than the previously designed algorithms for individual cases. We also get new results for more general graphical models, in particular, our algorithm learns general Ising models on the Erdos-Renyi random graph G(p, c/p) correctly with running time O(np^5).