Skip to main content

Showing 1–4 of 4 results for author: Shafto, P

Searching in archive cond-mat. Search in all archives.
.
  1. arXiv:2203.07411  [pdf, other

    cs.LG cond-mat.dis-nn stat.ML

    On Connecting Deep Trigonometric Networks with Deep Gaussian Processes: Covariance, Expressivity, and Neural Tangent Kernel

    Authors: Chi-Ken Lu, Patrick Shafto

    Abstract: Deep Gaussian Process (DGP) as a model prior in Bayesian learning intuitively exploits the expressive power in function composition. DGPs also offer diverse modeling capabilities, but inference is challenging because marginalization in latent function space is not tractable. With Bochner's theorem, DGP with squared exponential kernel can be viewed as a deep trigonometric network consisting of the… ▽ More

    Submitted 30 July, 2022; v1 submitted 14 March, 2022; originally announced March 2022.

    Comments: version 2 contains new analysis on finite width effect on kernel and NTK, and more connection to other works; v3 includes numerical simulation connecting exact DGP predictive mean with weight space results

  2. arXiv:2002.02826  [pdf, other

    cs.LG cond-mat.dis-nn stat.ML

    Conditional Deep Gaussian Processes: multi-fidelity kernel learning

    Authors: Chi-Ken Lu, Patrick Shafto

    Abstract: Deep Gaussian Processes (DGPs) were proposed as an expressive Bayesian model capable of a mathematically grounded estimation of uncertainty. The expressivity of DPGs results from not only the compositional character but the distribution propagation within the hierarchy. Recently, [1] pointed out that the hierarchical structure of DGP well suited modeling the multi-fidelity regression, in which one… ▽ More

    Submitted 1 October, 2021; v1 submitted 7 February, 2020; originally announced February 2020.

    Comments: Submitted on Oct 1st 2021 with a sequel titled "Conditional Deep Gaussian Process: empirical Bayes hyperdata learning"

    Journal ref: Entropy 2021, 23(11), 1545

  3. arXiv:1905.10963  [pdf, other

    cs.LG cond-mat.dis-nn stat.ML

    Interpretable deep Gaussian processes with moments

    Authors: Chi-Ken Lu, Scott Cheng-Hsin Yang, Xiaoran Hao, Patrick Shafto

    Abstract: Deep Gaussian Processes (DGPs) combine the expressiveness of Deep Neural Networks (DNNs) with quantified uncertainty of Gaussian Processes (GPs). Expressive power and intractable inference both result from the non-Gaussian distribution over composition functions. We propose interpretable DGP based on approximating DGP as a GP by calculating the exact moments, which additionally identify the heavy-… ▽ More

    Submitted 3 December, 2019; v1 submitted 26 May, 2019; originally announced May 2019.

    Comments: Preprint with 12 pages and 3 figures. The updated version (Oct 9 2019) consider the second and fourth moments, inspecting the heavy-tailed nature of DGP distribution, justifying the validity of approximating DGP as GP. A connection with the expressivity parameter in Poole et al NIPS paper is also added. New reference and 4th moments of SC[]

    Journal ref: Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics (AISTATS) 2020

  4. arXiv:1803.03666  [pdf, other

    stat.ML cond-mat.dis-nn cs.LG

    Standing Wave Decomposition Gaussian Process

    Authors: Chi-Ken Lu, Scott Cheng-Hsin Yang, Patrick Shafto

    Abstract: We propose a Standing Wave Decomposition (SWD) approximation to Gaussian Process regression (GP). GP involves a costly matrix inversion operation, which limits applicability to large data analysis. For an input space that can be approximated by a grid and when correlations among data are short-ranged, the kernel matrix inversion can be replaced by analytic diagonalization using the SWD. We show th… ▽ More

    Submitted 17 September, 2018; v1 submitted 9 March, 2018; originally announced March 2018.

    Comments: 10 pages, 8 figures; updated version includes a modified introduction and a new discussion on time complexity of our approximated GP method. New references are added. Simulation package will be announced later; updated with discussion of validity of perturbation treatment of Eq. (25) with added Fig. 6 as evidence; simulation code at https://github.com/CoDaS-Lab/LG-SWD-GP

    Journal ref: Phys. Rev. E 98, 032303 (2018)