Showing 1–19 of 19 results for author: Eslami, A

Hybrid methods for missing categorical covariates in Cox model

Authors: Abdoulaye Dioni, Lynne Moore, Aida Eslami

Abstract: Survival analysis aims to explore the relationship between covariates and the time until the occurrence of an event. The Cox proportional hazards model is commonly used for right-censored data, but it is not strictly limited to this type of data. However, the presence of missing values among the covariates, particularly categorical ones, can compromise the validity of the estimates. To address thi… ▽ More Survival analysis aims to explore the relationship between covariates and the time until the occurrence of an event. The Cox proportional hazards model is commonly used for right-censored data, but it is not strictly limited to this type of data. However, the presence of missing values among the covariates, particularly categorical ones, can compromise the validity of the estimates. To address this issue, various classical methods for handling missing data have been proposed within the Cox model framework, including parametric imputation, nonparametric imputation, and semiparametric methods. It is well-documented that none of these methods is universally ideal or optimal, making the choice of the preferred method often complex and challenging. To overcome these limitations, we propose hybrid methods that combine the advantages of classical methods to enhance the robustness of the analyses. Through a simulation study, we demonstrate that these hybrid methods provide increased flexibility, simplified implementation, and improved robustness compared to classical methods. The results from the simulation study highlight that hybrid methods offer increased flexibility, simplified implementation, and greater robustness compared to classical approaches. In particular, they allow for a reduction in estimation bias; however, this improvement comes at the cost of reduced precision, due to increased variability. This observation reflects a well-known methodological trade-off between bias and variance, inherent to the combination of complementary imputation strategies. △ Less

Submitted 30 June, 2025; originally announced July 2025.

Sensitivity analysis method in the presence of a missing not at random ordinal independent variable

Authors: Abdoulaye Dioni, Alexandre Bureau, Lynne Moore, Aida Eslami

Abstract: Data analysis often encounters missing data, which can result in inaccurate conclusions, especially when it comes to ordinal variables. In trauma data, the Glasgow Coma Scale is useful for assessing the level of consciousness. This score is often missing in patients who are intubated or under sedation upon arrival at the hospital, and those with normal reactivity without head injury, suggesting a… ▽ More Data analysis often encounters missing data, which can result in inaccurate conclusions, especially when it comes to ordinal variables. In trauma data, the Glasgow Coma Scale is useful for assessing the level of consciousness. This score is often missing in patients who are intubated or under sedation upon arrival at the hospital, and those with normal reactivity without head injury, suggesting a Missing Not At Random (MNAR) mechanism. The problem with MNAR is the absence of a definitive analysis. While sensitivity analysis is often recommended, practical limitations sometimes restrict the analysis to a basic comparison between results under Missing Completely At Random (MCAR) and Missing At Random (MAR) assumptions, disregarding MNAR plausibility. Our objective is to propose a flexible and accessible sensitivity analysis method in the presence of a MNAR ordinal independent variable. The method is inspired by the sensitivity analysis approach proposed by Leurent et al. (2018) for a continuous response variable. We propose an extension for an independent ordinal variable. The method is evaluated on simulated data before being applied to Pan-Canadian trauma data from April 2013 to March 2018. The simulation shows that MNAR estimates are less biased than MAR estimates and more precise than complete case analysis (CC) estimates. The confidence intervals coverage rates are relatively better for MNAR estimates than CC and MAR estimates. In the application, it is observed that the Glasgow Coma Scale is significant under MNAR, unlike MCAR and MAR assumptions. △ Less

Submitted 30 June, 2025; originally announced June 2025.

Estimation of the attributable fraction for time to event outcomes using an inverse probability of exposure weighted Kaplan-Meier estimator

Authors: Denis Talbot, Miceline Mésidor, Kossi Clément Trenou, Mathilde Lavigne-Robichaud, Xavier Trudel, Aida Eslami

Abstract: Population attributable fractions aim to quantify the proportion of the cases of an outcome (for example, a disease) that would have been avoided had no individuals in the population been exposed to a given exposure. This quantity thus plays a crucial role in epidemiology and public health, notably to guide policies, interventions or to assess the burden of a disease due to a particular exposure.… ▽ More Population attributable fractions aim to quantify the proportion of the cases of an outcome (for example, a disease) that would have been avoided had no individuals in the population been exposed to a given exposure. This quantity thus plays a crucial role in epidemiology and public health, notably to guide policies, interventions or to assess the burden of a disease due to a particular exposure. Various statistical methods have been proposed to estimate attributable fractions using observational data. When time-to-event data are used, several of these formulas yield invalid results. Alternative valid formulas are available but remain scarcely used. We propose a new estimator of the attributable fraction that is both conceptually simple and easy to implement using common statistical software. Our proposed estimator makes use of the Kaplan-Meier estimator to address censoring and potentially non-proportional hazards, as well as inverse probability weighting to control confounding. Nonparametric bootstrap is proposed to produce inferences. A simulation study is used to illustrate and compare our proposed estimator to several alternatives. The results showcase the bias of many commonly used traditional approaches and the validity of our estimator under its working assumptions. △ Less

Submitted 19 December, 2022; originally announced December 2022.

Comments: 15 pages, 0 figure

Contrastive Training for Improved Out-of-Distribution Detection

Authors: Jim Winkens, Rudy Bunel, Abhijit Guha Roy, Robert Stanforth, Vivek Natarajan, Joseph R. Ledsam, Patricia MacWilliams, Pushmeet Kohli, Alan Karthikesalingam, Simon Kohl, Taylan Cemgil, S. M. Ali Eslami, Olaf Ronneberger

Abstract: Reliable detection of out-of-distribution (OOD) inputs is increasingly understood to be a precondition for deployment of machine learning systems. This paper proposes and investigates the use of contrastive training to boost OOD detection performance. Unlike leading methods for OOD detection, our approach does not require access to examples labeled explicitly as OOD, which can be difficult to coll… ▽ More Reliable detection of out-of-distribution (OOD) inputs is increasingly understood to be a precondition for deployment of machine learning systems. This paper proposes and investigates the use of contrastive training to boost OOD detection performance. Unlike leading methods for OOD detection, our approach does not require access to examples labeled explicitly as OOD, which can be difficult to collect in practice. We show in extensive experiments that contrastive training significantly helps OOD detection performance on a number of common benchmarks. By introducing and employing the Confusion Log Probability (CLP) score, which quantifies the difficulty of the OOD detection task by capturing the similarity of inlier and outlier datasets, we show that our method especially improves performance in the `near OOD' classes -- a particularly challenging setting for previous methods. △ Less

Submitted 10 July, 2020; originally announced July 2020.

PolyGen: An Autoregressive Generative Model of 3D Meshes

Authors: Charlie Nash, Yaroslav Ganin, S. M. Ali Eslami, Peter W. Battaglia

Abstract: Polygon meshes are an efficient representation of 3D geometry, and are of central importance in computer graphics, robotics and games development. Existing learning-based approaches have avoided the challenges of working with 3D meshes, instead using alternative object representations that are more compatible with neural architectures and training approaches. We present an approach which models th… ▽ More Polygon meshes are an efficient representation of 3D geometry, and are of central importance in computer graphics, robotics and games development. Existing learning-based approaches have avoided the challenges of working with 3D meshes, instead using alternative object representations that are more compatible with neural architectures and training approaches. We present an approach which models the mesh directly, predicting mesh vertices and faces sequentially using a Transformer-based architecture. Our model can condition on a range of inputs, including object classes, voxels, and images, and because the model is probabilistic it can produce samples that capture uncertainty in ambiguous scenarios. We show that the model is capable of producing high-quality, usable meshes, and establish log-likelihood benchmarks for the mesh-modelling task. We also evaluate the conditional models on surface reconstruction metrics against alternative methods, and demonstrate competitive performance despite not training directly on this task. △ Less

Submitted 23 February, 2020; originally announced February 2020.

Unsupervised Doodling and Painting with Improved SPIRAL

Authors: John F. J. Mellor, Eunbyung Park, Yaroslav Ganin, Igor Babuschkin, Tejas Kulkarni, Dan Rosenbaum, Andy Ballard, Theophane Weber, Oriol Vinyals, S. M. Ali Eslami

Abstract: We investigate using reinforcement learning agents as generative models of images (extending arXiv:1804.01118). A generative agent controls a simulated painting environment, and is trained with rewards provided by a discriminator network simultaneously trained to assess the realism of the agent's samples, either unconditional or reconstructions. Compared to prior work, we make a number of improvem… ▽ More We investigate using reinforcement learning agents as generative models of images (extending arXiv:1804.01118). A generative agent controls a simulated painting environment, and is trained with rewards provided by a discriminator network simultaneously trained to assess the realism of the agent's samples, either unconditional or reconstructions. Compared to prior work, we make a number of improvements to the architectures of the agents and discriminators that lead to intriguing and at times surprising results. We find that when sufficiently constrained, generative agents can learn to produce images with a degree of visual abstraction, despite having only ever seen real photographs (no human brush strokes). And given enough time with the painting environment, they can produce images with considerable realism. These results show that, under the right circumstances, some aspects of human drawing can emerge from simulated embodiment, without the need for external supervision, imitation or social cues. Finally, we note the framework's potential for use in creative applications. △ Less

Submitted 2 October, 2019; originally announced October 2019.

Comments: See https://learning-to-paint.github.io for an interactive version of this paper, with videos

ACM Class: I.2; I.4

Learning Interpretable Disentangled Representations using Adversarial VAEs

Authors: Mhd Hasan Sarhan, Abouzar Eslami, Nassir Navab, Shadi Albarqouni

Abstract: Learning Interpretable representation in medical applications is becoming essential for adopting data-driven models into clinical practice. It has been recently shown that learning a disentangled feature representation is important for a more compact and explainable representation of the data. In this paper, we introduce a novel adversarial variational autoencoder with a total correlation constrai… ▽ More Learning Interpretable representation in medical applications is becoming essential for adopting data-driven models into clinical practice. It has been recently shown that learning a disentangled feature representation is important for a more compact and explainable representation of the data. In this paper, we introduce a novel adversarial variational autoencoder with a total correlation constraint to enforce independence on the latent representation while preserving the reconstruction fidelity. Our proposed method is validated on a publicly available dataset showing that the learned disentangled representation is not only interpretable, but also superior to the state-of-the-art methods. We report a relative improvement of 81.50% in terms of disentanglement, 11.60% in clustering, and 2% in supervised classification with a few amounts of labeled data. △ Less

Submitted 17 April, 2019; originally announced April 2019.

Meta-Learning surrogate models for sequential decision making

Authors: Alexandre Galashov, Jonathan Schwarz, Hyunjik Kim, Marta Garnelo, David Saxton, Pushmeet Kohli, S. M. Ali Eslami, Yee Whye Teh

Abstract: We introduce a unified probabilistic framework for solving sequential decision making problems ranging from Bayesian optimisation to contextual bandits and reinforcement learning. This is accomplished by a probabilistic model-based approach that explains observed data while capturing predictive uncertainty during the decision making process. Crucially, this probabilistic model is chosen to be a Me… ▽ More We introduce a unified probabilistic framework for solving sequential decision making problems ranging from Bayesian optimisation to contextual bandits and reinforcement learning. This is accomplished by a probabilistic model-based approach that explains observed data while capturing predictive uncertainty during the decision making process. Crucially, this probabilistic model is chosen to be a Meta-Learning system that allows learning from a distribution of related problems, allowing data efficient adaptation to a target task. As a suitable instantiation of this framework, we explore the use of Neural processes due to statistical and computational desiderata. We apply our framework to a broad range of problem domains, such as control problems, recommender systems and adversarial attacks on RL agents, demonstrating an efficient and general black-box learning approach. △ Less

Submitted 12 June, 2019; v1 submitted 28 March, 2019; originally announced March 2019.

Attentive Neural Processes

Authors: Hyunjik Kim, Andriy Mnih, Jonathan Schwarz, Marta Garnelo, Ali Eslami, Dan Rosenbaum, Oriol Vinyals, Yee Whye Teh

Abstract: Neural Processes (NPs) (Garnelo et al 2018a;b) approach regression by learning to map a context set of observed input-output pairs to a distribution over regression functions. Each function models the distribution of the output given an input, conditioned on the context. NPs have the benefit of fitting observed data efficiently with linear complexity in the number of context input-output pairs, an… ▽ More Neural Processes (NPs) (Garnelo et al 2018a;b) approach regression by learning to map a context set of observed input-output pairs to a distribution over regression functions. Each function models the distribution of the output given an input, conditioned on the context. NPs have the benefit of fitting observed data efficiently with linear complexity in the number of context input-output pairs, and can learn a wide family of conditional distributions; they learn predictive distributions conditioned on context sets of arbitrary size. Nonetheless, we show that NPs suffer a fundamental drawback of underfitting, giving inaccurate predictions at the inputs of the observed data they condition on. We address this issue by incorporating attention into NPs, allowing each input location to attend to the relevant context points for the prediction. We show that this greatly improves the accuracy of predictions, results in noticeably faster training, and expands the range of functions that can be modelled. △ Less

Submitted 9 July, 2019; v1 submitted 17 January, 2019; originally announced January 2019.

Generating Diverse Programs with Instruction Conditioned Reinforced Adversarial Learning

Authors: Aishwarya Agrawal, Mateusz Malinowski, Felix Hill, Ali Eslami, Oriol Vinyals, Tejas Kulkarni

Abstract: Advances in Deep Reinforcement Learning have led to agents that perform well across a variety of sensory-motor domains. In this work, we study the setting in which an agent must learn to generate programs for diverse scenes conditioned on a given symbolic instruction. Final goals are specified to our agent via images of the scenes. A symbolic instruction consistent with the goal images is used as… ▽ More Advances in Deep Reinforcement Learning have led to agents that perform well across a variety of sensory-motor domains. In this work, we study the setting in which an agent must learn to generate programs for diverse scenes conditioned on a given symbolic instruction. Final goals are specified to our agent via images of the scenes. A symbolic instruction consistent with the goal images is used as the conditioning input for our policies. Since a single instruction corresponds to a diverse set of different but still consistent end-goal images, the agent needs to learn to generate a distribution over programs given an instruction. We demonstrate that with simple changes to the reinforced adversarial learning objective, we can learn instruction conditioned policies to achieve the corresponding diverse set of goals. Most importantly, our agent's stochastic policy is shown to more accurately capture the diversity in the goal distribution than a fixed pixel-based reward function baseline. We demonstrate the efficacy of our approach on two domains: (1) drawing MNIST digits with a paint software conditioned on instructions and (2) constructing scenes in a 3D editor that satisfies a certain instruction. △ Less

Submitted 3 December, 2018; originally announced December 2018.

Learning models for visual 3D localization with implicit mapping

Authors: Dan Rosenbaum, Frederic Besse, Fabio Viola, Danilo J. Rezende, S. M. Ali Eslami

Abstract: We consider learning based methods for visual localization that do not require the construction of explicit maps in the form of point clouds or voxels. The goal is to learn an implicit representation of the environment at a higher, more abstract level. We propose to use a generative approach based on Generative Query Networks (GQNs, Eslami et al. 2018), asking the following questions: 1) Can GQN c… ▽ More We consider learning based methods for visual localization that do not require the construction of explicit maps in the form of point clouds or voxels. The goal is to learn an implicit representation of the environment at a higher, more abstract level. We propose to use a generative approach based on Generative Query Networks (GQNs, Eslami et al. 2018), asking the following questions: 1) Can GQN capture more complex scenes than those it was originally demonstrated on? 2) Can GQN be used for localization in those scenes? To study this approach we consider procedurally generated Minecraft worlds, for which we can generate images of complex 3D scenes along with camera pose coordinates. We first show that GQNs, enhanced with a novel attention mechanism can capture the structure of 3D scenes in Minecraft, as evidenced by their samples. We then apply the models to the localization problem, comparing the results to a discriminative baseline, and comparing the ways each approach captures the task uncertainty. △ Less

Submitted 12 December, 2018; v1 submitted 4 July, 2018; originally announced July 2018.

Consistent Generative Query Networks

Authors: Ananya Kumar, S. M. Ali Eslami, Danilo J. Rezende, Marta Garnelo, Fabio Viola, Edward Lockhart, Murray Shanahan

Abstract: Stochastic video prediction models take in a sequence of image frames, and generate a sequence of consecutive future image frames. These models typically generate future frames in an autoregressive fashion, which is slow and requires the input and output frames to be consecutive. We introduce a model that overcomes these drawbacks by generating a latent representation from an arbitrary set of fram… ▽ More Stochastic video prediction models take in a sequence of image frames, and generate a sequence of consecutive future image frames. These models typically generate future frames in an autoregressive fashion, which is slow and requires the input and output frames to be consecutive. We introduce a model that overcomes these drawbacks by generating a latent representation from an arbitrary set of frames that can then be used to simultaneously and efficiently sample temporally consistent frames at arbitrary time-points. For example, our model can "jump" and directly sample frames at the end of the video, without sampling intermediate frames. Synthetic video evaluations confirm substantial gains in speed and functionality without loss in fidelity. We also apply our framework to a 3D scene reconstruction dataset. Here, our model is conditioned on camera location and can sample consistent sets of images for what an occluded region of a 3D scene might look like, even if there are multiple possibilities for what that region might contain. Reconstructions and videos are available at https://bit.ly/2O4Pc4R. △ Less

Submitted 21 April, 2019; v1 submitted 5 July, 2018; originally announced July 2018.

Neural Processes

Authors: Marta Garnelo, Jonathan Schwarz, Dan Rosenbaum, Fabio Viola, Danilo J. Rezende, S. M. Ali Eslami, Yee Whye Teh

Abstract: A neural network (NN) is a parameterised function that can be tuned via gradient descent to approximate a labelled collection of data with high precision. A Gaussian process (GP), on the other hand, is a probabilistic model that defines a distribution over possible functions, and is updated in light of data via the rules of probabilistic inference. GPs are probabilistic, data-efficient and flexibl… ▽ More A neural network (NN) is a parameterised function that can be tuned via gradient descent to approximate a labelled collection of data with high precision. A Gaussian process (GP), on the other hand, is a probabilistic model that defines a distribution over possible functions, and is updated in light of data via the rules of probabilistic inference. GPs are probabilistic, data-efficient and flexible, however they are also computationally intensive and thus limited in their applicability. We introduce a class of neural latent variable models which we call Neural Processes (NPs), combining the best of both worlds. Like GPs, NPs define distributions over functions, are capable of rapid adaptation to new observations, and can estimate the uncertainty in their predictions. Like NNs, NPs are computationally efficient during training and evaluation but also learn to adapt their priors to data. We demonstrate the performance of NPs on a range of learning tasks, including regression and optimisation, and compare and contrast with related models in the literature. △ Less

Submitted 4 July, 2018; originally announced July 2018.

Conditional Neural Processes

Authors: Marta Garnelo, Dan Rosenbaum, Chris J. Maddison, Tiago Ramalho, David Saxton, Murray Shanahan, Yee Whye Teh, Danilo J. Rezende, S. M. Ali Eslami

Abstract: Deep neural networks excel at function approximation, yet they are typically trained from scratch for each new function. On the other hand, Bayesian methods, such as Gaussian Processes (GPs), exploit prior knowledge to quickly infer the shape of a new function at test time. Yet GPs are computationally expensive, and it can be hard to design appropriate priors. In this paper we propose a family of… ▽ More Deep neural networks excel at function approximation, yet they are typically trained from scratch for each new function. On the other hand, Bayesian methods, such as Gaussian Processes (GPs), exploit prior knowledge to quickly infer the shape of a new function at test time. Yet GPs are computationally expensive, and it can be hard to design appropriate priors. In this paper we propose a family of neural models, Conditional Neural Processes (CNPs), that combine the benefits of both. CNPs are inspired by the flexibility of stochastic processes such as GPs, but are structured as neural networks and trained via gradient descent. CNPs make accurate predictions after observing only a handful of training data points, yet scale to complex functions and large datasets. We demonstrate the performance and versatility of the approach on a range of canonical machine learning tasks, including regression, classification and image completion. △ Less

Submitted 4 July, 2018; originally announced July 2018.

A Probabilistic U-Net for Segmentation of Ambiguous Images

Authors: Simon A. A. Kohl, Bernardino Romera-Paredes, Clemens Meyer, Jeffrey De Fauw, Joseph R. Ledsam, Klaus H. Maier-Hein, S. M. Ali Eslami, Danilo Jimenez Rezende, Olaf Ronneberger

Abstract: Many real-world vision problems suffer from inherent ambiguities. In clinical applications for example, it might not be clear from a CT scan alone which particular region is cancer tissue. Therefore a group of graders typically produces a set of diverse but plausible segmentations. We consider the task of learning a distribution over segmentations given an input. To this end we propose a generativ… ▽ More Many real-world vision problems suffer from inherent ambiguities. In clinical applications for example, it might not be clear from a CT scan alone which particular region is cancer tissue. Therefore a group of graders typically produces a set of diverse but plausible segmentations. We consider the task of learning a distribution over segmentations given an input. To this end we propose a generative segmentation model based on a combination of a U-Net with a conditional variational autoencoder that is capable of efficiently producing an unlimited number of plausible hypotheses. We show on a lung abnormalities segmentation task and on a Cityscapes segmentation task that our model reproduces the possible segmentation variants as well as the frequencies with which they occur, doing so significantly better than published approaches. These models could have a high impact in real-world applications, such as being used as clinical decision-making algorithms accounting for multiple plausible semantic segmentation hypotheses to provide possible diagnoses and recommend further actions to resolve the present ambiguities. △ Less

Submitted 29 January, 2019; v1 submitted 13 June, 2018; originally announced June 2018.

Comments: Last update: added further details about the LIDC experiment. 11 pages for the main paper, 28 pages including appendix. 5 figures in the main paper, 18 figures in total, Advances in Neural Information Processing Systems (NeurIPS), 2018

Generative Temporal Models with Spatial Memory for Partially Observed Environments

Authors: Marco Fraccaro, Danilo Jimenez Rezende, Yori Zwols, Alexander Pritzel, S. M. Ali Eslami, Fabio Viola

Abstract: In model-based reinforcement learning, generative and temporal models of environments can be leveraged to boost agent performance, either by tuning the agent's representations during training or via use as part of an explicit planning mechanism. However, their application in practice has been limited to simplistic environments, due to the difficulty of training such models in larger, potentially p… ▽ More In model-based reinforcement learning, generative and temporal models of environments can be leveraged to boost agent performance, either by tuning the agent's representations during training or via use as part of an explicit planning mechanism. However, their application in practice has been limited to simplistic environments, due to the difficulty of training such models in larger, potentially partially-observed and 3D environments. In this work we introduce a novel action-conditioned generative model of such challenging environments. The model features a non-parametric spatial memory system in which we store learned, disentangled representations of the environment. Low-dimensional spatial updates are computed using a state-space model that makes use of knowledge on the prior dynamics of the moving agent, and high-dimensional visual observations are modelled with a Variational Auto-Encoder. The result is a scalable architecture capable of performing coherent predictions over hundreds of time steps across a range of partially observed 2D and 3D environments. △ Less

Submitted 19 July, 2018; v1 submitted 25 April, 2018; originally announced April 2018.

Comments: ICML 2018

Synthesizing Programs for Images using Reinforced Adversarial Learning

Authors: Yaroslav Ganin, Tejas Kulkarni, Igor Babuschkin, S. M. Ali Eslami, Oriol Vinyals

Abstract: Advances in deep generative networks have led to impressive results in recent years. Nevertheless, such models can often waste their capacity on the minutiae of datasets, presumably due to weak inductive biases in their decoders. This is where graphics engines may come in handy since they abstract away low-level details and represent images as high-level programs. Current methods that combine deep… ▽ More Advances in deep generative networks have led to impressive results in recent years. Nevertheless, such models can often waste their capacity on the minutiae of datasets, presumably due to weak inductive biases in their decoders. This is where graphics engines may come in handy since they abstract away low-level details and represent images as high-level programs. Current methods that combine deep learning and renderers are limited by hand-crafted likelihood or distance functions, a need for large amounts of supervision, or difficulties in scaling their inference algorithms to richer datasets. To mitigate these issues, we present SPIRAL, an adversarially trained agent that generates a program which is executed by a graphics engine to interpret and sample images. The goal of this agent is to fool a discriminator network that distinguishes between real and rendered data, trained with a distributed reinforcement learning setup without any supervision. A surprising finding is that using the discriminator's output as a reward signal is the key to allow the agent to make meaningful progress at matching the desired output rendering. To the best of our knowledge, this is the first demonstration of an end-to-end, unsupervised and adversarial inverse graphics agent on challenging real world (MNIST, Omniglot, CelebA) and synthetic 3D datasets. △ Less

Submitted 3 April, 2018; originally announced April 2018.

Comments: 12 pages, 13 figures

Unsupervised Learning of 3D Structure from Images

Authors: Danilo Jimenez Rezende, S. M. Ali Eslami, Shakir Mohamed, Peter Battaglia, Max Jaderberg, Nicolas Heess

Abstract: A key goal of computer vision is to recover the underlying 3D structure from 2D observations of the world. In this paper we learn strong deep generative models of 3D structures, and recover these structures from 3D and 2D images via probabilistic inference. We demonstrate high-quality samples and report log-likelihoods on several datasets, including ShapeNet [2], and establish the first benchmarks… ▽ More A key goal of computer vision is to recover the underlying 3D structure from 2D observations of the world. In this paper we learn strong deep generative models of 3D structures, and recover these structures from 3D and 2D images via probabilistic inference. We demonstrate high-quality samples and report log-likelihoods on several datasets, including ShapeNet [2], and establish the first benchmarks in the literature. We also show how these models and their inference networks can be trained end-to-end from 2D images. This demonstrates for the first time the feasibility of learning to infer 3D representations of the world in a purely unsupervised manner. △ Less

Submitted 19 June, 2018; v1 submitted 3 July, 2016; originally announced July 2016.

Comments: Appears in Advances in Neural Information Processing Systems 29 (NIPS 2016)

Kernel-Based Just-In-Time Learning for Passing Expectation Propagation Messages

Authors: Wittawat Jitkrittum, Arthur Gretton, Nicolas Heess, S. M. Ali Eslami, Balaji Lakshminarayanan, Dino Sejdinovic, Zoltán Szabó

Abstract: We propose an efficient nonparametric strategy for learning a message operator in expectation propagation (EP), which takes as input the set of incoming messages to a factor node, and produces an outgoing message as output. This learned operator replaces the multivariate integral required in classical EP, which may not have an analytic expression. We use kernel-based regression, which is trained o… ▽ More We propose an efficient nonparametric strategy for learning a message operator in expectation propagation (EP), which takes as input the set of incoming messages to a factor node, and produces an outgoing message as output. This learned operator replaces the multivariate integral required in classical EP, which may not have an analytic expression. We use kernel-based regression, which is trained on a set of probability distributions representing the incoming messages, and the associated outgoing messages. The kernel approach has two main advantages: first, it is fast, as it is implemented using a novel two-layer random feature representation of the input message distributions; second, it has principled uncertainty estimates, and can be cheaply updated online, meaning it can request and incorporate new training data when it encounters inputs on which it is uncertain. In experiments, our approach is able to solve learning problems where a single message operator is required for multiple, substantially different data sets (logistic regression for a variety of classification problems), where it is essential to accurately assess uncertainty and to efficiently and robustly update the message operator. △ Less

Submitted 9 June, 2015; v1 submitted 9 March, 2015; originally announced March 2015.

Comments: accepted to UAI 2015. Correct typos. Add more content to the appendix. Main results unchanged

MSC Class: 62F15; 46e22; 62-09; 62F30 ACM Class: G.3; I.2.6