Showing 1–10 of 10 results for author: Hill, D

Axiomatic Explainer Globalness via Optimal Transport

Authors: Davin Hill, Josh Bone, Aria Masoomi, Max Torop, Jennifer Dy

Abstract: Explainability methods are often challenging to evaluate and compare. With a multitude of explainers available, practitioners must often compare and select explainers based on quantitative evaluation metrics. One particular differentiator between explainers is the diversity of explanations for a given dataset; i.e. whether all explanations are identical, unique and uniformly distributed, or somewh… ▽ More Explainability methods are often challenging to evaluate and compare. With a multitude of explainers available, practitioners must often compare and select explainers based on quantitative evaluation metrics. One particular differentiator between explainers is the diversity of explanations for a given dataset; i.e. whether all explanations are identical, unique and uniformly distributed, or somewhere between these two extremes. In this work, we define a complexity measure for explainers, globalness, which enables deeper understanding of the distribution of explanations produced by feature attribution and feature selection methods for a given dataset. We establish the axiomatic properties that any such measure should possess and prove that our proposed measure, Wasserstein Globalness, meets these criteria. We validate the utility of Wasserstein Globalness using image, tabular, and synthetic datasets, empirically showing that it both facilitates meaningful comparison between explainers and improves the selection process for explainability methods. △ Less

Submitted 11 March, 2025; v1 submitted 2 November, 2024; originally announced November 2024.

Comments: Proceedings of the 28th International Conference on Artificial Intelligence and Statistics (AISTATS) 2025

Bayesian regularization of empirical MDPs

Authors: Samarth Gupta, Daniel N. Hill, Lexing Ying, Inderjit Dhillon

Abstract: In most applications of model-based Markov decision processes, the parameters for the unknown underlying model are often estimated from the empirical data. Due to noise, the policy learnedfrom the estimated model is often far from the optimal policy of the underlying model. When applied to the environment of the underlying model, the learned policy results in suboptimal performance, thus calling f… ▽ More In most applications of model-based Markov decision processes, the parameters for the unknown underlying model are often estimated from the empirical data. Due to noise, the policy learnedfrom the estimated model is often far from the optimal policy of the underlying model. When applied to the environment of the underlying model, the learned policy results in suboptimal performance, thus calling for solutions with better generalization performance. In this work we take a Bayesian perspective and regularize the objective function of the Markov decision process with prior information in order to obtain more robust policies. Two approaches are proposed, one based on $L^1$ regularization and the other on relative entropic regularization. We evaluate our proposed algorithms on synthetic simulations and on real-world search logs of a large scale online shopping store. Our results demonstrate the robustness of regularized MDP policies against the noise present in the models. △ Less

Submitted 20 September, 2022; v1 submitted 3 August, 2022; originally announced August 2022.

Counterfactual Learning To Rank for Utility-Maximizing Query Autocompletion

Authors: Adam Block, Rahul Kidambi, Daniel N. Hill, Thorsten Joachims, Inderjit S. Dhillon

Abstract: Conventional methods for query autocompletion aim to predict which completed query a user will select from a list. A shortcoming of this approach is that users often do not know which query will provide the best retrieval performance on the current information retrieval system, meaning that any query autocompletion methods trained to mimic user behavior can lead to suboptimal query suggestions. To… ▽ More Conventional methods for query autocompletion aim to predict which completed query a user will select from a list. A shortcoming of this approach is that users often do not know which query will provide the best retrieval performance on the current information retrieval system, meaning that any query autocompletion methods trained to mimic user behavior can lead to suboptimal query suggestions. To overcome this limitation, we propose a new approach that explicitly optimizes the query suggestions for downstream retrieval performance. We formulate this as a problem of ranking a set of rankings, where each query suggestion is represented by the downstream item ranking it produces. We then present a learning method that ranks query suggestions by the quality of their item rankings. The algorithm is based on a counterfactual learning approach that is able to leverage feedback on the items (e.g., clicks, purchases) to evaluate query suggestions through an unbiased estimator, thus avoiding the assumption that users write or select optimal queries. We establish theoretical support for the proposed approach and provide learning-theoretic guarantees. We also present empirical results on publicly available datasets, and demonstrate real-world applicability using data from an online shopping store. △ Less

Submitted 22 April, 2022; originally announced April 2022.

Top-$k$ eXtreme Contextual Bandits with Arm Hierarchy

Authors: Rajat Sen, Alexander Rakhlin, Lexing Ying, Rahul Kidambi, Dean Foster, Daniel Hill, Inderjit Dhillon

Abstract: Motivated by modern applications, such as online advertisement and recommender systems, we study the top-$k$ extreme contextual bandits problem, where the total number of arms can be enormous, and the learner is allowed to select $k$ arms and observe all or some of the rewards for the chosen arms. We first propose an algorithm for the non-extreme realizable setting, utilizing the Inverse Gap Weigh… ▽ More Motivated by modern applications, such as online advertisement and recommender systems, we study the top-$k$ extreme contextual bandits problem, where the total number of arms can be enormous, and the learner is allowed to select $k$ arms and observe all or some of the rewards for the chosen arms. We first propose an algorithm for the non-extreme realizable setting, utilizing the Inverse Gap Weighting strategy for selecting multiple arms. We show that our algorithm has a regret guarantee of $O(k\sqrt{(A-k+1)T \log (|\mathcal{F}|T)})$, where $A$ is the total number of arms and $\mathcal{F}$ is the class containing the regression function, while only requiring $\tilde{O}(A)$ computation per time step. In the extreme setting, where the total number of arms can be in the millions, we propose a practically-motivated arm hierarchy model that induces a certain structure in mean rewards to ensure statistical and computational efficiency. The hierarchical structure allows for an exponential reduction in the number of relevant arms for each context, thus resulting in a regret guarantee of $O(k\sqrt{(\log A-k+1)T \log (|\mathcal{F}|T)})$. Finally, we implement our algorithm using a hierarchical linear function class and show superior performance with respect to well-known benchmarks on simulated bandit feedback experiments using extreme multi-label classification datasets. On a dataset with three million arms, our reduction scheme has an average inference time of only 7.9 milliseconds, which is a 100x improvement. △ Less

Submitted 15 February, 2021; originally announced February 2021.

Measurement Error Correction in Particle Tracking Microrheology

Authors: Yun Ling, Martin Lysy, Ian Seim, Jay M. Newby, David B. Hill, Jeremy Cribb, M. Gregory Forest

Abstract: In diverse biological applications, particle tracking of passive microscopic species has become the experimental measurement of choice -- when either the materials are of limited volume, or so soft as to deform uncontrollably when manipulated by traditional instruments. In a wide range of particle tracking experiments, a ubiquitous finding is that the mean squared displacement (MSD) of particle po… ▽ More In diverse biological applications, particle tracking of passive microscopic species has become the experimental measurement of choice -- when either the materials are of limited volume, or so soft as to deform uncontrollably when manipulated by traditional instruments. In a wide range of particle tracking experiments, a ubiquitous finding is that the mean squared displacement (MSD) of particle positions exhibits a power-law signature, the parameters of which reveal valuable information about the viscous and elastic properties of various biomaterials. However, MSD measurements are typically contaminated by complex and interacting sources of instrumental noise. As these often affect the high-frequency bandwidth to which MSD estimates are particularly sensitive, inadequate error correction can lead to severe bias in power law estimation and thereby, the inferred viscoelastic properties. In this article, we propose a novel strategy to filter high-frequency noise from particle tracking measurements. Our filters are shown theoretically to cover a broad spectrum of high-frequency noises, and lead to a parametric estimator of MSD power-law coefficients for which an efficient computational implementation is presented. Based on numerous analyses of experimental and simulated data, results suggest our methods perform very well compared to other denoising procedures. △ Less

Submitted 14 November, 2019; originally announced November 2019.

Comments: 31 pages, 12 figures

MSC Class: 62M10; 62P10 (Primary) 76A10 (Secondary)

An Efficient Bandit Algorithm for Realtime Multivariate Optimization

Authors: Daniel N Hill, Houssam Nassif, Yi Liu, Anand Iyer, S V N Vishwanathan

Abstract: Optimization is commonly employed to determine the content of web pages, such as to maximize conversions on landing pages or click-through rates on search engine result pages. Often the layout of these pages can be decoupled into several separate decisions. For example, the composition of a landing page may involve deciding which image to show, which wording to use, what color background to displa… ▽ More Optimization is commonly employed to determine the content of web pages, such as to maximize conversions on landing pages or click-through rates on search engine result pages. Often the layout of these pages can be decoupled into several separate decisions. For example, the composition of a landing page may involve deciding which image to show, which wording to use, what color background to display, etc. Such optimization is a combinatorial problem over an exponentially large decision space. Randomized experiments do not scale well to this setting, and therefore, in practice, one is typically limited to optimizing a single aspect of a web page at a time. This represents a missed opportunity in both the speed of experimentation and the exploitation of possible interactions between layout decisions. Here we focus on multivariate optimization of interactive web pages. We formulate an approach where the possible interactions between different components of the page are modeled explicitly. We apply bandit methodology to explore the layout space efficiently and use hill-climbing to select optimal content in realtime. Our algorithm also extends to contextualization and personalization of layout selection. Simulation results show the suitability of our approach to large decision spaces with strong interactions between content. We further apply our algorithm to optimize a message that promotes adoption of an Amazon service. After only a single week of online optimization, we saw a 21% conversion increase compared to the median layout. Our technique is currently being deployed to optimize content across several locations at Amazon.com. △ Less

Submitted 22 October, 2018; originally announced October 2018.

Comments: KDD'17 Audience Appreciation Award

Journal ref: Daniel N. Hill, Houssam Nassif, Yi Liu, Anand Iyer, and S. V. N. Vishwanathan. 2017. An Efficient Bandit Algorithm for Realtime Multivariate Optimization. In Proceedings of KDD'17, Halifax, NS, Canada, pp. 1813-1821, 2017

Adaptive, Personalized Diversity for Visual Discovery

Authors: Choon Hui Teo, Houssam Nassif, Daniel Hill, Sriram Srinavasan, Mitchell Goodman, Vijai Mohan, SVN Vishwanathan

Abstract: Search queries are appropriate when users have explicit intent, but they perform poorly when the intent is difficult to express or if the user is simply looking to be inspired. Visual browsing systems allow e-commerce platforms to address these scenarios while offering the user an engaging shopping experience. Here we explore extensions in the direction of adaptive personalization and item diversi… ▽ More Search queries are appropriate when users have explicit intent, but they perform poorly when the intent is difficult to express or if the user is simply looking to be inspired. Visual browsing systems allow e-commerce platforms to address these scenarios while offering the user an engaging shopping experience. Here we explore extensions in the direction of adaptive personalization and item diversification within Stream, a new form of visual browsing and discovery by Amazon. Our system presents the user with a diverse set of interesting items while adapting to user interactions. Our solution consists of three components (1) a Bayesian regression model for scoring the relevance of items while leveraging uncertainty, (2) a submodular diversification framework that re-ranks the top scoring items based on category, and (3) personalized category preferences learned from the user's behavior. When tested on live traffic, our algorithms show a strong lift in click-through-rate and session duration. △ Less

Submitted 2 October, 2018; originally announced October 2018.

Comments: Best Paper Award

Journal ref: Adaptive, Personalized Diversity for Visual Discovery. Teo CH, Nassif H, Hill D, Srinavasan S, Goodman M, Mohan V, and Vishwanathan SVN. ACM Conference on Recommender Systems (RecSys'16), Boston, pp. 35-38, 2016

Maximum Likelihood Estimation for Single Particle, Passive Microrheology Data with Drift

Authors: John W. R. Mellnik, Martin Lysy, Paula A. Vasquez, Natesh S. Pillai, David B. Hill, Jeremy Crib, Scott A. McKinley, M. Gregory Forest

Abstract: Volume limitations and low yield thresholds of biological fluids have led to widespread use of passive microparticle rheology. The mean-squared-displacement (MSD) statistics of bead position time series (bead paths) are either applied directly to determine the creep compliance [Xu et al (1998)] or transformed to determine dynamic storage and loss moduli [Mason & Weitz (1995)]. A prevalent hurdle a… ▽ More Volume limitations and low yield thresholds of biological fluids have led to widespread use of passive microparticle rheology. The mean-squared-displacement (MSD) statistics of bead position time series (bead paths) are either applied directly to determine the creep compliance [Xu et al (1998)] or transformed to determine dynamic storage and loss moduli [Mason & Weitz (1995)]. A prevalent hurdle arises when there is a non-diffusive experimental drift in the data. Commensurate with the magnitude of drift relative to diffusive mobility, quantified by a Péclet number, the MSD statistics are distorted, and thus the path data must be "corrected" for drift. The standard approach is to estimate and subtract the drift from particle paths, and then calculate MSD statistics. We present an alternative, parametric approach using maximum likelihood estimation that simultaneously fits drift and diffusive model parameters from the path data; the MSD statistics (and consequently the compliance and dynamic moduli) then follow directly from the best-fit model. We illustrate and compare both methods on simulated path data over a range of Péclet numbers, where exact answers are known. We choose fractional Brownian motion as the numerical model because it affords tunable, sub-diffusive MSD statistics consistent with typical 30 second long, experimental observations of microbeads in several biological fluids. Finally, we apply and compare both methods on data from human bronchial epithelial cell culture mucus. △ Less

Submitted 21 February, 2016; v1 submitted 10 September, 2015; originally announced September 2015.

Comments: 29 pages, 12 figures

Model comparison and assessment for single particle tracking in biological fluids

Authors: Martin Lysy, Natesh S. Pillai, David B. Hill, M. Gregory Forest, John Mellnik, Paula Vasquez, Scott A. McKinley

Abstract: State-of-the-art techniques in passive particle-tracking microscopy provide high-resolution path trajectories of diverse foreign particles in biological fluids. For particles on the order of 1 micron diameter, these paths are generally inconsistent with simple Brownian motion. Yet, despite an abundance of data confirming these findings and their wide-ranging scientific implications, stochastic mod… ▽ More State-of-the-art techniques in passive particle-tracking microscopy provide high-resolution path trajectories of diverse foreign particles in biological fluids. For particles on the order of 1 micron diameter, these paths are generally inconsistent with simple Brownian motion. Yet, despite an abundance of data confirming these findings and their wide-ranging scientific implications, stochastic modeling of the complex particle motion has received comparatively little attention. Even among posited models, there is virtually no literature on likelihood-based inference, model comparisons, and other quantitative assessments. In this article, we develop a rigorous and computationally efficient Bayesian methodology to address this gap. We analyze two of the most prevalent candidate models for 30 second paths of 1 micron diameter tracer particles in human lung mucus: fractional Brownian motion (fBM) and a Generalized Langevin Equation (GLE) consistent with viscoelastic theory. Our model comparisons distinctly favor GLE over fBM, with the former describing the data remarkably well up to the timescales for which we have reliable information. △ Less

Submitted 29 November, 2015; v1 submitted 22 July, 2014; originally announced July 2014.

Comments: 24 pages, 10 figures + supplementary material

MSC Class: 62P10 (Primary)

Statistical Challenges in Microrheology

Authors: Gustavo Didier, Scott McKinley, David B. Hill, John Fricks

Abstract: Microrheology is the study of the properties of a complex fluid through the diffusion dynamics of small particles, typically latex beads, moving through that material. Currently, it is the dominant technique in the study of the physical properties of biological fluids, of the material properties of membranes or the cytoplasm of cells, or of the entire cell. The theoretical underpinning of microrhe… ▽ More Microrheology is the study of the properties of a complex fluid through the diffusion dynamics of small particles, typically latex beads, moving through that material. Currently, it is the dominant technique in the study of the physical properties of biological fluids, of the material properties of membranes or the cytoplasm of cells, or of the entire cell. The theoretical underpinning of microrheology was given in Mason and Weitz (Physical Review Letters; 1995), who introduced a framework for the use of path data of diffusing particles to infer viscoelastic properties of its fluid environment. The multi-particle tracking techniques that were subsequently developed have presented numerous challenges for experimentalists and theoreticians. This paper describes some specific challenges that await the attention of statisticians and applied probabilists. We describe relevant aspects of the physical theory, current inferential efforts and simulation aspects of a central model for the dynamics of nano-scale particles in viscoelastic fluids, the generalized Langevin equation. △ Less

Submitted 9 February, 2012; v1 submitted 28 January, 2012; originally announced January 2012.