-
Ablation Based Counterfactuals
Authors:
Zheng Dai,
David K Gifford
Abstract:
Diffusion models are a class of generative models that generate high-quality samples, but at present it is difficult to characterize how they depend upon their training data. This difficulty raises scientific and regulatory questions, and is a consequence of the complexity of diffusion models and their sampling process. To analyze this dependence, we introduce Ablation Based Counterfactuals (ABC),…
▽ More
Diffusion models are a class of generative models that generate high-quality samples, but at present it is difficult to characterize how they depend upon their training data. This difficulty raises scientific and regulatory questions, and is a consequence of the complexity of diffusion models and their sampling process. To analyze this dependence, we introduce Ablation Based Counterfactuals (ABC), a method of performing counterfactual analysis that relies on model ablation rather than model retraining. In our approach, we train independent components of a model on different but overlapping splits of a training set. These components are then combined into a single model, from which the causal influence of any training sample can be removed by ablating a combination of model components. We demonstrate how we can construct a model like this using an ensemble of diffusion models. We then use this model to study the limits of training data attribution by enumerating full counterfactual landscapes, and show that single source attributability diminishes with increasing training data size. Finally, we demonstrate the existence of unattributable samples.
△ Less
Submitted 12 June, 2024;
originally announced June 2024.
-
Training Data Attribution for Diffusion Models
Authors:
Zheng Dai,
David K Gifford
Abstract:
Diffusion models have become increasingly popular for synthesizing high-quality samples based on training datasets. However, given the oftentimes enormous sizes of the training datasets, it is difficult to assess how training data impact the samples produced by a trained diffusion model. The difficulty of relating diffusion model inputs and outputs poses significant challenges to model explainabil…
▽ More
Diffusion models have become increasingly popular for synthesizing high-quality samples based on training datasets. However, given the oftentimes enormous sizes of the training datasets, it is difficult to assess how training data impact the samples produced by a trained diffusion model. The difficulty of relating diffusion model inputs and outputs poses significant challenges to model explainability and training data attribution. Here we propose a novel solution that reveals how training data influence the output of diffusion models through the use of ensembles. In our approach individual models in an encoded ensemble are trained on carefully engineered splits of the overall training data to permit the identification of influential training examples. The resulting model ensembles enable efficient ablation of training data influence, allowing us to assess the impact of training data on model outputs. We demonstrate the viability of these ensembles as generative models and the validity of our approach to assessing influence.
△ Less
Submitted 3 June, 2023;
originally announced June 2023.
-
Constrained Submodular Optimization for Vaccine Design
Authors:
Zheng Dai,
David Gifford
Abstract:
Advances in machine learning have enabled the prediction of immune system responses to prophylactic and therapeutic vaccines. However, the engineering task of designing vaccines remains a challenge. In particular, the genetic variability of the human immune system makes it difficult to design peptide vaccines that provide widespread immunity in vaccinated populations. We introduce a framework for…
▽ More
Advances in machine learning have enabled the prediction of immune system responses to prophylactic and therapeutic vaccines. However, the engineering task of designing vaccines remains a challenge. In particular, the genetic variability of the human immune system makes it difficult to design peptide vaccines that provide widespread immunity in vaccinated populations. We introduce a framework for evaluating and designing peptide vaccines that uses probabilistic machine learning models, and demonstrate its ability to produce designs for a SARS-CoV-2 vaccine that outperform previous designs. We provide a theoretical analysis of the approximability, scalability, and complexity of our framework.
△ Less
Submitted 26 January, 2023; v1 submitted 16 June, 2022;
originally announced June 2022.
-
Image classifiers can not be made robust to small perturbations
Authors:
Zheng Dai,
David K. Gifford
Abstract:
The sensitivity of image classifiers to small perturbations in the input is often viewed as a defect of their construction. We demonstrate that this sensitivity is a fundamental property of classifiers. For any arbitrary classifier over the set of $n$-by-$n$ images, we show that for all but one class it is possible to change the classification of all but a tiny fraction of the images in that class…
▽ More
The sensitivity of image classifiers to small perturbations in the input is often viewed as a defect of their construction. We demonstrate that this sensitivity is a fundamental property of classifiers. For any arbitrary classifier over the set of $n$-by-$n$ images, we show that for all but one class it is possible to change the classification of all but a tiny fraction of the images in that class with a perturbation of size $O(n^{1/\max{(p,1)}})$ when measured in any $p$-norm for $p \geq 0$. We then discuss how this phenomenon relates to human visual perception and the potential implications for the design considerations of computer vision systems.
△ Less
Submitted 9 August, 2022; v1 submitted 7 December, 2021;
originally announced December 2021.
-
Lost in Pruning: The Effects of Pruning Neural Networks beyond Test Accuracy
Authors:
Lucas Liebenwein,
Cenk Baykal,
Brandon Carter,
David Gifford,
Daniela Rus
Abstract:
Neural network pruning is a popular technique used to reduce the inference costs of modern, potentially overparameterized, networks. Starting from a pre-trained network, the process is as follows: remove redundant parameters, retrain, and repeat while maintaining the same test accuracy. The result is a model that is a fraction of the size of the original with comparable predictive performance (tes…
▽ More
Neural network pruning is a popular technique used to reduce the inference costs of modern, potentially overparameterized, networks. Starting from a pre-trained network, the process is as follows: remove redundant parameters, retrain, and repeat while maintaining the same test accuracy. The result is a model that is a fraction of the size of the original with comparable predictive performance (test accuracy). Here, we reassess and evaluate whether the use of test accuracy alone in the terminating condition is sufficient to ensure that the resulting model performs well across a wide spectrum of "harder" metrics such as generalization to out-of-distribution data and resilience to noise. Across evaluations on varying architectures and data sets, we find that pruned networks effectively approximate the unpruned model, however, the prune ratio at which pruned networks achieve commensurate performance varies significantly across tasks. These results call into question the extent of \emph{genuine} overparameterization in deep learning and raise concerns about the practicability of deploying pruned networks, specifically in the context of safety-critical systems, unless they are widely evaluated beyond test accuracy to reliably predict their performance. Our code is available at https://github.com/lucaslie/torchprune.
△ Less
Submitted 4 March, 2021;
originally announced March 2021.
-
Maximum n-times Coverage for Vaccine Design
Authors:
Ge Liu,
Alexander Dimitrakakis,
Brandon Carter,
David Gifford
Abstract:
We introduce the maximum $n$-times coverage problem that selects $k$ overlays to maximize the summed coverage of weighted elements, where each element must be covered at least $n$ times. We also define the min-cost $n$-times coverage problem where the objective is to select the minimum set of overlays such that the sum of the weights of elements that are covered at least $n$ times is at least $τ$.…
▽ More
We introduce the maximum $n$-times coverage problem that selects $k$ overlays to maximize the summed coverage of weighted elements, where each element must be covered at least $n$ times. We also define the min-cost $n$-times coverage problem where the objective is to select the minimum set of overlays such that the sum of the weights of elements that are covered at least $n$ times is at least $τ$. Maximum $n$-times coverage is a generalization of the multi-set multi-cover problem, is NP-complete, and is not submodular. We introduce two new practical solutions for $n$-times coverage based on integer linear programming and sequential greedy optimization. We show that maximum $n$-times coverage is a natural way to frame peptide vaccine design, and find that it produces a pan-strain COVID-19 vaccine design that is superior to 29 other published designs in predicted population coverage and the expected number of peptides displayed by each individual's HLA molecules.
△ Less
Submitted 4 May, 2022; v1 submitted 24 January, 2021;
originally announced January 2021.
-
Epistemic stances toward group work in learning physics: Interactions between epistemology and social dynamics in a collaborative problem solving context
Authors:
Jessica R. Hoehn,
Julian D. Gifford,
Noah D. Finkelstein
Abstract:
As educators we often ask our physics students to work in groups---on tutorials, during in-class discussions, and on homework, projects, or exams. Researchers have documented the benefits of group work for students' conceptual mastery and problem solving skills, and have worked to optimize the productivity of group work by assigning roles and composing groups based on performance levels or gender.…
▽ More
As educators we often ask our physics students to work in groups---on tutorials, during in-class discussions, and on homework, projects, or exams. Researchers have documented the benefits of group work for students' conceptual mastery and problem solving skills, and have worked to optimize the productivity of group work by assigning roles and composing groups based on performance levels or gender. However, it is less common for us as a physics education research community to attend to the social dynamics and interactions among students within a collaborative setting, or to address students' views about group work. In this paper, we define \textit{epistemic stances toward group work}: stances towards what it means to generate and apply knowledge in a group. Through a case study analysis of a collaborative problem solving session among four physics students, we investigate how epistemic stances toward group work interact with social dynamics. We find that misalignment of stances between students can inform, and be informed by, the social positioning of group members. Understanding these fine-grained interactions is one way to begin to understand how to support students in engaging in productive and equitable group work.
△ Less
Submitted 5 May, 2020;
originally announced May 2020.
-
Overinterpretation reveals image classification model pathologies
Authors:
Brandon Carter,
Siddhartha Jain,
Jonas Mueller,
David Gifford
Abstract:
Image classifiers are typically scored on their test set accuracy, but high accuracy can mask a subtle type of model failure. We find that high scoring convolutional neural networks (CNNs) on popular benchmarks exhibit troubling pathologies that allow them to display high accuracy even in the absence of semantically salient features. When a model provides a high-confidence decision without salient…
▽ More
Image classifiers are typically scored on their test set accuracy, but high accuracy can mask a subtle type of model failure. We find that high scoring convolutional neural networks (CNNs) on popular benchmarks exhibit troubling pathologies that allow them to display high accuracy even in the absence of semantically salient features. When a model provides a high-confidence decision without salient supporting input features, we say the classifier has overinterpreted its input, finding too much class-evidence in patterns that appear nonsensical to humans. Here, we demonstrate that neural networks trained on CIFAR-10 and ImageNet suffer from overinterpretation, and we find models on CIFAR-10 make confident predictions even when 95% of input images are masked and humans cannot discern salient features in the remaining pixel-subsets. We introduce Batched Gradient SIS, a new method for discovering sufficient input subsets for complex datasets, and use this method to show the sufficiency of border pixels in ImageNet for training and testing. Although these patterns portend potential model fragility in real-world deployment, they are in fact valid statistical patterns of the benchmark that alone suffice to attain high test accuracy. Unlike adversarial examples, overinterpretation relies upon unmodified image pixels. We find ensembling and input dropout can each help mitigate overinterpretation.
△ Less
Submitted 7 December, 2021; v1 submitted 19 March, 2020;
originally announced March 2020.
-
Information Condensing Active Learning
Authors:
Siddhartha Jain,
Ge Liu,
David Gifford
Abstract:
We introduce Information Condensing Active Learning (ICAL), a batch mode model agnostic Active Learning (AL) method targeted at Deep Bayesian Active Learning that focuses on acquiring labels for points which have as much information as possible about the still unacquired points. ICAL uses the Hilbert Schmidt Independence Criterion (HSIC) to measure the strength of the dependency between a candidat…
▽ More
We introduce Information Condensing Active Learning (ICAL), a batch mode model agnostic Active Learning (AL) method targeted at Deep Bayesian Active Learning that focuses on acquiring labels for points which have as much information as possible about the still unacquired points. ICAL uses the Hilbert Schmidt Independence Criterion (HSIC) to measure the strength of the dependency between a candidate batch of points and the unlabeled set. We develop key optimizations that allow us to scale our method to large unlabeled sets. We show significant improvements in terms of model accuracy and negative log likelihood (NLL) on several image datasets compared to state of the art batch mode AL methods for deep learning.
△ Less
Submitted 19 February, 2020; v1 submitted 18 February, 2020;
originally announced February 2020.
-
Maximizing Overall Diversity for Improved Uncertainty Estimates in Deep Ensembles
Authors:
Siddhartha Jain,
Ge Liu,
Jonas Mueller,
David Gifford
Abstract:
The inaccuracy of neural network models on inputs that do not stem from the training data distribution is both problematic and at times unrecognized. Model uncertainty estimation can address this issue, where uncertainty estimates are often based on the variation in predictions produced by a diverse ensemble of models applied to the same input. Here we describe Maximize Overall Diversity (MOD), a…
▽ More
The inaccuracy of neural network models on inputs that do not stem from the training data distribution is both problematic and at times unrecognized. Model uncertainty estimation can address this issue, where uncertainty estimates are often based on the variation in predictions produced by a diverse ensemble of models applied to the same input. Here we describe Maximize Overall Diversity (MOD), a straightforward approach to improve ensemble-based uncertainty estimates by encouraging larger overall diversity in ensemble predictions across all possible inputs that might be encountered in the future. When applied to various neural network ensembles, MOD significantly improves predictive performance for out-of-distribution test examples without sacrificing in-distribution performance on 38 Protein-DNA binding regression datasets, 9 UCI datasets, and the IMDB-Wiki image dataset. Across many Bayesian optimization tasks, the performance of UCB acquisition is also greatly improved by leveraging MOD uncertainty estimates.
△ Less
Submitted 12 February, 2020; v1 submitted 18 June, 2019;
originally announced June 2019.
-
What made you do this? Understanding black-box decisions with sufficient input subsets
Authors:
Brandon Carter,
Jonas Mueller,
Siddhartha Jain,
David Gifford
Abstract:
Local explanation frameworks aim to rationalize particular decisions made by a black-box prediction model. Existing techniques are often restricted to a specific type of predictor or based on input saliency, which may be undesirably sensitive to factors unrelated to the model's decision making process. We instead propose sufficient input subsets that identify minimal subsets of features whose obse…
▽ More
Local explanation frameworks aim to rationalize particular decisions made by a black-box prediction model. Existing techniques are often restricted to a specific type of predictor or based on input saliency, which may be undesirably sensitive to factors unrelated to the model's decision making process. We instead propose sufficient input subsets that identify minimal subsets of features whose observed values alone suffice for the same decision to be reached, even if all other input feature values are missing. General principles that globally govern a model's decision-making can also be revealed by searching for clusters of such input patterns across many data points. Our approach is conceptually straightforward, entirely model-agnostic, simply implemented using instance-wise backward selection, and able to produce more concise rationales than existing techniques. We demonstrate the utility of our interpretation method on various neural network models trained on text, image, and genomic data.
△ Less
Submitted 8 February, 2019; v1 submitted 9 October, 2018;
originally announced October 2018.
-
Galaxy Cluster Mass Reconstruction Project - IV. Understanding the effects of imperfect membership on cluster mass estimation
Authors:
R. Wojtak,
L. Old,
G. A. Mamon,
F. R. Pearce,
R. de Carvalho,
C. Sifón,
M. E. Gray,
R. A. Skibba,
D. Croton,
S. Bamford,
D. Gifford,
A. von der Linden,
J. C. Muñoz-Cuartas,
V. Müller,
R. J. Pearson,
E. Rozo,
E. Rykoff,
A. Saro,
T. Sepp,
E. Tempel
Abstract:
The primary difficulty in measuring dynamical masses of galaxy clusters from galaxy data lies in the separation between true cluster members from interloping galaxies along the line of sight. We study the impact of membership contamination and incompleteness on cluster mass estimates obtained with 25 commonly used techniques applied to nearly 1000 mock clusters. We show that all methods overestima…
▽ More
The primary difficulty in measuring dynamical masses of galaxy clusters from galaxy data lies in the separation between true cluster members from interloping galaxies along the line of sight. We study the impact of membership contamination and incompleteness on cluster mass estimates obtained with 25 commonly used techniques applied to nearly 1000 mock clusters. We show that all methods overestimate or underestimate cluster masses when applied to contaminated or incomplete galaxy samples respectively. This appears to be the main source of the intrinsic scatter in the mass scaling relation. Applying corrections based on a prior knowledge of contamination and incompleteness can reduce the scatter to the level of shot noise expected for poorly sampled clusters. We establish an empirical model quantifying the effect of imperfect membership on cluster mass estimation and discuss its universal and method-dependent features. We find that both imperfect membership and the response of the mass estimators depend on cluster mass, effectively causing a flattening of the estimated - true mass relation. Imperfect membership thus alters cluster counts determined from spectroscopic surveys, hence the cosmological parameters that depend on such counts.
△ Less
Submitted 16 August, 2018; v1 submitted 8 June, 2018;
originally announced June 2018.
-
Inferring Gravitational Potentials from Mass Densities in Cluster-sized Halos
Authors:
Christopher J. Miller,
Alejo Stark,
Daniel Gifford,
Nicholas Kern
Abstract:
We use N-body simulations to quantify how the escape velocity in cluster-sized halos maps to the gravitational potential in a LambdaCDM universe. Using spherical density-potential pairs and the Poisson equation, we find that the matter density inferred gravitational potential profile predicts the escape velocity profile to within a few percent accuracy for group and cluster-sized halos (10^13 < M_…
▽ More
We use N-body simulations to quantify how the escape velocity in cluster-sized halos maps to the gravitational potential in a LambdaCDM universe. Using spherical density-potential pairs and the Poisson equation, we find that the matter density inferred gravitational potential profile predicts the escape velocity profile to within a few percent accuracy for group and cluster-sized halos (10^13 < M_200 < 10^15 M_sun, with respect to the critical density). The accuracy holds from just outside the core to beyond the virial radius. We show the importance of explicitly incorporating a cosmological constant when inferring the potential from the Poisson equation. We consider three density models and find that the Einasto and Gamma profiles provide a better joint estimate of the density and potential profiles than the Navarro, Frenk and White profile, which fails to accurately represent the escape velocity. For individual halos, the 1 sigma scatter between the measured escape velocity and the density-inferred potential profile is small (<5%). Finally, while the sub-halos show 15% biases in their representation of the particle velocity dispersion profile, the sub-halo escape velocity profile matches the dark matter escape velocity profile to high accuracy with no evidence for velocity bias outside 0.4r_200.
△ Less
Submitted 16 December, 2016;
originally announced December 2016.
-
Stacking Caustic Masses from Galaxy Clusters
Authors:
Daniel Gifford,
Nicholas Kern,
Christopher J. Miller
Abstract:
Ongoing and future spectroscopic surveys will measure numerous galaxy redshifts within tens of thousands of galaxy clusters. However, the sampling within these clusters will be low, 15 < N < 50 per cluster. With such data, it will be difficult to achieve accurate and precise mass estimates for individual clusters using phase-space mass estimation techniques. We develop and test a new stacking algo…
▽ More
Ongoing and future spectroscopic surveys will measure numerous galaxy redshifts within tens of thousands of galaxy clusters. However, the sampling within these clusters will be low, 15 < N < 50 per cluster. With such data, it will be difficult to achieve accurate and precise mass estimates for individual clusters using phase-space mass estimation techniques. We develop and test a new stacking algorithm based on the caustic technique, which reduces the mass scatter in <log M_caustic | M_200 > for ensemble clusters from 70% for individual clusters to less than 10% for ensemble clusters with only 15 galaxies per cluster and 100 clusters per ensemble. With > 1000 galaxies per ensemble phase-space, the escape-velocity edge becomes readily identifiable and the presence of interloping galaxies is minimized. We develop and test an algorithm to trace the projected phase-space surface directly, which results in minimally biased dynamical mass estimates. We then quantify how binning and sampling affect the phase-space-based mass estimates when using an observational proxy that incorporates realistic mass scatter, like richness, and find the added uncertainty in the binning procedure has minimal influence on the resulting bias and scatter of the stacked mass estimates.
△ Less
Submitted 16 December, 2016;
originally announced December 2016.
-
On Escaping a Galaxy Cluster in an Accelerating Universe
Authors:
Alejo Stark,
Christopher J. Miller,
Daniel Gifford
Abstract:
We derive the escape velocity profile for an Einasto density field in an accelerating universe and demonstrate its physical viability by comparing theoretical expectations to both light-cone data generated from N-body simulations and archival data on 20 galaxy clusters. We demonstrate that the projection function ($g(β)$) is deemed physically viable only for the theoretical expectation that includ…
▽ More
We derive the escape velocity profile for an Einasto density field in an accelerating universe and demonstrate its physical viability by comparing theoretical expectations to both light-cone data generated from N-body simulations and archival data on 20 galaxy clusters. We demonstrate that the projection function ($g(β)$) is deemed physically viable only for the theoretical expectation that includes a cosmology-dependent term. Using simulations, we show that the inferred velocity anisotropy is more than 6σ away from the expected value for the theoretical profile that ignores the acceleration of the universe. In the archival data, we constrain the average velocity anisotropy parameter of a sample of 20 clusters to be $β={0.248}_{-0.360}^{+0.164}$ at the 68% confidence level. Lastly, we briefly discuss how our analytic model may be used as a novel cosmological probe based on galaxy clusters.
△ Less
Submitted 17 November, 2016;
originally announced November 2016.
-
Probing Theories of Gravity with Phase Space-Inferred Potentials of Galaxy Clusters
Authors:
Alejo Stark,
Christopher J. Miller,
Nicholas Kern,
Daniel Gifford,
Gong-Bo Zhao,
Baojiu Li,
Kazuya Koyama,
Robert C. Nichol
Abstract:
Modified theories of gravity provide us with a unique opportunity to generate innovative tests of gravity. In Chameleon f(R) gravity, the gravitational potential differs from the weak-field limit of general relativity (GR) in a mass dependent way. We develop a probe of gravity which compares high mass clusters, where Chameleon effects are weak, to low mass clusters, where the effects can be strong…
▽ More
Modified theories of gravity provide us with a unique opportunity to generate innovative tests of gravity. In Chameleon f(R) gravity, the gravitational potential differs from the weak-field limit of general relativity (GR) in a mass dependent way. We develop a probe of gravity which compares high mass clusters, where Chameleon effects are weak, to low mass clusters, where the effects can be strong. We utilize the escape velocity edges in the radius/velocity phase space to infer the gravitational potential profiles on scales of 0.3-1 virial radii. We show that the escape edges of low mass clusters are enhanced compared to GR, where the magnitude of the difference depends on the background field value |fR0|. We validate our probe using N-body simulations and simulated light cone galaxy data. For a DESI (Dark Energy Spectroscopic Instrument) Bright Galaxy Sample, including observational systematics, projection effects, and cosmic variance, our test can differentiate between GR and Chameleon f(R) gravity models, |fR0| = 4e-6 (2e-6) at > 5σ (> 2σ), more than an order of magnitude better than current cluster-scale constraints.
△ Less
Submitted 29 February, 2016;
originally announced March 2016.
-
The XMM Cluster Survey: evolution of the velocity dispersion -- temperature relation over half a Hubble time
Authors:
Susan Wilson,
Matt Hilton,
Philip J. Rooney,
Caroline Caldwell,
Scott T. Kay,
Chris A. Collins,
Ian G. McCarthy,
A. Kathy Romer,
Alberto Bermeo-Hernandez,
Rebecca Bernstein,
Luiz da Costa,
Daniel Gifford,
Devon Hollowood,
Ben Hoyle,
Tesla Jeltema,
Andrew R. Liddle,
Marcio A. G Maia,
Robert G. Mann,
Julian A. Mayers,
Nicola Mehrtens,
Christopher J. Miller,
Robert C. Nichol,
Ricardo Ogando,
Martin Sahlén,
Benjamin Stahl
, et al. (4 additional authors not shown)
Abstract:
We measure the evolution of the velocity dispersion--temperature ($σ_{\rm v}$--$T_{\rm X}$) relation up to $z = 1$ using a sample of 38 galaxy clusters drawn from the \textit{XMM} Cluster Survey. This work improves upon previous studies by the use of a homogeneous cluster sample and in terms of the number of high redshift clusters included. We present here new redshift and velocity dispersion meas…
▽ More
We measure the evolution of the velocity dispersion--temperature ($σ_{\rm v}$--$T_{\rm X}$) relation up to $z = 1$ using a sample of 38 galaxy clusters drawn from the \textit{XMM} Cluster Survey. This work improves upon previous studies by the use of a homogeneous cluster sample and in terms of the number of high redshift clusters included. We present here new redshift and velocity dispersion measurements for 12 $z > 0.5$ clusters observed with the GMOS instruments on the Gemini telescopes. Using an orthogonal regression method, we find that the slope of the relation is steeper than that expected if clusters were self-similar, and that the evolution of the normalisation is slightly negative, but not significantly different from zero ($σ_{\rm v} \propto T^{0.86 \pm 0.14} E(z)^{-0.37 \pm 0.33}$). We verify our results by applying our methods to cosmological hydrodynamical simulations. The lack of evolution seen in our data is consistent with simulations that include both feedback and radiative cooling.
△ Less
Submitted 3 August, 2016; v1 submitted 9 December, 2015;
originally announced December 2015.
-
Modeling Persistent Trends in Distributions
Authors:
Jonas Mueller,
Tommi Jaakkola,
David Gifford
Abstract:
We present a nonparametric framework to model a short sequence of probability distributions that vary both due to underlying effects of sequential progression and confounding noise. To distinguish between these two types of variation and estimate the sequential-progression effects, our approach leverages an assumption that these effects follow a persistent trend. This work is motivated by the rece…
▽ More
We present a nonparametric framework to model a short sequence of probability distributions that vary both due to underlying effects of sequential progression and confounding noise. To distinguish between these two types of variation and estimate the sequential-progression effects, our approach leverages an assumption that these effects follow a persistent trend. This work is motivated by the recent rise of single-cell RNA-sequencing experiments over a brief time course, which aim to identify genes relevant to the progression of a particular biological process across diverse cell populations. While classical statistical tools focus on scalar-response regression or order-agnostic differences between distributions, it is desirable in this setting to consider both the full distributions as well as the structure imposed by their ordering. We introduce a new regression model for ordinal covariates where responses are univariate distributions and the underlying relationship reflects consistent changes in the distributions over increasing levels of the covariate. This concept is formalized as a "trend" in distributions, which we define as an evolution that is linear under the Wasserstein metric. Implemented via a fast alternating projections algorithm, our method exhibits numerous strengths in simulations and analyses of single-cell gene expression data.
△ Less
Submitted 24 May, 2017; v1 submitted 13 November, 2015;
originally announced November 2015.
-
A Multi-Wavelength Mass Analysis of RCS2 J232727.6-020437, a ~3x10$^{15}$M$_{\odot}$ Galaxy Cluster at z=0.7
Authors:
K. Sharon,
M. D. Gladders,
D. P. Marrone,
H. Hoekstra,
E. Rasia,
H. Bourdin,
D. Gifford,
A. K. Hicks,
C. Greer,
T. Mroczkowski,
L. F. Barrientos,
M. Bayliss,
J. E. Carlstrom,
D. G. Gilbank,
M. Gralla,
J. Hlavacek-Larrondo,
E. Leitch,
P. Mazzotta,
C. Miller,
S. J. C. Muchovej,
T. Schrabback,
H. K. C. Yee
Abstract:
We present an initial study of the mass and evolutionary state of a massive and distant cluster, RCS2 J232727.6-020437. This cluster, at z=0.6986, is the richest cluster discovered in the RCS2 project. The mass measurements presented in this paper are derived from all possible mass proxies: X-ray measurements, weak-lensing shear, strong lensing, Sunyaev Zel'dovich effect decrement, the velocity di…
▽ More
We present an initial study of the mass and evolutionary state of a massive and distant cluster, RCS2 J232727.6-020437. This cluster, at z=0.6986, is the richest cluster discovered in the RCS2 project. The mass measurements presented in this paper are derived from all possible mass proxies: X-ray measurements, weak-lensing shear, strong lensing, Sunyaev Zel'dovich effect decrement, the velocity distribution of cluster member galaxies, and galaxy richness. While each of these observables probe the mass of the cluster at a different radius, they all indicate that RCS2 J232727.6-020437 is among the most massive clusters at this redshift, with an estimated mass of M_200 ~3 x10^15 h^-1 Msun. In this paper, we demonstrate that the various observables are all reasonably consistent with each other to within their uncertainties. RCS2 J232727.6-020437 appears to be well relaxed -- with circular and concentric X-ray isophotes, with a cool core, and no indication of significant substructure in extensive galaxy velocity data.
△ Less
Submitted 3 November, 2015; v1 submitted 24 March, 2015;
originally announced March 2015.
-
Galaxy Cluster Mass Reconstruction Project: II. Quantifying scatter and bias using contrasting mock catalogues
Authors:
L. Old,
R. Wojtak,
G. A. Mamon,
R. A. Skibba,
F. R. Pearce,
D. Croton,
S. Bamford,
P. Behroozi,
R. de Carvalho,
J. C. Muñoz-Cuartas,
D. Gifford,
M. E. Gray,
A. von der Linden,
M. R. Merrifield,
S. I. Muldrew,
V. Müller,
R. J. Pearson,
T. J. Ponman,
E. Rozo,
E. Rykoff,
A. Saro,
T. Sepp,
C. Sifón,
E. Tempel
Abstract:
This article is the second in a series in which we perform an extensive comparison of various galaxy-based cluster mass estimation techniques that utilise the positions, velocities and colours of galaxies. Our aim is to quantify the scatter, systematic bias and completeness of cluster masses derived from a diverse set of 25 galaxy-based methods using two contrasting mock galaxy catalogues based on…
▽ More
This article is the second in a series in which we perform an extensive comparison of various galaxy-based cluster mass estimation techniques that utilise the positions, velocities and colours of galaxies. Our aim is to quantify the scatter, systematic bias and completeness of cluster masses derived from a diverse set of 25 galaxy-based methods using two contrasting mock galaxy catalogues based on a sophisticated halo occupation model and a semi-analytic model. Analysing 968 clusters, we find a wide range in the RMS errors in log M200c delivered by the different methods (0.18 to 1.08 dex, i.e., a factor of ~1.5 to 12), with abundance matching and richness methods providing the best results, irrespective of the input model assumptions. In addition, certain methods produce a significant number of catastrophic cases where the mass is under- or over-estimated by a factor greater than 10. Given the steeply falling high-mass end of the cluster mass function, we recommend that richness or abundance matching-based methods are used in conjunction with these methods as a sanity check for studies selecting high mass clusters. We see a stronger correlation of the recovered to input number of galaxies for both catalogues in comparison with the group/cluster mass, however, this does not guarantee that the correct member galaxies are being selected. We do not observe significantly higher scatter for either mock galaxy catalogues. Our results have implications for cosmological analyses that utilise the masses, richnesses, or abundances of clusters, which have different uncertainties when different methods are used.
△ Less
Submitted 25 February, 2015;
originally announced February 2015.
-
Galaxy Cluster Mass Reconstruction Project: I. Methods and first results on galaxy-based techniques
Authors:
L. Old,
R. A. Skibba,
F. R. Pearce,
D. Croton,
S. I. Muldrew,
J. C. Muñoz-Cuartas,
D. Gifford,
M. E. Gray,
A. von der Linden,
G. A. Mamon,
M. R. Merrifield,
V. Müller,
R. J. Pearson,
T. J. Ponman,
A. Saro,
T. Sepp,
C. Sifón,
E. Tempel,
E. Tundo,
Y. O. Wang,
R. Wojtak
Abstract:
This paper is the first in a series in which we perform an extensive comparison of various galaxy-based cluster mass estimation techniques that utilise the positions, velocities and colours of galaxies. Our primary aim is to test the performance of these cluster mass estimation techniques on a diverse set of models that will increase in complexity. We begin by providing participating methods with…
▽ More
This paper is the first in a series in which we perform an extensive comparison of various galaxy-based cluster mass estimation techniques that utilise the positions, velocities and colours of galaxies. Our primary aim is to test the performance of these cluster mass estimation techniques on a diverse set of models that will increase in complexity. We begin by providing participating methods with data from a simple model that delivers idealised clusters, enabling us to quantify the underlying scatter intrinsic to these mass estimation techniques. The mock catalogue is based on a Halo Occupation Distribution (HOD) model that assumes spherical Navarro, Frenk and White (NFW) haloes truncated at R_200, with no substructure nor colour segregation, and with isotropic, isothermal Maxwellian velocities. We find that, above 10^14 M_solar, recovered cluster masses are correlated with the true underlying cluster mass with an intrinsic scatter of typically a factor of two. Below 10^14 M_solar, the scatter rises as the number of member galaxies drops and rapidly approaches an order of magnitude. We find that richness-based methods deliver the lowest scatter, but it is not clear whether such accuracy may simply be the result of using an over-simplistic model to populate the galaxies in their haloes. Even when given the true cluster membership, large scatter is observed for the majority non-richness-based approaches, suggesting that mass reconstruction with a low number of dynamical tracers is inherently problematic.
△ Less
Submitted 18 March, 2014;
originally announced March 2014.
-
Velocity Anisotropy and Shape Bias in the Caustic Technique
Authors:
Daniel Gifford,
Christopher J. Miller
Abstract:
We use the Millennium Simulation to quantify the statistical accuracy and precision of the escape velocity technique for measuring cluster-sized halo masses at z~0.1. We show that in 3D, one can measure nearly unbiased (<4%) halo masses (>1.5x10^14 M_solar h^-1) with 10-15% scatter. Line-of-sight projection effects increase the scatter to ~25%, where we include the known velocity anisotropies. The…
▽ More
We use the Millennium Simulation to quantify the statistical accuracy and precision of the escape velocity technique for measuring cluster-sized halo masses at z~0.1. We show that in 3D, one can measure nearly unbiased (<4%) halo masses (>1.5x10^14 M_solar h^-1) with 10-15% scatter. Line-of-sight projection effects increase the scatter to ~25%, where we include the known velocity anisotropies. The classical "caustic" technique incorporates a calibration factor which is determined from N-body simulations. We derive and test a new implementation which eliminates the need for calibration and utilizes only the observables: the galaxy velocities with respect to the cluster mean v, the projected positions r_p, an estimate of the Navarro-Frenk-White (NFW) density concentration and an estimate of the velocity anisotropies, beta. We find that differences between the potential and density NFW concentrations induce a 10% bias in the caustic masses. We also find that large (100%) systematic errors in the observed ensemble average velocity anisotropies and concentrations translate to small (5%-10%) biases in the inferred masses.
△ Less
Submitted 28 June, 2013;
originally announced July 2013.
-
A Systematic Analysis of Caustic Methods for Galaxy Cluster Masses
Authors:
Daniel Gifford,
Christopher J. Miller,
Nicholas Kern
Abstract:
We quantify the expected observed statistical and systematic uncertainties of the escape velocity as a measure of the gravitational potential and total mass of galaxy clusters. We focus our attention on low redshift (z < 0.15) clusters, where large and deep spectroscopic datasets currently exist. Utilizing a suite of Millennium Simulation semi-analytic galaxy catalogs, we find that the dynamical m…
▽ More
We quantify the expected observed statistical and systematic uncertainties of the escape velocity as a measure of the gravitational potential and total mass of galaxy clusters. We focus our attention on low redshift (z < 0.15) clusters, where large and deep spectroscopic datasets currently exist. Utilizing a suite of Millennium Simulation semi-analytic galaxy catalogs, we find that the dynamical mass, as traced by either the virial relation or the escape velocity, is robust to variations in how dynamical friction is applied to "orphan" galaxies in the mock catalogs (i.e., those galaxies whose dark matter halos have fallen below the resolution limit). We find that the caustic technique recovers the known halo masses (M_200) with a third less scatter compared to the virial masses. The bias we measure increases quickly as the number of galaxies used decreases. For N_gal > 25, the scatter in the escape velocity mass is dominated by projections along the line-of-sight. Algorithmic uncertainties from the determination of the projected escape velocity profile are negligible. We quantify how target selection based on magnitude, color, and projected radial separation can induce small additional biases into the escape velocity masses. Using N_gal = 150 (25), the caustic technique has a per cluster scatter in ln(M|M_200) of 0.3 (0.5) and bias 1+/-3% (16+/-5%) for clusters with masses > 10^14M_solar at z<0.15.
△ Less
Submitted 28 June, 2013;
originally announced July 2013.
