-
Time-series attribution maps with regularized contrastive learning
Authors:
Steffen Schneider,
Rodrigo González Laiz,
Anastasiia Filippova,
Markus Frey,
Mackenzie Weygandt Mathis
Abstract:
Gradient-based attribution methods aim to explain decisions of deep learning models but so far lack identifiability guarantees. Here, we propose a method to generate attribution maps with identifiability guarantees by developing a regularized contrastive learning algorithm trained on time-series data plus a new attribution method called Inverted Neuron Gradient (collectively named xCEBRA). We show…
▽ More
Gradient-based attribution methods aim to explain decisions of deep learning models but so far lack identifiability guarantees. Here, we propose a method to generate attribution maps with identifiability guarantees by developing a regularized contrastive learning algorithm trained on time-series data plus a new attribution method called Inverted Neuron Gradient (collectively named xCEBRA). We show theoretically that xCEBRA has favorable properties for identifying the Jacobian matrix of the data generating process. Empirically, we demonstrate robust approximation of zero vs. non-zero entries in the ground-truth attribution map on synthetic datasets, and significant improvements across previous attribution methods based on feature ablation, Shapley values, and other gradient-based methods. Our work constitutes a first example of identifiable inference of time-series attribution maps and opens avenues to a better understanding of time-series data, such as for neural dynamics and decision-processes within neural networks.
△ Less
Submitted 17 February, 2025;
originally announced February 2025.
-
Self-supervised contrastive learning performs non-linear system identification
Authors:
Rodrigo González Laiz,
Tobias Schmidt,
Steffen Schneider
Abstract:
Self-supervised learning (SSL) approaches have brought tremendous success across many tasks and domains. It has been argued that these successes can be attributed to a link between SSL and identifiable representation learning: Temporal structure and auxiliary variables ensure that latent representations are related to the true underlying generative factors of the data. Here, we deepen this connect…
▽ More
Self-supervised learning (SSL) approaches have brought tremendous success across many tasks and domains. It has been argued that these successes can be attributed to a link between SSL and identifiable representation learning: Temporal structure and auxiliary variables ensure that latent representations are related to the true underlying generative factors of the data. Here, we deepen this connection and show that SSL can perform system identification in latent space. We propose dynamics contrastive learning, a framework to uncover linear, switching linear and non-linear dynamics under a non-linear observation model, give theoretical guarantees and validate them empirically.
△ Less
Submitted 1 June, 2025; v1 submitted 18 October, 2024;
originally announced October 2024.
-
Unsupervised Object Learning via Common Fate
Authors:
Matthias Tangemann,
Steffen Schneider,
Julius von Kügelgen,
Francesco Locatello,
Peter Gehler,
Thomas Brox,
Matthias Kümmerer,
Matthias Bethge,
Bernhard Schölkopf
Abstract:
Learning generative object models from unlabelled videos is a long standing problem and required for causal scene modeling. We decompose this problem into three easier subtasks, and provide candidate solutions for each of them. Inspired by the Common Fate Principle of Gestalt Psychology, we first extract (noisy) masks of moving objects via unsupervised motion segmentation. Second, generative model…
▽ More
Learning generative object models from unlabelled videos is a long standing problem and required for causal scene modeling. We decompose this problem into three easier subtasks, and provide candidate solutions for each of them. Inspired by the Common Fate Principle of Gestalt Psychology, we first extract (noisy) masks of moving objects via unsupervised motion segmentation. Second, generative models are trained on the masks of the background and the moving objects, respectively. Third, background and foreground models are combined in a conditional "dead leaves" scene model to sample novel scene configurations where occlusions and depth layering arise naturally. To evaluate the individual stages, we introduce the Fishbowl dataset positioned between complex real-world scenes and common object-centric benchmarks of simplistic objects. We show that our approach allows learning generative models that generalize beyond the occlusions present in the input videos, and represent scenes in a modular fashion that allows sampling plausible scenes outside the training distribution by permitting, for instance, object numbers or densities not observed in the training set.
△ Less
Submitted 15 May, 2023; v1 submitted 13 October, 2021;
originally announced October 2021.
-
Improving robustness against common corruptions by covariate shift adaptation
Authors:
Steffen Schneider,
Evgenia Rusak,
Luisa Eck,
Oliver Bringmann,
Wieland Brendel,
Matthias Bethge
Abstract:
Today's state-of-the-art machine vision models are vulnerable to image corruptions like blurring or compression artefacts, limiting their performance in many real-world applications. We here argue that popular benchmarks to measure model robustness against common corruptions (like ImageNet-C) underestimate model robustness in many (but not all) application scenarios. The key insight is that in man…
▽ More
Today's state-of-the-art machine vision models are vulnerable to image corruptions like blurring or compression artefacts, limiting their performance in many real-world applications. We here argue that popular benchmarks to measure model robustness against common corruptions (like ImageNet-C) underestimate model robustness in many (but not all) application scenarios. The key insight is that in many scenarios, multiple unlabeled examples of the corruptions are available and can be used for unsupervised online adaptation. Replacing the activation statistics estimated by batch normalization on the training set with the statistics of the corrupted images consistently improves the robustness across 25 different popular computer vision models. Using the corrected statistics, ResNet-50 reaches 62.2% mCE on ImageNet-C compared to 76.7% without adaptation. With the more robust DeepAugment+AugMix model, we improve the state of the art achieved by a ResNet50 model up to date from 53.6% mCE to 45.4% mCE. Even adapting to a single sample improves robustness for the ResNet-50 and AugMix models, and 32 samples are sufficient to improve the current state of the art for a ResNet-50 architecture. We argue that results with adapted statistics should be included whenever reporting scores in corruption benchmarks and other out-of-distribution generalization settings.
△ Less
Submitted 23 October, 2020; v1 submitted 30 June, 2020;
originally announced June 2020.
-
Explaining temporal trends in annualized relapse rates in placebo groups of randomized controlled trials in relapsing multiple sclerosis: systematic review and meta-regression
Authors:
Simon M. Steinvorth,
Christian Röver,
Simon Schneider,
Richard Nicholas,
Sebastian Straube,
Tim Friede
Abstract:
Background: Recent studies have shown a decrease in annualised relapse rates (ARRs) in placebo groups of randomised controlled trials (RCTs) in relapsing multiple sclerosis (RMS).
Methods: We conducted a systematic literature search of RCTs in RMS. Data on eligibility criteria and baseline characteristics were extracted and tested for significant trends over time. A meta-regression was conducted…
▽ More
Background: Recent studies have shown a decrease in annualised relapse rates (ARRs) in placebo groups of randomised controlled trials (RCTs) in relapsing multiple sclerosis (RMS).
Methods: We conducted a systematic literature search of RCTs in RMS. Data on eligibility criteria and baseline characteristics were extracted and tested for significant trends over time. A meta-regression was conducted to estimate their contribution to the decrease of trial ARRs over time.
Results: We identified 56 studies. Patient age at baseline (p < 0.001), mean duration of multiple sclerosis (MS) at baseline (p = 0.048), size of treatment groups (p = 0.003), Oxford Quality Scale scores (p = 0.021), and the number of eligibility criteria (p<0.001) increased significantly, whereas pre-trial ARR (p = 0.001), the time span over which pre-trial ARR was calculated (p < 0.001), and the duration of placebo-controlled follow-up (p = 0.006) decreased significantly over time. In meta-regression of trial placebo ARR, the temporal trend was found to be insignificant, with major factors explaining the variation: pre-trial ARR, the number of years used to calculate pre-trial ARR and study duration. Conclusion: The observed decline in trial ARRs may result from decreasing pre-trial ARRs and a shorter time period over which pre-trial ARRs were calculated. Increasing patient age and duration of illness may also contribute.
△ Less
Submitted 17 March, 2014; v1 submitted 12 March, 2013;
originally announced March 2013.
