-
Unlearning-based Neural Interpretations
Authors:
Ching Lam Choi,
Alexandre Duplessis,
Serge Belongie
Abstract:
Gradient-based interpretations often require an anchor point of comparison to avoid saturation in computing feature importance. We show that current baselines defined using static functions--constant mapping, averaging or blurring--inject harmful colour, texture or frequency assumptions that deviate from model behaviour. This leads to accumulation of irregular gradients, resulting in attribution m…
▽ More
Gradient-based interpretations often require an anchor point of comparison to avoid saturation in computing feature importance. We show that current baselines defined using static functions--constant mapping, averaging or blurring--inject harmful colour, texture or frequency assumptions that deviate from model behaviour. This leads to accumulation of irregular gradients, resulting in attribution maps that are biased, fragile and manipulable. Departing from the static approach, we propose UNI to compute an (un)learnable, debiased and adaptive baseline by perturbing the input towards an unlearning direction of steepest ascent. Our method discovers reliable baselines and succeeds in erasing salient features, which in turn locally smooths the high-curvature decision boundaries. Our analyses point to unlearning as a promising avenue for generating faithful, efficient and robust interpretations.
△ Less
Submitted 10 February, 2025; v1 submitted 10 October, 2024;
originally announced October 2024.
-
Accurate and Interactive Visual-Inertial Sensor Calibration with Next-Best-View and Next-Best-Trajectory Suggestion
Authors:
Christopher L. Choi,
Binbin Xu,
Stefan Leutenegger
Abstract:
Visual-Inertial (VI) sensors are popular in robotics, self-driving vehicles, and augmented and virtual reality applications. In order to use them for any computer vision or state-estimation task, a good calibration is essential. However, collecting informative calibration data in order to render the calibration parameters observable is not trivial for a non-expert. In this work, we introduce a nov…
▽ More
Visual-Inertial (VI) sensors are popular in robotics, self-driving vehicles, and augmented and virtual reality applications. In order to use them for any computer vision or state-estimation task, a good calibration is essential. However, collecting informative calibration data in order to render the calibration parameters observable is not trivial for a non-expert. In this work, we introduce a novel VI calibration pipeline that guides a non-expert with the use of a graphical user interface and information theory in collecting informative calibration data with Next-Best-View and Next-Best-Trajectory suggestions to calibrate the intrinsics, extrinsics, and temporal misalignment of a VI sensor. We show through experiments that our method is faster, more accurate, and more consistent than state-of-the-art alternatives. Specifically, we show how calibrations with our proposed method achieve higher accuracy estimation results when used by state-of-the-art VI Odometry as well as VI-SLAM approaches. The source code of our software can be found on: https://github.com/chutsu/yac.
△ Less
Submitted 25 September, 2023;
originally announced September 2023.
-
Universal Adversarial Directions
Authors:
Ching Lam Choi,
Farzan Farnia
Abstract:
Despite their great success in image recognition tasks, deep neural networks (DNNs) have been observed to be susceptible to universal adversarial perturbations (UAPs) which perturb all input samples with a single perturbation vector. However, UAPs often struggle in transferring across DNN architectures and lead to challenging optimization problems. In this work, we study the transferability of UAP…
▽ More
Despite their great success in image recognition tasks, deep neural networks (DNNs) have been observed to be susceptible to universal adversarial perturbations (UAPs) which perturb all input samples with a single perturbation vector. However, UAPs often struggle in transferring across DNN architectures and lead to challenging optimization problems. In this work, we study the transferability of UAPs by analyzing equilibrium in the universal adversarial example game between the classifier and UAP adversary players. We show that under mild assumptions the universal adversarial example game lacks a pure Nash equilibrium, indicating UAPs' suboptimal transferability across DNN classifiers. To address this issue, we propose Universal Adversarial Directions (UADs) which only fix a universal direction for adversarial perturbations and allow the perturbations' magnitude to be chosen freely across samples. We prove that the UAD adversarial example game can possess a Nash equilibrium with a pure UAD strategy, implying the potential transferability of UADs. We also connect the UAD optimization problem to the well-known principal component analysis (PCA) and develop an efficient PCA-based algorithm for optimizing UADs. We evaluate UADs over multiple benchmark image datasets. Our numerical results show the superior transferability of UADs over standard gradient-based UAPs.
△ Less
Submitted 16 April, 2023; v1 submitted 28 October, 2022;
originally announced October 2022.
-
Visual-Inertial Multi-Instance Dynamic SLAM with Object-level Relocalisation
Authors:
Yifei Ren,
Binbin Xu,
Christopher L. Choi,
Stefan Leutenegger
Abstract:
In this paper, we present a tightly-coupled visual-inertial object-level multi-instance dynamic SLAM system. Even in extremely dynamic scenes, it can robustly optimise for the camera pose, velocity, IMU biases and build a dense 3D reconstruction object-level map of the environment. Our system can robustly track and reconstruct the geometries of arbitrary objects, their semantics and motion by incr…
▽ More
In this paper, we present a tightly-coupled visual-inertial object-level multi-instance dynamic SLAM system. Even in extremely dynamic scenes, it can robustly optimise for the camera pose, velocity, IMU biases and build a dense 3D reconstruction object-level map of the environment. Our system can robustly track and reconstruct the geometries of arbitrary objects, their semantics and motion by incrementally fusing associated colour, depth, semantic, and foreground object probabilities into each object model thanks to its robust sensor and object tracking. In addition, when an object is lost or moved outside the camera field of view, our system can reliably recover its pose upon re-observation. We demonstrate the robustness and accuracy of our method by quantitatively and qualitatively testing it in real-world data sequences.
△ Less
Submitted 8 August, 2022;
originally announced August 2022.
-
Self-distillation with Batch Knowledge Ensembling Improves ImageNet Classification
Authors:
Yixiao Ge,
Xiao Zhang,
Ching Lam Choi,
Ka Chun Cheung,
Peipei Zhao,
Feng Zhu,
Xiaogang Wang,
Rui Zhao,
Hongsheng Li
Abstract:
The recent studies of knowledge distillation have discovered that ensembling the "dark knowledge" from multiple teachers or students contributes to creating better soft targets for training, but at the cost of significantly more computations and/or parameters. In this work, we present BAtch Knowledge Ensembling (BAKE) to produce refined soft targets for anchor images by propagating and ensembling…
▽ More
The recent studies of knowledge distillation have discovered that ensembling the "dark knowledge" from multiple teachers or students contributes to creating better soft targets for training, but at the cost of significantly more computations and/or parameters. In this work, we present BAtch Knowledge Ensembling (BAKE) to produce refined soft targets for anchor images by propagating and ensembling the knowledge of the other samples in the same mini-batch. Specifically, for each sample of interest, the propagation of knowledge is weighted in accordance with the inter-sample affinities, which are estimated on-the-fly with the current network. The propagated knowledge can then be ensembled to form a better soft target for distillation. In this way, our BAKE framework achieves online knowledge ensembling across multiple samples with only a single network. It requires minimal computational and memory overhead compared to existing knowledge ensembling methods. Extensive experiments demonstrate that the lightweight yet effective BAKE consistently boosts the classification performance of various architectures on multiple datasets, e.g., a significant +0.7% gain of Swin-T on ImageNet with only +1.5% computational overhead and zero additional parameters. BAKE does not only improve the vanilla baselines, but also surpasses the single-network state-of-the-arts on all the benchmarks.
△ Less
Submitted 20 November, 2021; v1 submitted 27 April, 2021;
originally announced April 2021.
-
DivCo: Diverse Conditional Image Synthesis via Contrastive Generative Adversarial Network
Authors:
Rui Liu,
Yixiao Ge,
Ching Lam Choi,
Xiaogang Wang,
Hongsheng Li
Abstract:
Conditional generative adversarial networks (cGANs) target at synthesizing diverse images given the input conditions and latent codes, but unfortunately, they usually suffer from the issue of mode collapse. To solve this issue, previous works mainly focused on encouraging the correlation between the latent codes and their generated images, while ignoring the relations between images generated from…
▽ More
Conditional generative adversarial networks (cGANs) target at synthesizing diverse images given the input conditions and latent codes, but unfortunately, they usually suffer from the issue of mode collapse. To solve this issue, previous works mainly focused on encouraging the correlation between the latent codes and their generated images, while ignoring the relations between images generated from various latent codes. The recent MSGAN tried to encourage the diversity of the generated image but only considers "negative" relations between the image pairs. In this paper, we propose a novel DivCo framework to properly constrain both "positive" and "negative" relations between the generated images specified in the latent space. To the best of our knowledge, this is the first attempt to use contrastive learning for diverse conditional image synthesis. A novel latent-augmented contrastive loss is introduced, which encourages images generated from adjacent latent codes to be similar and those generated from distinct latent codes to be dissimilar. The proposed latent-augmented contrastive loss is well compatible with various cGAN architectures. Extensive experiments demonstrate that the proposed DivCo can produce more diverse images than state-of-the-art methods without sacrificing visual quality in multiple unpaired and paired image generation tasks.
△ Less
Submitted 26 April, 2021; v1 submitted 14 March, 2021;
originally announced March 2021.
-
Encoderless Gimbal Calibration of Dynamic Multi-Camera Clusters
Authors:
Christopher L. Choi,
Jason Rebello,
Leonid Koppel,
Pranav Ganti,
Arun Das,
Steven L. Waslander
Abstract:
Dynamic Camera Clusters (DCCs) are multi-camera systems where one or more cameras are mounted on actuated mechanisms such as a gimbal. Existing methods for DCC calibration rely on joint angle measurements to resolve the time-varying transformation between the dynamic and static camera. This information is usually provided by motor encoders, however, joint angle measurements are not always readily…
▽ More
Dynamic Camera Clusters (DCCs) are multi-camera systems where one or more cameras are mounted on actuated mechanisms such as a gimbal. Existing methods for DCC calibration rely on joint angle measurements to resolve the time-varying transformation between the dynamic and static camera. This information is usually provided by motor encoders, however, joint angle measurements are not always readily available on off-the-shelf mechanisms. In this paper, we present an encoderless approach for DCC calibration which simultaneously estimates the kinematic parameters of the transformation chain as well as the unknown joint angles. We also demonstrate the integration of an encoderless gimbal mechanism with a state-of-the art VIO algorithm, and show the extensions required in order to perform simultaneous online estimation of the joint angles and vehicle localization state. The proposed calibration approach is validated both in simulation and on a physical DCC composed of a 2-DOF gimbal mounted on a UAV. Finally, we show the experimental results of the calibrated mechanism integrated into the OKVIS VIO package, and demonstrate successful online joint angle estimation while maintaining localization accuracy that is comparable to a standard static multi-camera configuration.
△ Less
Submitted 24 July, 2018;
originally announced July 2018.
-
Self-Assembly of Semiconducting Single-Walled Carbon Nanotubes into Dense, Aligned Rafts
Authors:
Justin Wu,
Liying Jiao,
Alexander Antaris,
Charina L. Choi,
Liming Xie,
Yingpeng Wu,
Shuo Diao,
Changxin Chen,
Yongsheng Chen,
Hongjie Dai
Abstract:
Single-walled carbon nanotubes are promising nanoelectronic materials but face long-standing challenges including production of pure semiconducting SWNTs and integration into ordered structures. Here, highly pure semiconducting single-walled carbon nanotubes are separated from bulk materials and self-assembled into densely aligned rafts driven by depletion attraction forces. Microscopy and spectro…
▽ More
Single-walled carbon nanotubes are promising nanoelectronic materials but face long-standing challenges including production of pure semiconducting SWNTs and integration into ordered structures. Here, highly pure semiconducting single-walled carbon nanotubes are separated from bulk materials and self-assembled into densely aligned rafts driven by depletion attraction forces. Microscopy and spectroscopy revealed a high degree of alignment and a high packing density of ~100 tubes/micron within SWNT rafts. Field-effect transistors made from aligned SWNT rafts afforded short channel (~150 nm long) devices comprised of tens of purely semiconducting SWNTs derived from chemical separation within a < 1 micron channel width, achieving unprecedented high on-currents (up to ~120 microamperes per device) with high on/off ratios. The average on-current was ~ 3-4 microamperes per tube. The results demonstrated densely aligned high quality semiconducting SWNTs for integration into high performance nanoelectronics.
△ Less
Submitted 12 July, 2013;
originally announced July 2013.
