-
SHARDeg: A Benchmark for Skeletal Human Action Recognition in Degraded Scenarios
Authors:
Simon Malzard,
Nitish Mital,
Richard Walters,
Victoria Nockles,
Raghuveer Rao,
Celso M. De Melo
Abstract:
Computer vision (CV) models for detection, prediction or classification tasks operate on video data-streams that are often degraded in the real world, due to deployment in real-time or on resource-constrained hardware. It is therefore critical that these models are robust to degraded data, but state of the art (SoTA) models are often insufficiently assessed with these real-world constraints in min…
▽ More
Computer vision (CV) models for detection, prediction or classification tasks operate on video data-streams that are often degraded in the real world, due to deployment in real-time or on resource-constrained hardware. It is therefore critical that these models are robust to degraded data, but state of the art (SoTA) models are often insufficiently assessed with these real-world constraints in mind. This is exemplified by Skeletal Human Action Recognition (SHAR), which is critical in many CV pipelines operating in real-time and at the edge, but robustness to degraded data has previously only been shallowly and inconsistently assessed. Here we address this issue for SHAR by providing an important first data degradation benchmark on the most detailed and largest 3D open dataset, NTU-RGB+D-120, and assess the robustness of five leading SHAR models to three forms of degradation that represent real-world issues. We demonstrate the need for this benchmark by showing that the form of degradation, which has not previously been considered, has a large impact on model accuracy; at the same effective frame rate, model accuracy can vary by >40% depending on degradation type. We also identify that temporal regularity of frames in degraded SHAR data is likely a major driver of differences in model performance, and harness this to improve performance of existing models by up to >40%, through employing a simple mitigation approach based on interpolation. Finally, we highlight how our benchmark has helped identify an important degradation-resistant SHAR model based in Rough Path Theory; the LogSigRNN SHAR model outperforms the SoTA DeGCN model in five out of six cases at low frame rates by an average accuracy of 6%, despite trailing the SoTA model by 11-12% on un-degraded data at high frame rates (30 FPS).
△ Less
Submitted 27 May, 2025; v1 submitted 23 May, 2025;
originally announced May 2025.
-
Improving Object Detection by Modifying Synthetic Data with Explainable AI
Authors:
Nitish Mital,
Simon Malzard,
Richard Walters,
Celso M. De Melo,
Raghuveer Rao,
Victoria Nockles
Abstract:
Limited real-world data severely impacts model performance in many computer vision domains, particularly for samples that are underrepresented in training. Synthetically generated images are a promising solution, but 1) it remains unclear how to design synthetic training data to optimally improve model performance (e.g, whether and where to introduce more realism or more abstraction) and 2) the do…
▽ More
Limited real-world data severely impacts model performance in many computer vision domains, particularly for samples that are underrepresented in training. Synthetically generated images are a promising solution, but 1) it remains unclear how to design synthetic training data to optimally improve model performance (e.g, whether and where to introduce more realism or more abstraction) and 2) the domain expertise, time and effort required from human operators for this design and optimisation process represents a major practical challenge. Here we propose a novel conceptual approach to improve the efficiency of designing synthetic images, by using robust Explainable AI (XAI) techniques to guide a human-in-the-loop process of modifying 3D mesh models used to generate these images. Importantly, this framework allows both modifications that increase and decrease realism in synthetic data, which can both improve model performance. We illustrate this concept using a real-world example where data are sparse; detection of vehicles in infrared imagery. We fine-tune an initial YOLOv8 model on the ATR DSIAC infrared dataset and synthetic images generated from 3D mesh models in the Unity gaming engine, and then use XAI saliency maps to guide modification of our Unity models. We show that synthetic data can improve detection of vehicles in orientations unseen in training by 4.6% (to mAP50 = 94.6%). We further improve performance by an additional 1.5% (to 96.1%) through our new XAI-guided approach, which reduces misclassifications through both increasing and decreasing the realism of different parts of the synthetic data. Our proof-of-concept results pave the way for fine, XAI-controlled curation of synthetic datasets tailored to improve object detection performance, whilst simultaneously reducing the burden on human operators in designing and optimising these datasets.
△ Less
Submitted 3 April, 2025; v1 submitted 2 December, 2024;
originally announced December 2024.
-
2DSig-Detect: a semi-supervised framework for anomaly detection on image data using 2D-signatures
Authors:
Xinheng Xie,
Kureha Yamaguchi,
Margaux Leblanc,
Simon Malzard,
Varun Chhabra,
Victoria Nockles,
Yue Wu
Abstract:
The rapid advancement of machine learning technologies raises questions about the security of machine learning models, with respect to both training-time (poisoning) and test-time (evasion, impersonation, and inversion) attacks. Models performing image-related tasks, e.g. detection, and classification, are vulnerable to adversarial attacks that can degrade their performance and produce undesirable…
▽ More
The rapid advancement of machine learning technologies raises questions about the security of machine learning models, with respect to both training-time (poisoning) and test-time (evasion, impersonation, and inversion) attacks. Models performing image-related tasks, e.g. detection, and classification, are vulnerable to adversarial attacks that can degrade their performance and produce undesirable outcomes. This paper introduces a novel technique for anomaly detection in images called 2DSig-Detect, which uses a 2D-signature-embedded semi-supervised framework rooted in rough path theory. We demonstrate our method in adversarial settings for training-time and test-time attacks, and benchmark our framework against other state of the art methods. Using 2DSig-Detect for anomaly detection, we show both superior performance and a reduction in the computation time to detect the presence of adversarial perturbations in images.
△ Less
Submitted 20 March, 2025; v1 submitted 8 September, 2024;
originally announced September 2024.
-
Husimi dynamics generated by non-Hermitian Hamiltonians
Authors:
Katherine Holmes,
Wasim Rehman,
Simon Malzard,
Eva-Maria Graefe
Abstract:
The dynamics generated by non-Hermitian Hamiltonians are often less intuitive than those of conventional Hermitian systems. Even for models as simple as a complexified harmonic oscillator, the dynamics for generic initial states shows surprising features. Here we analyse the dynamics of the Husimi distribution in a semiclassical limit, illuminating the foundations of the full quantum evolution. Th…
▽ More
The dynamics generated by non-Hermitian Hamiltonians are often less intuitive than those of conventional Hermitian systems. Even for models as simple as a complexified harmonic oscillator, the dynamics for generic initial states shows surprising features. Here we analyse the dynamics of the Husimi distribution in a semiclassical limit, illuminating the foundations of the full quantum evolution. The classical Husimi evolution is composed of two factors, (i) the initial Husimi distribution evaluated along phase-space trajectories, and (ii) the final value of the norm corresponding to each phase-space point. Both factors conspire to lead to intriguing dynamical behaviours. We demonstrate how the full quantum dynamics unfolds on top of the classical Husimi dynamics for two instructive examples.
△ Less
Submitted 11 April, 2023; v1 submitted 7 December, 2022;
originally announced December 2022.
-
Semiclassical Husimi distributions for non-Hermitian quantum systems
Authors:
Joseph Hall,
Simon Malzard,
Eva-Maria Graefe
Abstract:
We construct a semiclassical phase-space density of Schur vectors in non-Hermitian quantum systems. Each Schur vector is associated to a single Planck cell. The Schur states are organised according to a classical norm landscape on phase space - a classical manifestation of the lifetimes which are characteristic of non-Hermitian systems. To demonstrate the generality of this construction we apply i…
▽ More
We construct a semiclassical phase-space density of Schur vectors in non-Hermitian quantum systems. Each Schur vector is associated to a single Planck cell. The Schur states are organised according to a classical norm landscape on phase space - a classical manifestation of the lifetimes which are characteristic of non-Hermitian systems. To demonstrate the generality of this construction we apply it to a highly non-trivial example, a PT-symmetric kicked rotor in the regimes of mixed and chaotic classical dynamics.
△ Less
Submitted 9 May, 2023; v1 submitted 28 November, 2022;
originally announced November 2022.
-
Landau-Zener transitions through a pair of higher order exceptional points
Authors:
Rishindra Melanathuru,
Simon Malzard,
Eva-Maria Graefe
Abstract:
Non-Hermitian quantum systems with explicit time dependence are of ever-increasing importance. There are only a handful of models that have been analytically studied in this context. Here, a PT-symmetric non-Hermitian $N$-level Landau-Zener type problem with two exceptional points of $N$th order is introduced. The system is Hermitian for asymptotically large times, far away from the exceptional po…
▽ More
Non-Hermitian quantum systems with explicit time dependence are of ever-increasing importance. There are only a handful of models that have been analytically studied in this context. Here, a PT-symmetric non-Hermitian $N$-level Landau-Zener type problem with two exceptional points of $N$th order is introduced. The system is Hermitian for asymptotically large times, far away from the exceptional points, and has purely imaginary eigenvalues between the exceptional points. The full Landau-Zener transition probabilities are derived, and found to show a characteristic binomial behaviour. In the adiabatic limit the final populations are given by the ratios of binomial coefficients. It is demonstrated how this behaviour can be understood on the basis of adiabatic analysis, despite the breakdown of adiabaticity that is often associated with non-Hermitian systems.
△ Less
Submitted 4 July, 2022; v1 submitted 19 May, 2022;
originally announced May 2022.
-
Nonhermitian defect states from lifetime differences
Authors:
Martà Bosch,
Simon Malzard,
Martina Hentschel,
Henning Schomerus
Abstract:
Nonhermitian systems provide new avenues to create topological defect states. An unresolved general question is how much the formation of these states depends on asymmetric backscattering, be it nonreciprocal as in the nonhermitian skin effect or reciprocal as encountered between the internal states of asymmetric microresonators. Here, we demonstrate in a concrete, practically accessible setting o…
▽ More
Nonhermitian systems provide new avenues to create topological defect states. An unresolved general question is how much the formation of these states depends on asymmetric backscattering, be it nonreciprocal as in the nonhermitian skin effect or reciprocal as encountered between the internal states of asymmetric microresonators. Here, we demonstrate in a concrete, practically accessible setting of a lossy coupled-resonator optical waveguide that nonhermitian defect states can exist in open optical systems due to lifetime differences, without the need for asymmetric backscattering within or between the individual resonators. We apply our findings to a finite system of coupled circular resonators perturbed by nanoparticles, following the concept of creating an interface by inverting the position of the nanoparticles in half of the chain. We compare a coupled-mode tight-binding approximation to full-wave numerical simulations, showing that spectrally isolated defect states can indeed be implemented in this simple nonhermitian photonic device.
△ Less
Submitted 6 November, 2019; v1 submitted 23 August, 2019;
originally announced August 2019.
-
Nonlinear mode competition and symmetry-protected power oscillations in topological lasers
Authors:
Simon Malzard,
Henning Schomerus
Abstract:
Topological photonics started out as a pursuit to engineer systems that mimic fermionic single-particle Hamiltonians with symmetry-protected modes, whose number can only change in spectral phase transitions such as band inversions. The paradigm of topological lasing, realized in three recent experiments, offers entirely new interpretations of these states, as they can be selectively amplified by d…
▽ More
Topological photonics started out as a pursuit to engineer systems that mimic fermionic single-particle Hamiltonians with symmetry-protected modes, whose number can only change in spectral phase transitions such as band inversions. The paradigm of topological lasing, realized in three recent experiments, offers entirely new interpretations of these states, as they can be selectively amplified by distributed gain and loss. A key question is whether such topological mode selection persists when one accounts for the nonlinearities that stabilize such systems at their working point. Here we show that topological defect lasers can indeed stably operate in genuinely topological states. These comprise direct analogues of zero modes from the linear setting, as well as a novel class of states displaying symmetry-protected power oscillations, which appear in a spectral phase transition when the gain is increased. These effects show a remarkable practical resilience against imperfections, even if these break the underlying symmetries, and pave the way to harness the power of topological protection in nonlinear quantum devices.
△ Less
Submitted 4 June, 2018;
originally announced June 2018.
-
Bulk and edge-state arcs in non-hermitian coupled-resonator arrays
Authors:
Simon Malzard,
Henning Schomerus
Abstract:
We describe the formation of bulk and edge arcs in the dispersion relation of two-dimensional coupled-resonator arrays that are topologically trivial in the hermitian limit. Each resonator provides two asymmetrically coupled internal modes, as realized in noncircular open geometries, which enables the system to exhibit non-hermitian physics. Neighboring resonators are coupled chirally to induce no…
▽ More
We describe the formation of bulk and edge arcs in the dispersion relation of two-dimensional coupled-resonator arrays that are topologically trivial in the hermitian limit. Each resonator provides two asymmetrically coupled internal modes, as realized in noncircular open geometries, which enables the system to exhibit non-hermitian physics. Neighboring resonators are coupled chirally to induce non-hermitian symmetries. The bulk dispersion displays Fermi arcs connecting spectral singularities known as exceptional points, and can be tuned to display purely real and imaginary branches. At an interface between resonators of different shape, one-dimensional edge states form that spectrally align along complex arcs connecting different parts of the bulk bands. We also describe conditions under which the edge-state arcs are free standing. These features can be controlled via anisotropy in the resonator couplings.
△ Less
Submitted 21 May, 2018;
originally announced May 2018.
-
Topological Hybrid Silicon Microlasers
Authors:
Han Zhao,
Pei Miao,
Mohammad H. Teimourpour,
Simon Malzard,
Ramy El-Ganainy,
Henning Schomerus,
Liang Feng
Abstract:
Topological photonics provides a robust framework for strategically controlling light confinement and propagation dynamics. By exploiting the marriage between this notion and symmetry-constrained mode competition in an active setting, we experimentally demonstrate, for the first time, a hybrid silicon microlaser structure supporting a topologically protected zero-mode lasing. This mode is distribu…
▽ More
Topological photonics provides a robust framework for strategically controlling light confinement and propagation dynamics. By exploiting the marriage between this notion and symmetry-constrained mode competition in an active setting, we experimentally demonstrate, for the first time, a hybrid silicon microlaser structure supporting a topologically protected zero-mode lasing. This mode is distributed over an integrated array of optical microrings, each supporting multiple modes that compete for gain. However, the mode competition is diminished in favor of a pre-defined topological state, which reflects the charge conjugation symmetry induced by the gain profile. Robust single-mode operation in the desired state prevails even with intentionally introduced perturbations. The demonstrated microlaser is hybrid-implemented on a silicon-on-insulator substrate, thereby readily suitable for integrated silicon photonics with potential applications in optical communication and computing.
△ Less
Submitted 8 September, 2017;
originally announced September 2017.
-
Topological dynamics and excitations in lasers and condensates with saturable gain or loss
Authors:
Simon Malzard,
Emiliano Cancellieri,
Henning Schomerus
Abstract:
We classify symmetry-protected and symmetry-breaking dynamical solutions for nonlinear saturable bosonic systems that display a non-hermitian charge-conjugation symmetry, as realized in a series of recent groundbreaking experiments with lasers and exciton polaritons. In particular, we show that these systems support stable symmetry-protected modes that mirror the concept of zero-modes in topologic…
▽ More
We classify symmetry-protected and symmetry-breaking dynamical solutions for nonlinear saturable bosonic systems that display a non-hermitian charge-conjugation symmetry, as realized in a series of recent groundbreaking experiments with lasers and exciton polaritons. In particular, we show that these systems support stable symmetry-protected modes that mirror the concept of zero-modes in topological quantum systems, as well as symmetry-protected power-oscillations with no counterpart in the linear case. In analogy to topological phases in linear systems, the number and nature of symmetry-protected solutions can change. The spectral degeneracies signalling phase transitions in linear counterparts extend to bifurcations in the nonlinear context. As bifurcations relate to qualitative changes in the linear stability against changes of the initial conditions, the symmetry-protected solutions and phase transitions can also be characterized by topological excitations, which set them apart from symmetry-breaking solutions. The stipulated symmetry appears naturally when one introduces nonlinear gain or loss into spectrally symmetric bosonic systems, as we illustrate for one-dimensional topological laser arrays with saturable gain and two-dimensional flat-band polariton condensates with density-dependent loss.
△ Less
Submitted 17 August, 2018; v1 submitted 19 May, 2017;
originally announced May 2017.
-
Topologically protected defect states in open photonic systems with non-hermitian charge-conjugation and parity-time symmetry
Authors:
Simon Malzard,
Charles Poli,
Henning Schomerus
Abstract:
We show that topologically protected defect states can exist in open (leaky or lossy) systems even when these systems are topologically trivial in the closed limit. The states appear from within the continuum, thus in absence of a band gap, and are generated via exceptional points (a spectral transition that occurs in open wave and quantum systems with a generalized time-reversal symmetry), or via…
▽ More
We show that topologically protected defect states can exist in open (leaky or lossy) systems even when these systems are topologically trivial in the closed limit. The states appear from within the continuum, thus in absence of a band gap, and are generated via exceptional points (a spectral transition that occurs in open wave and quantum systems with a generalized time-reversal symmetry), or via a degeneracy induced by charge-conjugation-symmetry (which is related to the pole transition of Majorana zero modes). We demonstrate these findings for a leaking passive coupled-resonator optical waveguide with asymmmetric internal scattering, where the required symmetries (non-hermitian versions of time-reversal symmetry, chirality and charge-conjugation) emerge dynamically.
△ Less
Submitted 11 November, 2015; v1 submitted 17 August, 2015;
originally announced August 2015.
