-
An Alternative to FLOPS Regularization to Effectively Productionize SPLADE-Doc
Authors:
Aldo Porco,
Dhruv Mehra,
Igor Malioutov,
Karthik Radhakrishnan,
Moniba Keymanesh,
Daniel Preoţiuc-Pietro,
Sean MacAvaney,
Pengxiang Cheng
Abstract:
Learned Sparse Retrieval (LSR) models encode text as weighted term vectors, which need to be sparse to leverage inverted index structures during retrieval. SPLADE, the most popular LSR model, uses FLOPS regularization to encourage vector sparsity during training. However, FLOPS regularization does not ensure sparsity among terms - only within a given query or document. Terms with very high Documen…
▽ More
Learned Sparse Retrieval (LSR) models encode text as weighted term vectors, which need to be sparse to leverage inverted index structures during retrieval. SPLADE, the most popular LSR model, uses FLOPS regularization to encourage vector sparsity during training. However, FLOPS regularization does not ensure sparsity among terms - only within a given query or document. Terms with very high Document Frequencies (DFs) substantially increase latency in production retrieval engines, such as Apache Solr, due to their lengthy posting lists. To address the issue of high DFs, we present a new variant of FLOPS regularization: DF-FLOPS. This new regularization technique penalizes the usage of high-DF terms, thereby shortening posting lists and reducing retrieval latency. Unlike other inference-time sparsification methods, such as stopword removal, DF-FLOPS regularization allows for the selective inclusion of high-frequency terms in cases where the terms are truly salient. We find that DF-FLOPS successfully reduces the prevalence of high-DF terms and lowers retrieval latency (around 10x faster) in a production-grade engine while maintaining effectiveness both in-domain (only a 2.2-point drop in MRR@10) and cross-domain (improved performance in 12 out of 13 tasks on which we tested). With retrieval latencies on par with BM25, this work provides an important step towards making LSR practical for deployment in production-grade search engines.
△ Less
Submitted 20 May, 2025;
originally announced May 2025.
-
Morphology-preserving Autoregressive 3D Generative Modelling of the Brain
Authors:
Petru-Daniel Tudosiu,
Walter Hugo Lopez Pinaya,
Mark S. Graham,
Pedro Borges,
Virginia Fernandez,
Dai Yang,
Jeremy Appleyard,
Guido Novati,
Disha Mehra,
Mike Vella,
Parashkev Nachev,
Sebastien Ourselin,
Jorge Cardoso
Abstract:
Human anatomy, morphology, and associated diseases can be studied using medical imaging data. However, access to medical imaging data is restricted by governance and privacy concerns, data ownership, and the cost of acquisition, thus limiting our ability to understand the human body. A possible solution to this issue is the creation of a model able to learn and then generate synthetic images of th…
▽ More
Human anatomy, morphology, and associated diseases can be studied using medical imaging data. However, access to medical imaging data is restricted by governance and privacy concerns, data ownership, and the cost of acquisition, thus limiting our ability to understand the human body. A possible solution to this issue is the creation of a model able to learn and then generate synthetic images of the human body conditioned on specific characteristics of relevance (e.g., age, sex, and disease status). Deep generative models, in the form of neural networks, have been recently used to create synthetic 2D images of natural scenes. Still, the ability to produce high-resolution 3D volumetric imaging data with correct anatomical morphology has been hampered by data scarcity and algorithmic and computational limitations. This work proposes a generative model that can be scaled to produce anatomically correct, high-resolution, and realistic images of the human brain, with the necessary quality to allow further downstream analyses. The ability to generate a potentially unlimited amount of data not only enables large-scale studies of human anatomy and pathology without jeopardizing patient privacy, but also significantly advances research in the field of anomaly detection, modality synthesis, learning under limited data, and fair and ethical AI. Code and trained models are available at: https://github.com/AmigoLab/SynthAnatomy.
△ Less
Submitted 7 September, 2022;
originally announced September 2022.
-
PS$^2$F: Polarized Spiral Point Spread Function for Single-Shot 3D Sensing
Authors:
Bhargav Ghanekar,
Vishwanath Saragadam,
Dushyant Mehra,
Anna-Karin Gustavsson,
Aswin Sankaranarayanan,
Ashok Veeraraghavan
Abstract:
We propose a compact snapshot monocular depth estimation technique that relies on an engineered point spread function (PSF). Traditional approaches used in microscopic super-resolution imaging such as the Double-Helix PSF (DHPSF) are ill-suited for scenes that are more complex than a sparse set of point light sources. We show, using the Cramér-Rao lower bound, that separating the two lobes of the…
▽ More
We propose a compact snapshot monocular depth estimation technique that relies on an engineered point spread function (PSF). Traditional approaches used in microscopic super-resolution imaging such as the Double-Helix PSF (DHPSF) are ill-suited for scenes that are more complex than a sparse set of point light sources. We show, using the Cramér-Rao lower bound, that separating the two lobes of the DHPSF and thereby capturing two separate images leads to a dramatic increase in depth accuracy. A special property of the phase mask used for generating the DHPSF is that a separation of the phase mask into two halves leads to a spatial separation of the two lobes. We leverage this property to build a compact polarization-based optical setup, where we place two orthogonal linear polarizers on each half of the DHPSF phase mask and then capture the resulting image with a polarization-sensitive camera. Results from simulations and a lab prototype demonstrate that our technique achieves up to $50\%$ lower depth error compared to state-of-the-art designs including the DHPSF and the Tetrapod PSF, with little to no loss in spatial resolution.
△ Less
Submitted 4 August, 2022; v1 submitted 2 July, 2022;
originally announced July 2022.
-
Cyclostationary Spectrum Sensing in Cognitive Radios Using FRESH Filters
Authors:
Hemant Saggar,
D. K. Mehra
Abstract:
This paper deals with spectrum sensing in Cognitive Radios to enable unlicensed secondary users to opportunistically access a licensed band. The ability to detect the presence of a primary user at a low signal to noise ratio (SNR) is a challenging prerequisite to spectrum sensing and earlier proposed techniques like energy detection and cyclostationary detection have only been partially successful…
▽ More
This paper deals with spectrum sensing in Cognitive Radios to enable unlicensed secondary users to opportunistically access a licensed band. The ability to detect the presence of a primary user at a low signal to noise ratio (SNR) is a challenging prerequisite to spectrum sensing and earlier proposed techniques like energy detection and cyclostationary detection have only been partially successful. This paper proposes the use of FRESH (FREquency SHift) filters [1] to enable spectrum sensing at low SNR by optimally estimating a cyclostationary signal using its spectral coherence properties. We establish the mean square error convergence of the adaptive FRESH filter through simulation. Subsequently, we formulate a cyclostationarity based binary hypothesis test on the filtered signal and observe the resultant detection performance. Simulation results show that the proposed approach performs better than energy detection and cyclostationary detection techniques for spectrum sensing.
△ Less
Submitted 6 December, 2013;
originally announced December 2013.
